US Lawmaker Proposes Bill to Ban AI Recursive Self-Improvement, Mandate Human Control

California Democratic Representative Ro Khanna is reportedly preparing to introduce an artificial intelligence bill that would impose strict liability rules and ban "recursive" self-improving AI until the government establishes safety standards.

New Bill Aims to Address Potential Threats of Advanced AI

Khanna told CNBC on Monday that there is a risk of "civilizational extinction," as well as "uncontrolled safety risks and abuse risks," both of which must be taken seriously.

The bill, named the "Human Control over AI Act," whose summary was seen by the outlet, would prohibit AI models from self-upgrading or altering their own safety and shutdown settings. This restriction would remain in effect until federal rulemaking is completed and government agencies approve the technology.

Additionally, the bill proposes creating a new federal agency to specifically regulate advanced AI models developed by companies such as OpenAI, Anthropic, Alphabet Inc's Google DeepMind, and Space Exploration Technologies Corp's SpaceXAI.

This new agency would be responsible for establishing safety rules, licensing processes for training and releasing models, audits of frontier AI models, and strict safety rules ensuring human oversight and continuous testing.

Third-party reviewers would also be embedded directly in each advanced AI laboratory to provide immediate reports to the agency. The agency would also develop guidelines for isolated offline testing zones, emergency kill switches, and safeguards to confine models within laboratories to prevent their online proliferation. Furthermore, the agency would oversee AI chips used by top developers, implementing methods to regulate and monitor their applications.

AI Safety Debate Heats Up

Earlier this month, Anthropic researcher Evan Hubinger estimated a greater than 10% probability that AI could wipe out humanity within a decade, amid a broader industry safety debate. Last week, Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman warned the UN Security Council about the potential risks posed by AI, calling for strengthened international cooperation to manage increasingly powerful AI systems.

Meanwhile, on Sunday, "The Big Short" investor Michael Burry warned on social media platform X about the consequences of increasingly recursive AI-generated content. He argued that the growing use of AI-generated training data could amplify errors while exposing deeper limitations of large language models on the path to artificial general intelligence.

Last week, Vermont Independent Senator Bernie Sanders urged President Donald Trump and Chinese President Xi Jinping to negotiate an international agreement to pause advanced AI development and ban superintelligence.

See more:

Top Growth Stocks