Anthropic Proposes AI Safety Bill That Could Block High-Risk Model Releases

Anthropic has proposed draft legislation that would allow the release of advanced AI models to be halted when they pose a foreseeable and imminent risk of catastrophic harm, introducing a potential legal mechanism for regulating the most dangerous AI systems in the United States.

The proposal outlines circumstances in which a model could face restrictions if its creation or release risks killing or seriously injuring more than 50 people, or causing more than $1 billion in property damage. The specified threats include biological weapons, AI systems evading their developers’ controls and cyberattacks.

The draft would establish a formal assessment process involving independent evaluators and the US Department of Commerce. It remains a proposal under discussion with lawmakers, not an enacted federal law.

How the Proposed AI Safety Bill Would Work

Under the draft framework, independent evaluators registered with the Department of Commerce would periodically assess frontier AI developers and their models for specified catastrophic risks. AI companies could pay for these assessments, provided payment was not contingent on the findings.

If evaluators identified an imminent risk covered by the proposal, they would have 72 hours to notify the Department of Commerce after completing their assessment. The department could then ask a federal judge in Washington, DC, to halt the model’s release or use.

The proposed legislation would not give federal agencies unrestricted authority to suspend AI development or deployment. Instead, it would require the government to seek judicial intervention under the conditions specified in the draft. This structure is intended to establish an enforceable safety mechanism while limiting when and how authorities could intervene.

Why Anthropic Wants Legal Oversight

The proposal adds to an ongoing debate over whether advanced AI safety should depend primarily on voluntary commitments from developers or be backed by enforceable legislation. As AI models become more capable, policymakers are examining risks that could extend beyond misinformation and conventional software vulnerabilities to severe cyberattacks and the potential misuse of biological capabilities.

Anthropic has previously advocated stronger safeguards for advanced AI systems. Its proposed legislation reflects an effort to establish a legal process for responding to narrowly defined catastrophic risks rather than relying exclusively on companies to decide when a model is too dangerous to release.

The proposal also comes amid wider disagreement within the technology industry over the need for mandatory AI regulation. Companies differ on how much government oversight is appropriate, which risks warrant intervention and who should have the authority to make those decisions.

What Happens Next

The draft has been presented to US lawmakers, but its eventual legislative prospects remain uncertain. Further debate will likely focus on how catastrophic risk is defined, how independent evaluators would be selected and supervised, and whether the proposed thresholds provide sufficient protection without unnecessarily restricting AI development.

If adopted, the framework could create a legal route to delay or block the release of specific frontier AI models under defined circumstances. For now, its significance lies in advancing the debate over how governments should respond when the potential risks of an AI system extend beyond the safeguards its developer can reliably maintain.