Anthropic backs a proposed AI safety bill that would let the government halt new AI models if they could cause at least $1 billion in damages or 50 deaths.
The bill envisions an independent third party evaluating AI labs and their models, with payments allowed only if independence remains intact.
Only the Commerce Department could order a pause, while other agencies would not have that power, according to the proposal.
The proposal follows rising concerns about AI risks and efforts in Congress to regulate powerful AI systems.
Quick read · 1 min
Anthropic is backing a proposed AI safety bill that would let the government pause a new AI release if it could cause $1 billion in damages or 50 deaths. An independent third party would audit the lab and its models, and only the Commerce Department could order a pause after a 72-hour report and a legal review.
For readers, this could mean longer waits for powerful AI features or different safeguards on what tech companies can release. The specifics, how risk is calculated and what happens to ongoing research, will matter as the bill moves through Congress.
What happens next is up to lawmakers, regulators, and the labs themselves as they weigh safety, innovation, and public trust.
The surprise here isn’t the idea of AI safety checks. It’s the thresholds Anthropic is proposing to trigger a government pause on releasing a new AI model: if there’s a credible risk of causing $1 billion in damages or harming 50 people, a halt could be ordered. The control would rest with the Commerce Department, not a string of agencies, and an independent third party would periodically evaluate the lab and its models.
Anthropic has been pushing for a framework to slow development when risks look real, especially after recent high-profile AI incidents and ongoing debates in Washington about how to keep powerful AI systems in check. The company says these are sensible policies that Congress should consider. The proposed mechanism would require the evaluator to report within 72 hours if it believes the risk threshold is crossed, and then the department would bring all the evidence before a federal judge in the District of Columbia to decide whether to stop the release.
It’s worth noting that this is not a finalized law. Politico reported on the bill and its details, including the role of the independent evaluator and the narrow authority given to the Commerce Department. And as with many safety proposals, the plan raises questions about how to measure risk, who gets to define “damages,” and what happens to ongoing research and commercial development in the meantime.
01
What exactly would trigger a pause?
Under the proposal, the trigger is the potential for a release to cause either massive economic damage or lethal harm, specifically, $1 billion in damages or more than 50 deaths. If risk sensors flag those possibilities, the evaluator would report to the Commerce Department within 72 hours, setting off a formal review process that could end with a court order to stop the release.
02
Who would check the AI models and why does independence matter?
An independent third party would periodically audit the lab and its AI models. The idea is to avoid political interference or corporate influence shaping the safety verdict. The evaluator could be paid for its work, but only if the payment doesn’t undermine independence.
03
What would this mean for you and your devices?
For everyday users, the practical question is how often such a pause would come into play and what it would mean for products you rely on. If a model is paused, rollout timelines could slip, and features that depend on advanced AI capabilities could stall. On the flip side, proponents argue that clear guardrails help reduce the chance of a major mishap that could affect millions of users at once.
04
What’s next for AI safety rules in the U.S.?
Right now, this is one proposal among several that policymakers are weighing. Congress has shown interest in kill-switch-like powers and other safety requirements, but lifting a model off the market is a heavy move with wide implications for innovation, competition, and consumer trust. As debates continue, observers will watch how this idea balances risk management with the need for AI to advance responsibly.
If you’re curious about how AI safety rules could affect products you use, keep an eye on updates from lawmakers and tech companies. For now, the best practical step is to stay informed about upcoming hearings and proposed bills, and remember that safety frameworks often evolve before they become law.
06
Quick answers
What would trigger a pause under this bill?
Potential for $1 billion in damages or 50 deaths would trigger a formal pause process, with a possible court decision to halt release.
Who would enforce the pause?
The Commerce Department would have the power to stop a release, based on findings from an independent evaluator.
Is this law in effect now?
No. It’s a proposed framework being discussed in Congress and among AI labs, not a enacted law yet.
MIT Technology Review highlights a growing menu of biotech options for grieving pet owners, from longevity drugs to cloning and cryonics, with mixed ethics and uncertain futures.
