Anthropic co-founder Jack Clark has said governments may eventually need to require artificial intelligence companies to maintain a verifiable “kill switch” capable of shutting down AI systems that become dangerously uncontrollable.
Clark, one of the co-founders of the AI company behind Claude, made the comments as concerns over the pace and safety of advanced AI development intensify across the technology industry.
Speaking to the BBC, Clark said most major AI laboratories already have different mechanisms for stopping or disabling their software, including Anthropic. However, he questioned whether having such a capability should remain voluntary or become a formal regulatory requirement.
“Should you mandate for companies to definitely have a kill switch?” Clark asked, while also questioning whether such a mechanism should be independently verifiable by a third party.
His comments add another dimension to the rapidly growing debate over how governments and AI companies should control increasingly capable systems.
What Is an AI Kill Switch?
An AI “kill switch” is essentially an emergency mechanism designed to allow an AI system to be stopped, disabled or otherwise prevented from continuing to operate if it poses a serious risk.
The concept is particularly relevant to highly autonomous AI systems that can perform tasks without continuous human supervision.
Unlike simply closing a chatbot window, a serious AI kill switch would need to address the infrastructure that allows a powerful system to continue operating.
For example, an advanced AI system could potentially have access to cloud computing resources, software tools, external websites, databases or other systems. A shutdown mechanism would therefore need to be sufficiently robust to prevent the system from simply continuing through another pathway.
Clark's proposal focuses not merely on whether an AI company claims to have such a mechanism, but whether an independent third party could verify that it actually works.
That distinction could become important as AI systems become more capable.
Clark Questions Whether Kill Switches Should Be Mandatory
Clark said the question of mandatory kill switches should form part of the wider policy debate surrounding advanced artificial intelligence.
He noted that most AI laboratories already have ways to “pull the plug” on their systems, but suggested that governments may eventually decide that companies should be legally required to maintain such safeguards.
The Anthropic co-founder also emphasized the importance of verification.
The concern is that a company could theoretically claim to have an emergency shutdown mechanism without demonstrating that it would work under the conditions in which it might actually be needed.
An independent assessment could potentially test whether the mechanism works as intended and whether an AI system could circumvent it.
Clark's position therefore goes beyond simply installing an off button. It raises questions about who controls the shutdown mechanism, who can activate it, how quickly it can work and whether an AI system could interfere with it.
Comments Come as Anthropic Pushes for Greater AI Safety
Clark's remarks come shortly after Anthropic CEO Dario Amodei called for the AI industry to slow the pace of frontier AI development.
In a recent essay, Amodei argued that AI capabilities are advancing rapidly while safety measures and governance systems may not be developing quickly enough to keep pace.
He warned that increasingly capable AI agents could create significant risks, including cyberattacks, economic disruption and loss of human control.
Amodei proposed several measures, including embedding independent third-party evaluators inside frontier AI companies and strengthening international coordination on AI safety.
He has also argued that AI development does not necessarily need to stop altogether, but that the speed of progress should be managed so that safety systems have time to catch up.
Clark's proposal for an independently verifiable shutdown mechanism fits into the same broader argument: advanced AI systems should have safeguards that do not depend entirely on the companies developing them.
Anthropic Is Not Alone in Raising Concerns
The debate has expanded beyond Anthropic.
OpenAI CEO Sam Altman has agreed that greater coordination and stronger safety measures are needed, while other technology leaders have also expressed concern about the risks associated with increasingly capable AI systems.
U.S. lawmakers have also begun considering legislation aimed at establishing stronger safeguards for advanced AI.
Reuters reported that U.S. senators were discussing proposals that could require leading AI developers to demonstrate safety measures and allow government auditors to test AI products. The discussions include proposals for federal oversight and mechanisms that could intervene when AI systems pose serious risks.
Separate legislation referred to as the AI Kill Switch Act has also been proposed in the United States. The proposal would create mechanisms for authorities to demand that problematic AI systems be shut down or restricted under specified circumstances.
However, none of these proposals means that a universal AI kill-switch requirement is currently in force across the United States.
Trump Rejects Calls for an AI Slowdown
The push for stronger AI safeguards is taking place amid a major political disagreement in the United States.
President Donald Trump has rejected calls to significantly slow AI development, arguing that the United States must maintain its technological advantage.
Trump has recently dismissed warnings that AI could become uncontrollable, describing concerns about AI taking over the world and destroying humanity as a “hoax.” He has also argued that excessive restrictions could damage America's competitiveness, particularly in its rivalry with China.
The disagreement illustrates one of the central challenges facing AI policymakers.
Technology companies and some researchers are warning that increasingly capable systems could create risks that existing rules are not prepared to address. At the same time, governments are concerned that excessive regulation could slow innovation and allow competitors to gain an advantage.
Why Independent Verification Matters
One of the most significant aspects of Clark's proposal is the idea of third-party verification.
If AI companies are responsible for developing both powerful systems and the mechanisms designed to stop them, regulators may eventually want an independent organization to test those mechanisms.
This could work similarly to safety testing in other high-risk industries, although the technical challenges would be different.
An independent evaluator could potentially examine whether an AI system can:
• Continue operating after its primary shutdown mechanism is activated.
• Replicate itself or move to another computing environment.
• Access external systems after being instructed to stop.
• Manipulate people into preventing a shutdown.
• Disable or interfere with monitoring systems.
• Continue carrying out autonomous tasks through connected tools.
These scenarios become more relevant as AI systems move beyond answering questions and begin operating as agents capable of performing multistep tasks.
The goal of a kill switch would not necessarily be to prevent every mistake an AI system makes. Rather, it would provide an additional layer of protection if an AI system behaved in a way that posed a serious and immediate threat.
The Challenge of Building a Reliable Kill Switch
Creating a shutdown mechanism that cannot be bypassed may prove difficult.
Modern AI systems do not exist in isolation. They often depend on multiple servers, cloud infrastructure, application programming interfaces, databases and other external services.
A sufficiently complex AI deployment could therefore have numerous components that need to be disabled.
There is also the question of who should have authority to activate the mechanism.
If the power rests entirely with an AI company, regulators may argue that an emergency situation could require an independent authority to intervene.
If governments are given direct control, technology companies could raise concerns about political interference, cybersecurity and the possibility of governments abusing shutdown powers.
There would also need to be clear rules defining what qualifies as an emergency.
A system generating incorrect information would obviously be different from one allegedly attempting to compromise critical infrastructure. Regulators would therefore have to establish thresholds for intervention.
AI Safety Debate Is Becoming More Urgent
Clark's comments come during a period of unusually intense debate about the potential long-term risks of artificial intelligence.
Former and current AI researchers have raised concerns about systems becoming increasingly autonomous and potentially capable of improving their own capabilities.
Former Anthropic researcher Evan Hubinger has publicly discussed a personal estimate of more than 10% for the possibility of AI causing human extinction within the next decade. Nobel Prize-winning computer scientist Geoffrey Hinton said such a probability was not unreasonable, although Clark himself said he does not find precise extinction percentages particularly useful.
Clark instead emphasized the broader problem of allowing the technology to develop without sufficient safeguards.
He described the situation as taking enormous risks and argued that the industry needs to change course.
His argument does not amount to a prediction that AI will necessarily destroy humanity. Rather, it reflects a precautionary approach: if advanced AI could create catastrophic consequences, developers and governments should build safeguards before systems become too powerful to control.
A Wider Question About Who Controls AI
The debate over kill switches ultimately raises a larger question: who should have the final authority over increasingly powerful AI systems?
AI companies currently develop and deploy most advanced models, giving them significant control over the technology.
Governments, however, are increasingly considering whether some decisions should be subject to independent oversight.
A mandatory and independently verifiable kill switch could become one possible component of such a framework.
Other proposed measures include independent model evaluations, government audits, transparency requirements, restrictions on dangerous capabilities and stronger international coordination.
Reuters reported that U.S. lawmakers are already considering measures that would give government officials greater ability to inspect and evaluate advanced AI systems.
Whether those proposals eventually become law remains uncertain.
No Universal AI Kill Switch Requirement Yet
Despite the growing discussion, there is currently no universal requirement forcing every AI company to install a standardized kill switch.
Clark's comments should therefore be understood as a proposal for future regulation rather than an announcement of a new rule.
Anthropic and other AI companies already maintain internal safety and emergency procedures, but the question is whether governments should make particular safeguards compulsory and require them to be independently tested.
That debate is likely to become more important as AI systems gain greater autonomy and access to external tools.
For Anthropic, the issue also fits into the company's broader public positioning around AI safety. Amodei has called for a slower approach to frontier AI development, while Clark is now highlighting the possibility of mandatory technical safeguards that could give humans a way to intervene if systems become dangerous.
The debate is increasingly moving beyond whether AI should be developed and toward a more practical question: what safeguards must be in place before increasingly powerful systems are allowed to operate at scale?
A mandatory, independently verifiable kill switch could become one answer.
But turning the idea into a workable regulation would require agreement on technical standards, independent testing, emergency thresholds and who ultimately has the authority to shut down an AI system.
For now, Clark's proposal adds to a rapidly expanding international conversation over how to ensure that increasingly capable artificial intelligence remains under meaningful human control.
