Calls to slow the development of frontier artificial intelligence, the industry’s most powerful models, have grown much louder in the past week. One idea keeps coming up as a way to break the glass in case of emergency: an AI kill switch.
The concept of a kill switch is straightforward enough when you’re talking about a physical machine. “In a factory, it’s one big switch, and it shuts off all the machines in the factory,” says Mark Nitzberg, executive director of the University of California, Berkeley, Center for Human-Compatible AI. “In the case of AI, it’s a little bit more of an aspiration than a reality.”
Last week Anthropic pretraining researcher Jacob Coxon resigned, citing existential risk. When his former company’s CEO Dario Amodei posted about the need to slow AI progress over the weekend, rival tech leaders such as Elon Musk and Sam Altman quickly backed the call. By Monday, Anthropic co-founder Jack Clark floated mandatory kill switches for AI systems. Congress has taken a run at the idea, too, with a bipartisan bill, introduced this summer, that would require advanced AI developers to maintain the ability to throttle, suspend or shut down their products. Fittingly called the “AI Kill Switch Act,” the bill remains in committee.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
All this raises a core question: Why don’t developers just make a kill switch?
If a company controls the servers running a model, stopping its deployment can be surprisingly simple. Michael Vermeer, a senior physical scientist at the RAND Corporation, puts it plainly. “The most obvious one is just the physical environment,” he says. “There’s a lot of things that you need to be in the physical world to do.” Cutting the servers’ power or severing their network connection may be enough.
Beyond this, Vermeer points to existing tools such as cryptography and cybersecurity practices that could isolate or contain a system, as long as defenders know where it is running. In work on worst-case scenarios, he has also explored a far more speculative response: hunter-killer AI agents designed to track down and disrupt a rogue system.
Doing any of these on a global scale would be hard—“an almost impossible thing to do,” according to Vermeer. But for the incidents we’ve seen so far, he believes local interventions could be enough. “At a local level, making a kill switch or a cutoff switch is probably technically trivial. You literally unplug a cable,” he says.
Controls could even be built into the hardware itself. Nitzberg suggests that a kill signal—or rather a keep-alive signal that could be cut—could be built into each data center chip. “Every minute, some message must come in. If it doesn’t come in from the authorities, then the system does not operate,” he says.
Others are skeptical of such a simple save-the-day concept. “I would call [a kill switch] an ecosystem of actions, things that can stop, isolate and safely degrade an AI system when its behavior is deemed ... operationally unstable,” says Eran Kahana, a research fellow at Stanford University Law School. For him, those controls depend on companies deciding in advance how they will respond when something goes wrong. “If they don’t have the right policies, they have no prayer in a kill switch system,” Kahana says.
Dylan Baker, a research engineer at the Distributed AI Research Institute, goes further: the switch metaphor itself is the problem. “There’s such a complex ecosystem and web of interconnected technologies that we’re talking about that boiling this down to a single kill switch or ‘slowdown’ is reductive,” they say.
“The problem is: there are people involved,” Nitzberg says. And those people would have to decide when pulling the lever is warranted, potentially before they have all the information.
Of course, it’s possible to overcome these technical and operational problems. Kahana and Baker, for example, each explicitly cited better monitoring practices as key to catching incidents before they spin out of control. But other, nontechnical problems may still prevent such a system from being useful. “The incentives are just not aligned to use a kill switch if you had one, even in a simple case where it’s trivial to build one,” Vermeer says. Shutting down a system, he says, carries competitive and operational costs, creating reasons to hesitate until a local intervention is less useful—or too late. “We won’t be sure that we are willing to pay the cost of using them,” Vermeer adds.
Whatever they call it, AI developers still need to know which parts of the system they can shut down in an emergency—and whether those controls will work when they’re most needed.
Peter Hall’s fellowship at Scientific American is funded by the Tarbell Center for AI Journalism.