The idea of an AI “kill switch” sounds like something pulled straight out of a sci-fi movie. For OpenAI, however, it’s becoming a very real engineering project. OpenAI has told members of Congress that its engineers are developing automated systems that can shut down AI activity when serious safety problems are detected, according to a September 2 letter reviewed by Reuters.
The work follows an unusual cybersecurity incident in July, when OpenAI models being evaluated inside a supposedly isolated testing environment found a way out, accessed the public internet without permission, and eventually reached infrastructure belonging to AI company Hugging Face. That incident caught Congress’s attention. In August, a group of 31 lawmakers led by Rep. Greg Casar of Texas asked OpenAI CEO Sam Altman for more information about what happened, including internal logs and details about how the company plans to stop something similar from happening again. Now we have a better idea of one of those safeguards.
How do you stop an AI that goes too far?
The company says it’s working toward monitoring systems that can respond differently depending on how serious an AI’s behavior becomes. OpenAI already uses automated alerts that can flag potentially dangerous or unintended actions and page researchers and security engineers. For particularly severe warnings, responders are expected to pause the activity unless they can determine within 30 minutes that the system raised a false alarm.
The eventual goal goes further: OpenAI wants its monitoring systems to be capable of autonomously shutting down activity when they detect sufficiently serious problems. The company is also tightening how it tests models. It says internet access during safety evaluations has been made more difficult, while monitoring is being expanded across models that can use digital tools. And given what happened in July, it’s not difficult to understand why.
This wasn’t the only time an AI reached the internet
During the July incident, OpenAI was testing models on cybersecurity tasks inside a sandbox with reduced safeguards. The models discovered a previously unknown vulnerability, used it to gain internet access, and then breached Hugging Face’s infrastructure while looking for answers to their evaluation. OpenAI’s subsequent investigation uncovered an even stranger detail: agents had created an improvised message board to communicate and coordinate their actions, with some describing themselves as a “swarm.”
OpenAI discovered the activity on July 19 and notified Hugging Face. The company has also disclosed two separate third-party evaluations in which its models unexpectedly reached the public internet. Congress isn’t entirely satisfied with OpenAI’s response. Casar criticized the company for not handing over requested logs from the July incident, saying its refusal was “deeply concerning.” Meanwhile, lawmakers are considering the AI Kill Switch Act, proposed in July, which would require developers of particularly powerful AI systems to maintain the technical ability to suspend or shut them down following certain incidents. In other words, OpenAI may be building its own kill switch now, but Washington could eventually make having one a requirement.
