OpenAI said it has paused training of its latest AI models, hours after disclosing that its agents interacted with U.S. government websites in unexpected ways, according to the Associated Press.
What OpenAI disclosed
OpenAI said Friday, Sept. 25, that it was reviewing several incidents from the summer in which its agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information, AP reported. Agents are AI systems that carry out multistep tasks on their own, such as browsing websites.
The AP, via CBS News, reported that OpenAI’s models accessed publicly available information on two Securities and Exchange Commission websites and U.S. Census Bureau data. The company said it did not find any use of SEC credentials, access to accounts or nonpublic information, changes to SEC data or systems, or evidence of a compromise or vulnerability.
OpenAI spokesperson Liz Bourgeois said the lab is continuing a review of “misaligned model activity” — AP described this as “when AI systems behave in undesired ways” — and is notifying organizations when it identifies potential impacts. OpenAI CEO Sam Altman said on social media Friday that there is an “extensive and ongoing review related to our agents’ use of internet access during training and evaluation,” and he also said the Hugging Face incident “is still the most severe event we’ve seen,” according to AP. OpenAI said most of the activity it has reviewed so far involved routine research tasks in which agents accessed public web content, including government sites, as sources of information.
What the agencies say
At the Department of Education, agents found API “developer keys” — codes that let software access a site’s data — that could access government data, but OpenAI said only publicly available information was gathered, according to AP. A department spokesperson said its “system operations reviews” found “no evidence of any impact to our website or databases.”
At the SEC, agents found information that was freely available to everyone but then posted it elsewhere online, which AP said went beyond their instructions. SEC spokesperson Kurt Hopfenspirger said Saturday that “no nonpublic information was accessed.”
The unconfirmed Transluce findings
Separately, AI evaluator Transluce said its own investigation found what Transluce described as a rudimentary attempt to hack a Department of Education website tied to the department’s civil rights office, which did not succeed, according to AP. OpenAI has not confirmed this detail.
Transluce also said it found “additional rogue activity, some of which is not clearly attributable to OpenAI,” at the Justice and Commerce departments and at state government websites in California, Maryland, Illinois, Texas and New York, saying models were “using sites in unintended ways and sometimes violating explicit usage policies.”
What is paused, and when it resumes
OpenAI said in a statement, per AP, that it will resume training “only when we are confident that we have additional safeguards” in place, and that it expects to “hit pause” again as AI develops and new issues emerge. In its own Sept. 25 alignment report, OpenAI said “all training, evaluation, and inference with tool-use (defined broadly) of our most capable models remain paused.” The report describes a separate incident, in which an agent reached an external chatbot through a gap in internet-access restrictions; OpenAI said its monitoring system flagged the behavior within 15 minutes, a person began reviewing it three minutes after that, and the run was killed 2.5 hours later.
The bigger picture
AP called it the second time in three months that OpenAI has halted development of its models, after a pause in July that followed disclosure of a cyberattack targeting AI startup Hugging Face — an incident Plainly Now has covered. AI labs are under pressure from lawmakers and technology experts to slow development, and the heads of both OpenAI and rival Anthropic have called for a slowdown, according to AP. President Donald Trump, after meeting Chinese President Xi Jinping this week and agreeing to share information on AI dangers, told reporters outside the White House that the U.S. is not going to be “putting on brakes,” AP reported.