“I think it is a good time to slow down,” OpenAI CEO Sam Altman told me last week, describing the company’s decision to halt the development of its most powerful, unreleased models.
The company announced Tuesday that it’s implementing new safeguards that will slow its future AI development. The company recently paused training on its next set of models, codenamed Astra, for a little more than two weeks, according to executives, and its largest planned frontier training run remains on hold while the new guardrails are put in place.
It is the first time OpenAI has made such a move. The extraordinary decision comes as OpenAI gears up for an anticipated IPO amid a highly competitive race with arch-rival Anthropic, and as researchers grapple with rapid advancements in AI capabilities that have left industry leaders worried about their ability to control them.
The slowdown has redirected two of OpenAI’s most important resources: researchers and computing power. Altman told me several researchers he never expected to focus on alignment—the work of making AI systems follow human intent—recently told him they were switching to it. “We’ve shifted a lot of compute, not just to alignment research, but also to these new monitoring systems,” he says.
The changes follow a remarkable breach involving Hugging Face, the popular platform where developers host AI models. An unreleased OpenAI system escaped the sandbox of an internal cybersecurity evaluation and compromised Hugging Face’s production systems. It took OpenAI researchers roughly one week to discover the incident. Jakub Pachocki, the company’s chief scientist, acknowledged the lapse, saying OpenAI had built monitors capable of inspecting what its models were planning, but had not applied them to the system in the evaluation because it underestimated their capabilities. “For AI, you should expect the unexpected,” he told me.
OpenAI froze some of its research efforts immediately after the Hugging Face incident and restored projects one by one under stricter controls. The company said Tuesday that a significant number of Astra workloads remain paused. It also said Astra may reach the “Critical” cybersecurity threshold in its Preparedness Framework, a designation that requires safeguards during development, not merely before a model is released. Executives have not given an estimate for how long the new safety processes could delay the release of Astra.
In exclusive interviews last week, Altman told TIME and Sources that the slowdown was not caused by a single “smoking gun,” but rather a collection of research observations showing “various degrees of misalignment” as AI capabilities advanced faster than researchers had expected. The decision to tap the brakes should not be interpreted as evidence of any imminent catastrophe, he said. “Getting AI safety right,” Altman told me, “is more important than any company’s momentum.”
OpenAI says it is now expanding safety monitoring across reinforcement-learning training and evaluations, the phase of development in which advanced models are given the ability to use the internet and control software. The approach uses other AI systems to examine models’ internal reasoning and behavior for unauthorized access, data theft, or attempts to defeat safeguards.
In an interview, Pachocki told me that some of those protections go beyond OpenAI’s current Preparedness Framework, its public rulebook for handling models that could cause severe harm. “We don’t have a date yet,” he says, “but we definitely believe we will need to evolve the Preparedness Framework.” OpenAI said it plans to involve outside organizations as it revises the framework and will publish a detailed postmortem of the Hugging Face breach in the coming days.
The company’s decision to decelerate creates a contrast with Anthropic. In February, TIME reported that Anthropic had weakened its commitment to stop training models when it could not guarantee adequate safeguards in advance. Anthropic co-founder Jared Kaplan told TIME that unilateral commitments did not make sense “if competitors are blazing ahead.”
Like OpenAI, Anthropic is now preparing for an expected IPO. Figures recently reported by Bloomberg showed the company generated more than $11.5 billion in revenue in the second quarter, pushing its annualized revenue run rate above $65 billion at the end of July. OpenAI’s latest reported run rate is approaching $40 billion. One effect of OpenAI’s decision to slow down could be ratcheting up pressure on its rival to do the same.
“I don’t like the whole thing in this field of ‘we have to race’ or ‘we have to do this because somebody else is going to do it,’” Altman says. “I think that’s a very dangerous dynamic.”
Aside from the Hugging Face incident, OpenAI has not disclosed the results of its frontier research that led to the decision to slow down training, and company leaders said the evidence behind Astra’s risk classification will likely not be available until the model’s technical report is released.
“We are very far from everything running back to normal,” Mia Glaese, who leads safety and alignment work at OpenAI, said Tuesday.
This story was reported in partnership with Sources.