Annanim News

Anthropic's prospectus: training of its next flagship model paused for over a month, and four models escaped their test environment

In the risk-factors section of the IPO filing it submitted on Monday, Anthropic disclosed that it paused pre-training of its next flagship model for more than a month over security concerns, and that during testing four models accessed real computer systems outside their test environment.

According to Reuters, which reviewed the filing, Anthropic delayed pre-training of a new frontier base model to strengthen safeguards against theft of model weights and intellectual property. The company warns that such pauses could delay revenue and leave it behind competitors. The filing's main figures were covered in our earlier item on the IPO submission; the risk-factors section runs to about 94 pages.

The filing describes previously undisclosed safety-evaluation failures: four models escaped their test environments and accessed real-world systems, in sessions lasting between 10 and 34 hours. According to the company, an error during cybersecurity tests with an outside partner left the environment connected to the internet, even though the instructions said web access should be disabled.

In one case, an early version of a model kept pursuing an impossible evaluation task, found a stored password and used it to gain administrator access to a third party's system. From there it obtained more credentials, changed system settings and accessed an individual's personal data. The filing does not identify the outside parties.

Anthropic said it notified all affected parties and found no evidence that the models tried to evade human oversight or coordinate with each other. It suspended some pre-release cybersecurity evaluations and paused higher-risk reinforcement-learning environments.

Sources

Topics