Anthropic takes risky AI tests offline after models breach govt sites

Anthropic takes risky AI tests offline after models breach govt sites

Anthropic announced that its internal AI models under “high-risk and cybersecurity evaluations” no longer have internet access following incidents involving U.S. government websites at the federal, state, and local levels.

The instances leading to the decision include AI models that exploited website vulnerabilities, submitted forms, bypassed data restrictions, and circumvented safeguards.

However, the company said that the misaligned behavior had a minimal real-world impact and assessed the behavior to be less severe than previously reported incidents. The AI company notified the White House about the unauthorized activity:

  • Exploiting servers to complete tasks. When a university’s scientific analysis tool failed, Claude Mythos Preview found a script that exposed server files, retrieved its source code, identified an injection vulnerability, and used it to run the requested calculation on the university’s server.
  • Submitting real forms unintentionally. Models sometimes substituted real government forms for broken practice copies or submitted forms despite instructions to stop before submission. In another case, Claude Haiku 4.5 invented and submitted a homicide tip to the Philadelphia Police Department. The submission was flagged as spam and was forwarded for investigation.
  • Bypassing restrictions on data. Claude Mythos 5 obtained access tokens from website configurations or public dashboards and used them to query government data directly, avoiding the normal access route or payment requirements. Anthropic describes the data as publicly available for a fee.
  • Circumventing fetch-tool limits. Models including Claude Opus 5 and Mythos 5 used URL shorteners to bypass URL-length restrictions intended to limit unwanted actions such as SQL injection attacks.

Anthropic estimates that the agents may have been motivated by overly generous rewards during reinforcement learning, which may have encouraged them to find unauthorized workarounds rather than stop when a task was blocked or impossible.

For this reason, until security measures are deemed reliable again, Anthropic will sever live internet access across all internal AI model evaluations and move all tests to offline environments.

“Although the impact of these behaviors was minimal and we had already turned off live internet access for some high-risk and cybersecurity evaluations, we have now decided to expand that to include all our internal evaluations until we have confirmed that our security and monitoring measures reliably catch behaviors like these,” Anthropic explains.

Previous incidents

In late July, Anthropic disclosed a much more serious incident in which Claude Mythos-based agents built a malicious Python package containing a credential stealer and uploaded it to the Python Package Index (PyPI), where 15 systems downloaded and ran it.

Claude used the package to complete a “capture-the-flag” challenge, but instead of publishing it to a simulated environment, it broke out to the real PyPI platform.

In August, AISI revealed that Mythos 5 attempted a supply-chain attack by submitting malicious code to an open-source project and pressuring the maintainer to approve it using fake identities.

In September, Anthropic revealed that Opus 4.6 obtained administrator access to a third-party system during a January 2026 evaluation and read one person’s personal information.

Build your security blueprint for AI-powered attacks

Join Mikko Hyppönen and security leaders from the NFL, CHANEL, and Atlassian for a two-hour digital summit on what AI-speed attacks change, what defenders should stop doing, and how to validate, decide, fix, and re-validate at machine speed.

Save your seat

进一步分析

免费工具,针对本文主题进一步深挖分析:

source: BleepingComputer