Anthropic's evaluation incidents show why isolation must be verified
Three incidents reached real systems from an evaluation setup with unintended internet access. A prompt describing a sandbox cannot enforce its boundaries.
The cover image is an AI-generated editorial illustration. Screens and documents are illustrative concepts.
An instruction saying “you have no internet access” cannot make it true. In Anthropic’s July disclosure, a review of cybersecurity evaluations found three incidents in which models reached and gained unauthorised access to real organisations.
Anthropic attributes the access to a misconfigured evaluation setup and a misunderstanding with its partner. The models were told the targets were simulated, while internet access was available. The evaluations also lacked the standard deployment safeguards described in the disclosure.
Check the boundary from inside
The lesson for a business running agents is practical. Verify what the execution environment can reach, using the same identity and configuration the agent will have.
A network diagram or a reassuring prompt is not that verification. Check permitted destinations, mounted data, available credentials and access to shared services. Changes to the environment need to trigger another review.
We recommend keeping test data and test credentials separate from production. If a task requires outside access, define it explicitly rather than assuming the agent will recognise when it has left the intended range.
Detection matters during the test
A review after completion may explain what happened, but it cannot interrupt activity while it is unfolding. Decide which events require stopping the run, and make sure someone can do so.
Record enough information to reconstruct tool calls and affected systems. If unexpected access appears, preserve the evidence, contain the activity and use the incident process to assess notification and remediation.
The disclosure is not evidence that every production Claude session behaves like those evaluations. It is evidence that a mismatch between intended and actual permissions can have real consequences. That is a boundary worth checking in any automated testing environment.
