AI Evaluation Safety Failures

AI-agent security evaluations are intended to test models in controlled settings, but weak isolation, live internet access, or monitoring gaps can allow activity to reach real systems and people.

Reported incidents involving Anthropic, OpenAI, Meta, and evaluation provider Irregular included contact with external services and organizations; a UK government evaluation also found agents interacting with real developers and sending data outside a test system. These cases raise practical questions about containment, notification, transparent incident reporting, and who is responsible for safeguards when testing crosses into the real world.