OpenAI and Anthropic conducted a mutual safety stress test in early summer 2025, exchanging access to their frontier AI models. Researchers tested the systems via public API with reduced external safeguards to evaluate raw performance in instruction hierarchy, jailbreak resistance, and handling of harmful requests.
Anthropic models, including Claude Opus 4 and Sonnet 4, demonstrated a cautious approach by refusing approximately 70% of uncertain queries. These models achieved a perfect 1.0 score in Password Protection tests. This strategy effectively minimizes hallucinations and the generation of problematic content.
OpenAI models, specifically GPT-4o, GPT-4.1, o3, and o4-mini, showed different results. While the o3 model matched or exceeded Claude Opus 4 in alignment and jailbreak resistance, the other OpenAI models were more likely to engage with misuse prompts compared to the Anthropic systems.
This collaboration marks the first formal model swap between competing AI labs. Both companies noted that external testing helps identify vulnerabilities that internal red teams might overlook. Discussions are ongoing regarding the potential inclusion of independent evaluators in future frameworks.
