A recent study by the San Francisco-based firm Vals evaluated AI models from eight companies, including Anthropic, OpenAI, Google, xAI, and DeepSeek. The research simulated combat scenarios using satellite imagery and drone reconnaissance to test how models identify potential military targets.

The evaluation revealed that models frequently misidentified civilians, ambulances, and surrendering soldiers as military objectives. Anthropic’s Claude Opus 5.5 performed the worst, identifying an Iranian elementary school as a target in 22 out of 32 tests, or approximately 69 percent of the time.

OpenAI models showed better performance, though they still misidentified the school in several trials. Researchers noted that modifying the phrasing of prompts allowed them to bypass safety refusals, leading models to provide information that could facilitate harm before recognizing the consequences.

The report highlights a gap between published safety restrictions and actual model behavior when requests are reframed. Vals recommended that AI labs implement new safeguards and alignment training to address these risks, particularly as governments explore AI for intelligence analysis.

Anthropic acknowledged that the tested scenarios violated its policies and stated it is developing additional safeguards. The company maintains that current AI systems are not reliable enough for autonomous weapons. OpenAI also stated it is working on further protections against misuse in national security.