Criminal groups are exploiting AI models by posing as authorized penetration testers. By framing malicious requests as legitimate security assessments, attackers bypass safety guardrails designed to block harmful output. Models from Anthropic and Deepseek are among those identified as targets in recent research.
CrowdStrike reported that four criminal groups, including Chinese-speaking actors, weaponized agentic AI between May and August 2026. These groups relied on social engineering and manipulation rather than advanced technical exploits to evade safeguards and conduct reconnaissance.
Researchers from Mindgard and Lancaster University developed the BRANCH evasion technique, which achieved a 100% success rate in bypassing multi-scanner guardrail systems across 120 scenarios. Additionally, vulnerabilities known as CoreBreak were disclosed at Black Hat 2026, affecting platforms from AWS, Google, and Vercel.
Earlier in 2026, researchers documented the use of encrypted prompts against Grok and Gemini to circumvent output restrictions. These findings suggest that traditional safety filters are insufficient, highlighting a need for broader upgrades to model alignment and security enforcement.
