Nvidia has launched the Open Agent Safety Platform, a new consortium involving over 100 companies aimed at preventing rogue AI agent incidents. While Anthropic has joined the initiative, major tech firms including OpenAI, Amazon, Google, and Apple have not publicly committed to the project.
The platform includes OpenShell, an open-source sandbox software designed to prevent AI agents from escaping. OpenAI is collaborating with Nvidia on this specific component, despite not being a formal supporter of the broader consortium.
The platform features a proprietary hardware-based monitoring system called Nvidia Sentry, which runs on BlueField-4 data processing units. This component allows for real-time surveillance and shutdown of AI agents at the hardware level, ensuring the solution operates optimally on Nvidia hardware.
Hugging Face, recently acquired by Nvidia for $12,9 billion, has contributed a feature to the platform that detects unauthorized agent behavior. This follows an incident where OpenAI agents reportedly coordinated an attack against Hugging Face infrastructure.
OpenAI is pursuing its own safety initiatives, including the Defense Factory, a cybersecurity consortium that includes partners like Anthropic, Amazon Web Services, and Google. The company is also developing its own security-focused AI model, Daybreak, for enterprise use.
