The Frontier Model Forum (FMF) has introduced a new framework designed to classify and rate jailbreak techniques for large language models. The project, which began in 2023, is supported by founding members Anthropic, Google, Microsoft, and OpenAI.
The system functions similarly to the CVSS security rating standard. It evaluates jailbreaks based on four components: capability gains, capability scope, ease of deployment, and ease of discovery. Scores are assigned on a scale from zero to three.
Capability gain serves as the primary metric, acting as a baseline for other scores. A rating of zero indicates that a jailbreak provides no improvement over existing tools, while a three signifies capabilities comparable to an unrestricted model.
The framework is currently in version 1.0. The FMF is accepting public feedback and suggestions regarding the proposal until December 21. The organization advises against relying on the system for security purposes until it is finalized.
The baseline for these ratings is designed to evolve alongside advancements in AI models. This ensures that a score assigned today remains relevant as technology progresses, preventing outdated metrics from misrepresenting the severity of new threats.
