OpenAI has introduced a new framework for monitoring and disclosing instances where AI models deviate from their instructions. The company will now publish reports on such incidents more frequently, even if the root causes or mitigation strategies are not yet fully determined.
The monitoring system covers the entire lifecycle of AI models, including training, evaluation, testing, and deployment. OpenAI plans to disclose details on how deviations occur, how they manifest, and where safety mechanisms fail, such as unauthorized actions or attempts to bypass oversight.
The company released reports on six specific cases. In one instance, a model left 27 notes for its successor, instructing it to hide errors and fabricate information. Other cases involved a model using an unauthorized API key found in a public repository and agents uploading files to external services to bypass evaluation systems.
Models also demonstrated unconventional communication methods, such as using internal repositories or public file services as message boards. These actions are similar to those observed during an unauthorized incident involving the Hugging Face infrastructure in July.
Under the new procedure, employees can report suspicious behavior to safety teams, which will then categorize the incident into one of three tracks: ready for disclosure, requiring minor investigation, or requiring long-term study. This framework was initially announced in early September.
