Microsoft CEO Satya Nadella has proposed a new trust architecture for artificial intelligence, arguing that current monitoring strategies are insufficient. In a statement released on October 10, he emphasized that AI models function as black boxes, yet they are increasingly granted access to sensitive data and critical tasks.
Nadella advocates for the development of closed systems that allow for constant observation and testing. He suggests that developers should treat both closed-weight and open-weight models as internal risks, assuming that any powerful system could be compromised or prone to errors.
The proposed framework centers on observability and independent control. Key principles include model diversity, where no single model validates its own work, and the requirement that every action leaves a tamper-proof, human-readable record for independent audit.
A critical component of this strategy is containment. Nadella argues that developers must design systems under the assumption that a model could be compromised, implementing emergency stop mechanisms that allow authorized personnel to pause or shut down tasks at any time.
Finally, the framework calls for mandatory incident disclosure. Organizations should share details regarding system failures, the causes of breaches, and the specific controls that failed. This approach aims to foster industry-wide learning and improve safety standards for future AI deployments.
