Microsoft is upgrading its Windows ML framework for Windows 11, introducing experimental native support for GGUF and ONNX model inference. This update, announced during the October Windows and Surface event, includes the new Windows ML Runtime API, which offers deeper system integration and improved performance for local AI tasks.
The company is collaborating with the llama.cpp and GGUF open-source communities to enable developers to import models from Hugging Face for local execution. Additionally, Microsoft is working with Nvidia to contribute performance optimization code to the llama.cpp project, further enhancing the efficiency of local AI model deployment on Windows devices.
The new Windows ML Runtime API also introduces dedicated Text Generation and Speech Recognition APIs, allowing developers to build these features directly into their applications. While the existing ONNX Runtime API remains fully supported, Microsoft intends to prioritize future Windows-specific optimizations for the new Windows ML Runtime API.
Developers can now choose between the established ONNX Runtime and the new Windows ML Runtime API based on their current workflows. Both environments are included in the latest version of the system, providing flexibility for those looking to transition their AI projects to the updated framework.
