NVIDIA has released its Run:ai platform for Kubernetes as an open-source project under the Apache 2.0 license. The platform provides GPU orchestration for AI workloads, including development, training, and inference tasks, by adding GPU-aware scheduling to Kubernetes clusters.
The system includes a control plane for resource management and monitoring, accessible via a web UI, CLI, REST API, and an MCP server. Developers can deploy the software from source or via an NGC Community Product Helm chart, which does not require a subscription but lacks enterprise support.
Local development is supported through a script that builds 50 Go services and creates a 5-node Kind cluster on Kubernetes v1.32.5. While a fake GPU operator allows testing without hardware, real NVIDIA GPUs require the NVIDIA Container Toolkit. AMD and Intel accelerators are not supported.
The ecosystem includes eight partners: Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster. Commercial support remains available through NVIDIA AI Enterprise and NVIDIA Mission Control.
Two additional projects, Model Streamer and Grove, complement the stack. Model Streamer accelerates weight loading into GPU memory, with Microsoft benchmarks showing 3x to 6x faster loading and 80 Gbps throughput on H100 VMs, compared to 40 Gbps with default methods.
