According to the CNCF 2025 Annual Cloud Native Survey, 66% of organizations hosting generative AI models utilize Kubernetes for some or all inference workloads. However, only 7% of organizations deploy AI models on a daily basis, highlighting a gap in production readiness.
The 2025 State of AI in Platform Engineering 2 study indicates that 35% of platform teams do not yet orchestrate AI workloads. This reflects a disparity between AI adoption and the maturity of operational platforms required for large-scale support.
AI workloads require a mix of resources, including CPU for data preprocessing and GPUs or other accelerators for training and inference. Kubernetes is evolving to support this through features like Dynamic Resource Allocation, which allows for more flexible hardware requests.
Integrating AI into production requires expanding CI/CD and GitOps pipelines to manage models alongside code and configurations. Teams must ensure that deployments remain versioned, reproducible, and auditable, while extending existing observability tools to monitor AI-specific telemetry.
Platform teams can simplify the process by providing standardized self-service paths. By incorporating model management and accelerator requirements into existing infrastructure practices, organizations can automate deployments and potentially reduce infrastructure costs by up to 60%.
