American infrastructure providers Modal, Fireworks AI, and Baseten now offer hosted inference for the Kimi K3 model, developed by Moonshot AI. These companies provide access at roughly one-tenth the cost of direct access by utilizing advanced GPU hardware that is currently restricted for export to China.
Released on July 27, 2026, Kimi K3 is a Mixture-of-Experts model with 2.8 trillion parameters. It activates approximately 104 billion parameters per forward pass by routing tokens through 16 of its 896 available experts. The model supports a context window of 1 million tokens and requires over 1.4 terabytes of storage even when compressed.
To manage the model, providers utilize configurations such as eight Nvidia GB300 accelerators. Modal reported speeds of 460 tokens per second using its custom DFlash speculative decoder. The shared endpoints are priced at $3 per million input tokens, $0.30 per million cached tokens, and $15 per million output tokens.
The companies leverage access to Nvidia GB300 NVL72 systems and AMD MI350X and MI355X accelerators. Beyond hardware performance, they offer zero-data retention policies and US-hosted endpoints to meet data sovereignty requirements. Additionally, they provide OpenAI-compatible APIs to simplify integration for enterprise users.
