Iterate Studio Inc. has launched Lifeboat, an inference engine for large language models designed to optimize GPU utilization for AI agents. The software allows for two to six times as many concurrent AI agent sessions on a single GPU by implementing fair scheduling, admission control, and key-value cache optimizations.
In internal tests using an Nvidia RTX PRO 6000 Blackwell GPU with a Qwen 30B-A3B model, Lifeboat supported 2,048 concurrent sessions. With optimizations enabled, the engine achieved 8,714 tokens per second, compared to 4,965 tokens per second without them. Under memory pressure, the time to the first token was reduced to 1.5 seconds, down from 189 seconds in the baseline.
The engine includes a Confidential Computing edition that utilizes hardware attestation for AMD, Intel, and Nvidia H100, B200, and GB300 processors. This feature ensures model weights remain encrypted within trusted execution environments. The software is now generally available following an early access period.
Iterate.ai offers a free Developer License for noncommercial use on up to two inference servers. The Standard License is priced at $49.99 per month, while the Confidential Computing edition costs $499.99 per month. Both paid tiers include a seven-day free trial.
