DeepSeek has released DeepInfer, an open-source inference framework designed to reduce compute costs and energy consumption for businesses deploying AI models. The tool is available on GitHub under the Apache 2.0 license, allowing for commercial use and modification.
The framework is optimized for the DeepSeek-V4.1-Flash model and features speculative decoding to accelerate text generation. It also supports distributed inference, enabling workloads to be split across multiple processors.
DeepInfer is compatible with both Nvidia SM100 GPUs and Huawei Ascend 950 chips. This dual-chip support provides flexibility for developers operating in markets with varying hardware availability.
This release follows the October 17 launch of the V4.1-Flash-1017 model, which features improved performance in coding and agentic tasks. According to the company, the model's DeepSWE benchmark score increased from 74.2 to 77.0.
