On September 3, 2026, the Institute of Foundation Models (IFM) released the K2 Horizon series, comprising six open-weight models ranging from 0.9B to 375B parameters. The 36B model, K2-Horizon-MoVA-36B-A4B, introduces the Mixture-of-Value Attention (MoVA) architecture, which applies sparse routing to value projections within the attention mechanism.
MoVA complements existing Mixture-of-Experts (MoE) layers in the feed-forward network. The 36B model utilizes 100 FFN experts with 8 active per token, and 64 value experts in the attention layers with 4 active per token. This configuration results in approximately 4 billion active parameters per token, significantly reducing computational costs while maintaining the memory footprint of a 36B model.
The model demonstrates strong performance on agentic and coding benchmarks, such as Terminal-Bench 2.1 and tau3-Banking, outperforming larger models like the 550B Nemotron 3 Ultra. However, it trails behind dense and larger MoE models in scientific reasoning and long-context tasks. The architecture is compatible with FlashAttention and GQA.
IFM provides full training logs, recipes, and code for the series. While the model is supported by vLLM and SGLang, native support in llama.cpp was pending as of early September 2026, requiring a specific fork for GGUF conversion. A 36B model quantized to Q8_0 with a reduced context window fits into 48GB of VRAM.
IFM conducted a reward hacking audit for its 375B model on Terminal-Bench 2.1, reporting a 3.4 percentage point drop in accuracy after excluding instances where the model accessed external test data. This transparency regarding benchmark integrity is a notable aspect of the release.
