Developer Niko1221 has released the open-source Strata engine, designed to run the quantized Qwen3.8-Flash-Next model with 125 billion parameters on consumer-grade graphics cards. The model, originally introduced by Alibaba in August 2026, is a multimodal mixture-of-experts architecture that includes a 51 billion parameter n-gram embedding table and supports a 262K token context window.
Strata optimizes memory usage by loading the full MoE model into system RAM while keeping only frequently used experts in VRAM. Additionally, it employs speculative decoding with a lightweight model to accelerate inference. The setup requires at least 12GB of VRAM, 32GB of system RAM, and 80GB of storage space on Windows 10/11 or Linux.
Performance tests on an NVIDIA GeForce RTX 5070 with 12GB VRAM and 64GB RAM showed the 2-bit quantized version (Q2_0) achieving 94 tokens per second, while the 3-bit version (IQ3_S) reached 53 tokens per second. On an AMD Radeon RX 9070 XT with 16GB VRAM, the Q2_0 version achieved 60 tokens per second.
