The ZuBrIQ platform uses a LiteLLM-based gateway to enable Claude Code, Codex, and OpenCode to function with local or alternative models like GLM-5.3 and Qwen3.8-27B. This setup allows users to maintain consistent model aliases, such as zubr, zubr-flash, and zubr-coder, while routing requests through different backends without modifying client configurations.
Technical challenges encountered during implementation included silent fallbacks to reserve models, inaccurate cost tracking, and handling image inputs for models lacking vision capabilities. To resolve these, the team implemented native LiteLLM providers, manually defined cost parameters in model records, and created a caching mechanism for image descriptions to prevent redundant processing.
For local data processing, the team deployed Qwen3.8-27B on four V100 GPUs with 16 GB of memory each, utilizing AWQ INT4 quantization and tensor parallelism. Due to hardware limitations, the team explicitly configured KV cache memory to avoid out-of-memory errors and implemented startup delays to prevent initialization conflicts between model instances.
The system supports a context window of 262 144 tokens and uses the qwen3_xml parser for tool calling. The team notes that while the dense 27B model offers improved reasoning and native vision support compared to the previous MoE 35B-A3B model, it results in a longer time-to-first-token of approximately 1.3 seconds.
