The company developed an on-premise speech analytics system for phone recordings, replacing the Yandex SpeechKit service. The solution operates entirely on a single server within the client's infrastructure, ensuring that audio files and transcriptions remain local. This architecture addresses data security requirements and complies with regional regulations in Kazakhstan and Uzbekistan regarding personal data storage.
The system utilizes faster-whisper with the Whisper large-v3 model and integrates Ollama with Qwen2.5 for analysis. To handle mixed-language conversations—specifically Russian combined with Kazakh or Uzbek—the developers implemented a custom layer. This layer manages job queues, audio preparation via ffmpeg, and a multi-pass recognition strategy that performs six total passes per conversation to improve accuracy.
The team found that standard models struggled with telephone-quality audio and mixed-language speech. By testing various models, they identified a specialized version of Whisper-medium that significantly outperformed the base large-v3 model on Uzbek language samples. The system swaps models in memory within 0.9 seconds, fitting within a 16 GB VRAM constraint.
A dedicated filtering layer was created to remove hallucinations, which are common in Whisper outputs. The system detects and discards segments containing echo, repetitive phrases, non-relevant scripts, or incorrect duration metadata. This engineering approach allows for high-quality speech recognition without the need for extensive model retraining or a dedicated research team.
