Yandex has released its Alice AI Search Pretrain model as an open-source project. This language model serves as the foundation for the quick answer feature within the company's search engine. The model was trained from scratch and is designed to operate efficiently with limited computational resources while handling large document volumes.
The model utilizes a hybrid architecture combining Encoder-Decoder and Mixture of Experts (MoE) technologies. The Encoder-Decoder structure separates information retrieval from response generation. The MoE approach ensures efficiency: out of 35 billion total parameters, only 600 million are activated per token, representing less than 2% of the model's capacity.
Internal blind testing indicates that Alice AI Search Pretrain outperforms models such as Alibaba Qwen 3.5 2B, 4B, and Google T5 Gemma 2 4B-4B Base. Its performance is comparable to the more resource-intensive Qwen 3.5 35B-A3B. Yandex uses a fine-tuned version of this model in its search engine, incorporating data from actual user interactions.
