Yandex has released the source code for YTsaurus Flow, a technology designed for continuous real-time processing of large data streams. The tool enables immediate analysis of events, such as clicks, views, likes, and orders, without waiting for data accumulation.
The technology is currently deployed within Yandex services. In Yandex Ads, it handles over 100 GB of data and millions of events per second. Previously, preparing data for retraining advertising models took more than 10 hours, but YTsaurus Flow has nearly eliminated this latency.
Beyond advertising, the solution supports demand analysis in Yandex Market, fraud detection systems, and real-time application updates. It ensures fault tolerance by distributing loads across available servers and guarantees that each event is processed exactly once without duplication or loss.
Developed by the Yandex Infrastructure and Yandex Ads teams, the technology is now available under the Apache 2.0 license. It is open for use in commercial projects, allowing external developers to integrate these real-time processing capabilities into their own systems.
