The Kandinsky team at Sberbank has introduced Time Adapter, a lightweight module designed to improve temporal consistency in video generation. The tool allows for precise control over frame rates, ranging from 15 to 60 fps, and manages the dynamics of events within a scene. The module occupies less than 1% of the base model's size.
The development team trained the adapter using a dataset of 40,000 videos featuring diverse movement speeds, including human activities and natural phenomena. The module operates in two modes: as an add-on for pre-trained models to enhance motion smoothness, or through joint fine-tuning to alter the physics of the generated scene.
Testing on the Kandinsky 5.0 Video Lite model showed significant improvements compared to the standard version. Motion naturalness increased by 29%, visual quality by 8%, and adherence to text prompts by 19%. The research behind this approach was presented at the ICML conference.
The source code is available for free on Hugging Face under the MIT license, permitting commercial use. The technology aims to assist developers in creating synthetic data for physical AI, as well as producing advertising and cinematic content with strict tempo requirements.
