A research team led by Google has developed StreamVibe, a video agent designed to determine when to provide assistance during tasks. The system aims to interpret vague user requests and adapt to changing instructions, unlike existing assistants that primarily respond to explicit commands.
To evaluate the agent, researchers created a benchmark containing approximately 4,100 examples derived from over 183 hours of video. The training process involved Gemini 2.5 Flash for initial suggestions and reinforcement learning to minimize unnecessary interruptions.
StreamVibe-VL, a model based on Alibaba's Qwen3.5-9B, utilizes a dedicated memory system to track task progress. Data shows that StreamVibe-VL improves response timing and quality by 10.7 percent, while the memory system increases proactive interaction by 12.3 percent.
Testing revealed that performance across all models declines significantly during continuous operation, with content metrics dropping by up to 61.5 percent in longer videos. Models struggle to maintain context in sequences exceeding five minutes or when instructions change over time.
