StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos

1. 1Develop a model that can process temporally incoming frames in streaming videos
2. 2Integrate human gaze signals into the model to anticipate user intention
3. 3Evaluate the model's performance on streaming benchmarks that measure temporal reasoning and gaze-guided understanding
4. 4Apply the model to realistic applications such as Augmented Reality (AR) glasses

AI engineers and researchers working on multimodal large language models (MLLMs) and computer vision can benefit from StreamGaze, as it enables proactive understanding of user intention in streaming videos

💡 StreamGaze fills the gap in streaming benchmarks by measuring the ability of MLLMs to interpret and leverage human gaze signals

📹💡 StreamGaze: Gaze-guided temporal reasoning in streaming videos