The paper introduces “P…IBench”, a benchmark for proactive video understanding that tests daily perception, risk prediction, and intervention in real‑life streaming scenarios. It evaluates 12 video‑language models, finding that current systems struggle to make well‑calibrated response decisions. The authors also create new proactive training data and train “P…I-Assistant-4B”, which shows uneven improvements across the benchmark’s capabilities.
arXiv:2606. 09547v1 Announce Type: cross Abstract: Learning everyday skills, like cooking a dish, relies increasingly on instructional media such as online videos.
arXiv:2610.34662v1 Announce Type: new Abstract: Vision-language models are shifting from offline passive understanding toward online proactive interaction. However, prevailing methods largely rely on...
arXiv:2607. 11523v1 Announce Type: cross Abstract: When should an intelligent assistant speak up without being asked?
When should an intelligent assistant speak up without being asked? Continuous egocentric video offers rich, evolving context that enables a new form of assistance: one that is proactive rather than merely reactive.
OneStreamer is a streaming video model that jointly learns to record evidence and respond to tasks through a shared proactive generation process. Its Proactive Hierarchical Caption Memory creates time‑grounded local‑detail captions and event summaries, while Proactive State Transition Learning reduces waiting states by supervising all output anchors. The authors also built a large OneStreamer‑1M dataset and show that a 4B model outperforms baselines on eight streaming video benchmarks, with ablations confirming the benefits of generated captions and PSTL.
arXiv:2603. 10652v3 Announce Type: replace-cross Abstract: In real-world deployment, vision-language models often encounter disturbances such as weather, occlusion, and camera motion.
EgoArgus is a new, human‑annotated dataset that tests visual‑language models (VLMs) as situational assistants in five everyday dialogue‑video scenarios. It evaluates how well VLMs understand and decide when to intervene, especially when visual and textual cues are helpful, irrelevant, or conflicting. The study finds that current VLMs still struggle to reliably act as egocentric assistants and that existing modality‑bias mitigation methods offer limited improvement.
arXiv:2609.37950v1 Announce Type: new Abstract: Video understanding agents acquire evidence through an executable harness that controls what they observe and how they use those observations. However,...
arXiv:2606. 07433v1 Announce Type: cross Abstract: Video understanding is being rapidly transformed by multimodal large language models (MLLMs), as research moves from short clips to long, multimodal, and knowledge-intensive video scenarios.
arXiv:2608.23329v1 Announce Type: cross Abstract: Open-world video understanding often requires a model to locate sparse visual evidence and acquire external knowledge that is absent from the video a...
arXiv:2606. 07639v1 Announce Type: cross Abstract: Video understanding is shifting from the offline paradigm -- taking a fully recorded video as input and producing a single answer after it ends -- toward real-time interaction, in which the model perceives new frames while still replying, revises its answer as new evidence appears, and remains silent when there is nothing to say.
Today's multimodal assistants are typically prompted to interpret the visual evidence presented to them. To offload the cognitive burden of remembering across long horizons, wearable assistants should...