arXiv:2610.28323v1 Announce Type: new Abstract: Streaming video understanding requires a model to decide not only what to answer but also when to answer. Evidence arrives progressively, and recent be...
arXiv:2609.37559v1 Announce Type: new Abstract: To serve as real-world personal assistants, streaming video models need persistent memory that retains past experiences for later use. Yet existing str...
FastBench is a new benchmark that tests streaming Video‑Large‑Language Models (VLMs) on high‑dynamic real‑world video streams, where events occur rapidly and cannot be captured by sparse 1–2 FPS sampling. The benchmark uses a trajectory‑grounded pipeline that generates QA pairs from high‑FPS clips, filters questions answerable at 2 FPS, verifies answers with SAM3 and CoTracker3 trajectories, and incorporates three rounds of human inspection, yielding 306 QA pairs across eight domains and multiple temporal scopes. Experiments show that even the strongest model, Gemini‑3.5‑Flash, achieves only 50.7 % accuracy, and that denser sampling improves performance but saturates when history is compressed; a training‑free baseline, ProactiveFrame, outperforms uniform sampling but still falls short of oracle‑guided focusing.
TRACE (Temporal Audit and Condition-aware Evaluation) is a new benchmark and evaluation framework for streaming video understanding that explicitly records when evidence becomes valid, how visual history is maintained, and how responses are triggered. It combines temporally audited visual tasks, evidence timing, instruction-dependent trigger annotations, a unified causal Core–Adapter protocol, and multidimensional reporting of answer quality, timeliness, response-selection behavior, workload, completion, and reliability. Using 1,240 records from 517 videos, TRACE evaluated eight publicly available models in eight configurations, revealing that similar QA accuracy can hide significant differences in completion, answer validity, generation workload, and proactive performance metrics such as response delay, false alarms, and missed target windows.
arXiv:2606. 06991v1 Announce Type: cross Abstract: Online Video Large Language Models (Video-LLMs) have advanced toward seamless human-AI interaction through frame-by-frame processing and proactive responding.
arXiv:2606. 16353v1 Announce Type: cross Abstract: Streaming video understanding models must answer queries at any moment during an ongoing stream, using only what they have observed so far and under fixed memory and computation budgets.
Realtime-Bench is a new benchmark that evaluates whether frontier models can handle real‑time interactive tasks where delayed actions directly impact success. The suite includes dynamic tasks that advance under different time‑clock regimes—Wall Clock, Paused Clock, and Token Clock—to isolate serving latency from a model’s temporal reasoning ability. Results show that even the best models achieve only 44% success under real‑time constraints, highlighting the challenge of balancing decision quality with timely execution.
arXiv:2510. 09608v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) could power real-time assistants and autonomous agents, but they face a critical challenge: understanding near-infinite video streams without escalating latency and memory usage.
ProactiveBench evaluates streaming video models on their ability to interact proactively, rather than reactively. It tests models at one‑second intervals without explicit cues, using six subtasks that vary trigger ambiguity, timing tolerance, and response patterns. The benchmark measures both response and silence rates, distinguishing early, in‑window, and missed responses, and penalizes omissions and repetitions.
arXiv:2606. 17798v1 Announce Type: cross Abstract: Despite the remarkable progress of Video Large Language Models (Video-LLMs), current online architectures still struggle to simultaneously process continuous video streams, decide autonomously when to respond, and preserve long-horizon contextual memory.
The paper introduces Watch-Think-Interact (WTI), a closed-loop framework for multi-question streaming video reasoning that maintains compact natural-language memory entries linked to video time ranges. WTI decides whether to answer, continue watching, or recall relevant past intervals for each question, avoiding replay of the full history. The authors build a large dataset, WTI-82K, and a training method, Stream-GDPO, achieving state‑of‑the‑art performance on StreamingBench and OVO-Bench.
StreamTTT is a streaming vision-language model that balances real-time perception with long-term memory by writing long-range history into fast weights outside the attention context, while keeping a short sliding key-value cache for recent evidence. The model is trained on both offline long-video QA and a new real-time QA corpus, and it outperforms SimpleStream-4B on OVO-Bench by 1.4 points in real-time perception and 3.7 points in backward tracing. StreamTTT-4B also competes with the larger SimpleStream-8B on the StreamingBench Real-Time Visual Understanding subset.