StreamTTT is a streaming vision-language model that balances real-time perception with long-term memory by writing long-range history into fast weights outside the attention context, while keeping a short sliding key-value cache for recent evidence. The model is trained on both offline long-video QA and a new real-time QA corpus, and it outperforms SimpleStream-4B on OVO-Bench by 1.4 points in real-time perception and 3.7 points in backward tracing. StreamTTT-4B also competes with the larger SimpleStream-8B on the StreamingBench Real-Time Visual Understanding subset.
arXiv:2610.01192v1 Announce Type: new Abstract: Streaming vision-language models must process continuously growing video streams under a bounded compute budget, creating a persistent tension between...
FastBench is a new benchmark that tests streaming Video‑Large‑Language Models (VLMs) on high‑dynamic real‑world video streams, where events occur rapidly and cannot be captured by sparse 1–2 FPS sampling. The benchmark uses a trajectory‑grounded pipeline that generates QA pairs from high‑FPS clips, filters questions answerable at 2 FPS, verifies answers with SAM3 and CoTracker3 trajectories, and incorporates three rounds of human inspection, yielding 306 QA pairs across eight domains and multiple temporal scopes. Experiments show that even the strongest model, Gemini‑3.5‑Flash, achieves only 50.7 % accuracy, and that denser sampling improves performance but saturates when history is compressed; a training‑free baseline, ProactiveFrame, outperforms uniform sampling but still falls short of oracle‑guided focusing.
The paper introduces MaLoW, a streaming memory framework that internalizes long‑term video history into evolving lightweight parameter updates using a recurrent HyperNetwork and LoRA weights. MaLoW keeps current observations as input while carrying historical context forward without re‑visiting the full video stream. The authors also present MaLoW‑Instruct, a large streaming video dataset, and show that MaLoW outperforms prior memory methods on several benchmarks and improves action success in a vision‑language‑action setting.
arXiv:2510. 09608v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) could power real-time assistants and autonomous agents, but they face a critical challenge: understanding near-infinite video streams without escalating latency and memory usage.
arXiv:2609.23601v1 Announce Type: new Abstract: Long-video understanding must capture transient visual evidence under strict token budgets, yet existing methods compress frames, append memory tokens,...
arXiv:2606. 16353v1 Announce Type: cross Abstract: Streaming video understanding models must answer queries at any moment during an ongoing stream, using only what they have observed so far and under fixed memory and computation budgets.
arXiv:2609.00291v1 Announce Type: new Abstract: Streaming video understanding requires answering questions that arrive at arbitrary moments over an unbounded video stream. Existing systems primarily...
TwinPast is a new benchmark that tests whether streaming video models use historical evidence by presenting pairs of videos that differ only in critical past events while keeping recent context, questions, and options the same. The study evaluates 34 model configurations on 350 human‑reviewed pairs across five historical tasks, finding that most models rely on recent frames or the question alone, with only a few showing improved performance when historical inputs are provided. Even the best streaming system scores only 49.14% PairAcc, far below general‑purpose models, and case studies reveal that models often select states from the wrong time or contradict the recorded event order.
The paper introduces Multi-Stream Attention Residuals (MSAR), an extension of Attention Residuals that applies multi-head depth attention inspired by multi-head sequence attention. MSAR uses multi-scale causal convolutions to provide diverse contextual views for each head, then adaptively mixes the retrieved streams into a single-stream input, avoiding persistent multi-stream storage. Across 12 downstream benchmarks at 18B and 28B MoE scales, MSAR outperforms prior residual variants and achieves a 1.58× compute speedup over the standard residual baseline.
arXiv:2610.03123v1 Announce Type: cross Abstract: Existing streaming vision-language models (VLMs) continuously perceive and reason over visual streams, but their computational pathways remain fixed...
arXiv:2607.28627v2 Announce Type: replace-cross Abstract: Long visual contexts challenge vision-language models: performance degrades as the number of distractors grows, and processing all tokens at...
arXiv:2609.05533v1 Announce Type: cross Abstract: Long-horizon manipulation is partially observable: the information needed to choose the next action may appear only in observations from minutes earl...