arXiv:2510. 01483v3 Announce Type: replace-cross Abstract: Vision-language models (VLMs) demonstrate strong image-level scene understanding, but reasoning over long egocentric video remains costly: because VLMs maintain no persistent memory or explicit spatial representation, all sampled frames must be re-processed for every new query.
arXiv:2606. 05677v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have advanced image and video understanding and can increasingly handle longer visual inputs.
The paper introduces Caption‑once, Frames‑on‑Demand (CFD), a budget‑aware edge‑cloud framework for long‑video understanding. CFD first runs a single offline captioning pass on the edge to build a dual‑track narrative index—an event‑level story skeleton and a clip‑level micro‑log—that is cached for future queries. At query time, a cloud‑side MLLM uses a Visual‑Need Router to decide whether to retrieve keyframes for perceptual questions, thereby limiting visual processing while preserving temporal structure in language space.
arXiv:2608. 04765v1 Announce Type: cross Abstract: Vision-language-action (VLA) models provide a unified paradigm for connecting visual perception, language understanding, and robotic control.
Building memory is essential for long-horizon planning in zero-shot embodied navigation. Detector-centric scene graphs often compress observations into sparse nodes, discarding fine-grained visual evidence and accumulating noise, while 3D reconstruction-based methods remain computationally prohibitive.
Metric-Bench introduces a new benchmark for Vision‑Language Models (VLMs) that focuses on metric‑spatial reasoning in indoor scenes by using in‑image reference objects with known dimensions. The accompanying MetricReasoner fine‑tuning recipe employs structured prompts and numerical rewards to implicitly learn 2D‑to‑3D mapping without camera intrinsics. Experiments show that this approach improves spatial metric understanding by 43.1 % over existing models and boosts downstream embodied tasks, while also delivering gains on general VLM benchmarks.
arXiv:2607. 02959v1 Announce Type: cross Abstract: We introduce VSeek, an agentic framework that transforms long-video question answering (LVQA) from a passive, single-pass perception task into a multi-turn retrieval process.
arXiv:2606.20092v3 Announce Type: replace Abstract: Memory remains a critical bottleneck for long-horizon robotic manipulation, as standard Vision-Language-Action (VLA) policies often fail when task-...
LT-Mem introduces a volatility‑aware memory evolution framework for lifelong scene understanding, combining spatially aligned instance‑level 3D perception with temporal reasoning. It uses a multi‑session SLAM backbone, a reasoning layer that scores evidence and selects memory actions, and a Tri‑Memory structure (Live, Delta, Meta) to preserve current states and event histories. The accompanying LT‑VQA dataset provides multi‑session recordings, persistent identity annotations, and temporal QA pairs, and experiments show LT‑Mem outperforms baselines while using far fewer tokens.
The paper introduces inr2action, a memory framework that encodes entire robot episodes as fixed‑size neural implicit representations. Each episode’s video is fitted to a shared, meta‑learned initialization, producing a weight delta that serves as memory. This delta is projected into token embeddings to condition a language model for question answering and a decoder for command regression, enabling recall without retrieving individual frames.
STRAND is a new benchmark that tests multimodal large language models’ ability to track objects, their states, and relationships over time in videos. It evaluates intermediate reasoning by breaking queries into sub‑questions and uses Faithful Accuracy to ensure all parts of an answer are correct. The authors also propose an object‑centric framework that builds structured trajectories and shows reduced hallucinations and better temporal consistency compared to existing models.
FocusGraph is a framework for selecting keyframes in egocentric long‑video question answering. It uses a lightweight Scene‑Graph LLM Selector to identify query‑relevant clips from graph‑based captions, then extracts keyframes with Patch‑wise Sparse‑Flow Retention (PSFR) before feeding them to a multimodal large language model for answer generation. The method achieves state‑of‑the‑art performance on FindingDory and HourVideo while reducing question‑time inference cost.