The paper presents a lightweight framework for reliable egocentric action anticipation that tackles two failure modes caused by temporal corruption: unreliable temporal evidence during encoding and implausible verb‑noun compositions during decoding. It introduces Temporal Reliability Suppression (TRS), which predicts per‑frame suppression scores to penalize encoder attention and weight temporal pooling, and Robust Verb‑Noun Graph (RVG), which re‑ranks verb‑noun pairs using a PMI‑based compatibility graph derived from training labels. Experiments with corruption‑augmented training show that TRS+CA+RVG achieves 29.1% average corrupted accuracy and 88.2% relative robustness across six corruptions, while dramatically reducing rare verb‑noun predictions from 15.4% to 1.1%. whyItMatters":"The approach improves the robustness of wearable action anticipation systems to real‑world challenges such as missing frames, masking, and sensor noise, which are common in egocentric video settings."
arXiv:2608. 04510v1 Announce Type: cross Abstract: Diffusion-based vision-language-action (VLA) policies can generate plausible actions even when their predictions are weakly grounded in the visual and language evidence defining the task.
arXiv:2605.29858v3 Announce Type: replace Abstract: Temporal action localization (TAL) requires recognizing the target event and localizing its start and end times precisely in untrimmed videos. Rece...
arXiv:2604. 00767v2 Announce Type: replace Abstract: Wearable human activity recognition (HAR) has made steady progress, yet much of this progress remains grounded in fixed-window, closed-set classification benchmarks.
arXiv:2605.00913v2 Announce Type: replace-cross Abstract: Body-worn sensor networks capture human activities through complementary views, where nodes, modalities, and axes partially describe the unde...
The paper investigates how different action tokenization methods affect closed‑loop control in autoregressive vision‑language‑action models. It compares analytical, linear, and nonlinear representations, showing that lower reconstruction error does not guarantee better policy performance. The study highlights the need to evaluate tokenization on multiple criteria, including sequence predictability and decoder stability, rather than relying solely on reconstruction fidelity.
arXiv:2608. 16697v1 Announce Type: new Abstract: Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heterogeneous architectures.
The paper introduces REVEAL, a diagnostic benchmark that stresses Video‑Language Models (VidLMs) on five controlled probes—camera‑motion sensitivity, cross‑frame integration, video sycophancy, language‑only shortcuts, and temporal expectation bias—to assess how well these models encode and use visual evidence. Experiments on 12 VidLMs reveal systematic failures: some visual signals are never reliably encoded, while others are overridden by model priors, leading to performance below chance on several probes that humans solve with high accuracy. Mechanistic probes further pinpoint where and why visual evidence is lost, demonstrating that under assertive prompts a model’s output becomes nearly invariant to real versus random video input, rendering visual evidence causally inert.
arXiv:2606. 03954v1 Announce Type: cross Abstract: As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not.
As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount -- physical actions carry immediate and irreversible consequences that digital errors do not. We introduce the Vision-Language Embodied Safety Agent (VLESA), a framework that monitors human activities from egocentric video and triggers real-time safety interventions when dangerous actions are predicted.
The paper investigates whether modern video‑language models can reliably understand temporal dynamics in driving scenarios. Using controlled ego‑motion tasks, the authors find that various prompting strategies are inconsistent and heavily biased, while newer models only show modest few‑shot improvements. However, lightweight LoRA fine‑tuning and probes on intermediate visual features can recover strong motion discrimination, suggesting that motion cues exist within the models but are not effectively leveraged by the final video‑to‑language pathway.
ActionPiece rethinks how actions are tokenized for autoregressive vision‑language‑action models by introducing physical rank consistency (PRC) to evaluate relational fidelity of reconstructed actions. The method jointly supervises representation learning and quantization to preserve local physical distance rankings, improving both PRC and policy success. Experiments on LIBERO, LIBERO‑Plus, SimplerEnv, and VLA‑Arena show significant gains over baseline tokenizers.
arXiv:2608.16697v2 Announce Type: replace Abstract: Vision-Language-Action models (VLAs) integrate visual perception, language instruction, and action generation into end-to-end policies across heter...