arXiv Machine Learning By Federico Spurio, Olga Zatsarynna, Lars Doorenbos, Emad Bahrami, Gianpiero Francesca, Juergen Gall

Post-Training VLMs for Video Mistake Detection

Read the original on arXiv Machine Learning →

The paper introduces a new protocol, Mistake Detection Video Question Answering (MD‑VQA), to evaluate whether models can determine if a step in a video follows its description, covering both seen and unseen actions. It proposes a post‑training approach for video‑language models that uses a reward function to highlight discrepancies between instructions and video content. Experiments show this method surpasses zero‑shot, fine‑tuned, and other post‑training baselines, especially on unseen procedures, improving performance by up to 11.6% on EP‑VQA.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computation and Language
Oct 23

Through a PRISM, Clearly: Step-Sequence Reasoning for Mistake Recognition in Procedural Videos

The paper introduces PRISM, a method for recognizing procedural mistakes in videos by grounding video segments in language and mapping them to procedural states using a lightweight hidden Markov model trained on correct executions. It aligns reference annotations with four edit-based error types—insertions, deletions, substitutions, and transpositions—to evaluate across heterogeneous benchmarks. Experiments on three datasets show that using language grounding, semantic role labeling, and a trained local decoder consistently improves mistake detection performance compared to nearest-candidate matching.

By Olga Loginova, Oleksandr Bezrukov, Frank Keller
arXiv Computer Vision
Sep 23

Test-time Reinforcement Learning for Anomalous Video Understanding

The paper introduces a test‑time reinforcement learning framework for anomalous video understanding, addressing challenges such as unreliable pseudo‑labels, inadequate reward design, and collapsed group‑relative advantages. It proposes dual‑query consistency filtering, an entropy‑aware consensus reward, and a virtual negative anchor mechanism to improve sample reliability, reward quality, and policy‑gradient signals. Experiments on VAU‑Bench demonstrate significant performance gains, especially on the ECVA subset where accuracy rises from 75.81% to 90.00%.

By Huining Li, Yuxiang Duan, Jiyang Tan, Qian Li, MingCai Chen, Jian Zhang, Xingdong Sheng, Yuntao Du
arXiv Computer Vision
Oct 28

Diagnosing and Correcting Bias in MLLM for Long Video Understanding

The paper introduces S‑MME, a benchmark for diagnosing shortcut‑biased behavior in Multimodal Large Language Models (MLLMs) on long‑video question answering. It proposes CLEAR, a training‑free framework that estimates and mitigates bias by treating the full video‑question pair as a complete view and intervening on hidden states during inference. Experiments on five benchmarks demonstrate that CLEAR improves prediction accuracy and robustness across different MLLMs.

By Xusheng Liang, Jianqiao Sun, Hao Zhang, Yulei Niu, Hengshuang Zhao, Jiawei Ma