arXiv AI August 28, 2026 By Tobia Poppi, Burak Uzkent, Amanmeet Garg, Lucas Porto, Garin Kessler, Yezhou Yang, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara, Florian Schiffers

CounterVid: Counterfactual Video Generation for Mitigating Action and Temporal Hallucinations in Video-Language Models

Read the original on arXiv AI →

CounterVid introduces a scalable counterfactual video generation framework that creates videos differing only in actions or temporal structure while keeping scene context intact. The approach uses multimodal LLMs for action proposals and diffusion models for editing, producing a synthetic dataset of ~26k preference pairs for action recognition and sequence ordering. With the MixDPO optimization method, the authors demonstrate significant improvements in action recognition and temporal ordering on Qwen2.5‑VL and InternVL3 backbones, while maintaining overall video understanding.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 8

Enhancing Video Representations with Spatiotemporal-Semantic Residual to Mitigate Hallucinations in Video Large Multimodal Models

arXiv:2601. 22574v2 Announce Type: replace-cross Abstract: Although Video Large Multimodal Models have achieved strong performance in video understanding, they still suffer from hallucination.

By Yuansheng Gao, Jinman Zhao, Tong Zhang, Xingguo Xu, Wenbin Xing, Han Bao, Zonghui Wang, Wenzhi Chen

efficiency multimodal safety

arXiv AI

Jun 11

MultiToP: Learning to Patch Visual Tokens to Mitigate Hallucinations in Video Large Multimodal Models

arXiv:2606. 11792v1 Announce Type: cross Abstract: Video Large Multimodal Models have achieved remarkable progress in video understanding, yet they remain prone to hallucinations, where generated responses are not faithfully supported by the input video.

By Yuansheng Gao, Wenbin Xing, Jiahao Yuan, Kaiwen Zhou, Han Bao, Zonghui Wang, Wenzhi Chen

efficiency multimodal safety

arXiv AI

Jun 8

MACD: Model-Aware Contrastive Decoding via Counterfactual Data

arXiv:2602. 01740v3 Announce Type: replace Abstract: Video language models (Video-LLMs) are prone to hallucinations, generating plausible but ungrounded content when visual evidence is weak, ambiguous, or biased.

By Qixin Xiao, Kun Zhou

llms safety

arXiv AI

1d ago

Bernini: Latent Semantic Planning for Video Diffusion

Bernini proposes a unified framework that separates semantic planning and pixel rendering for video generation and editing. An MLLM-based planner predicts target semantics in ViT embedding space, while a DiT-based renderer synthesizes pixels conditioned on this plan, text features, and source VAE features for editing. The approach introduces Segment-Aware 3D Rotary Positional Embedding and chain-of-thought reasoning, achieving state‑of‑the‑art performance on diverse video benchmarks.

By Bernini Team, Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan

llms rag diffusion multimodal benchmarks

arXiv AI

Aug 7

Reducing Hallucination in Vision-Language Models via Stage-wise Preference Optimization under Distribution Shift

arXiv:2605. 16411v2 Announce Type: replace-cross Abstract: Hallucination remains a fundamental challenge in vision-language models (VLMs), where autoregressive generation may produce linguistically plausible yet physically inconsistent or visually ungrounded responses due to likelihood maximization under joint probabilistic modeling.

By Qinwu Xu

llms computer-vision reinforcement-learning multimodal benchmarks safety

arXiv Machine Learning

Jun 30

Vision-driven Preference Synthesis for Mitigating Hallucinations in VLMs

arXiv:2606. 28401v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have shown strong performance in visual understanding, yet they still suffer from hallucinations, generating content that is not grounded in the image.

By Yunhun Nam, Jongheon Jeong

llms computer-vision multimodal benchmarks safety

One email a morning, machine-written

Related stories

Enhancing Video Representations with Spatiotemporal-Semantic Residual to Mitigate Hallucinations in Video Large Multimodal Models

MultiToP: Learning to Patch Visual Tokens to Mitigate Hallucinations in Video Large Multimodal Models

MACD: Model-Aware Contrastive Decoding via Counterfactual Data

Bernini: Latent Semantic Planning for Video Diffusion

Reducing Hallucination in Vision-Language Models via Stage-wise Preference Optimization under Distribution Shift

Vision-driven Preference Synthesis for Mitigating Hallucinations in VLMs