arXiv Machine Learning September 23, 2026 By Jian Xu, Delu Zeng, John Paisley

Intervention, Not Shared Latents: Blocking Visual Shortcuts in Audio-Video Generation

Read the original on arXiv Machine Learning →

The paper investigates why joint audio–video generators often learn to predict sound from visual appearance rather than from the underlying event, a problem termed the visual shortcut. By constructing a controlled causal model where audio is independent of video appearance, the authors show that common remedies such as shared latent spaces fail to prevent this shortcut. They propose that intervening on the nuisance appearance is necessary and sufficient for counterfactual invariance, and validate this approach across synthetic and real datasets, highlighting the remaining challenge of unknown nuisances.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

diffusion

arXiv Machine Learning

Sep 22

Common Cause, Not Cross-Attention: Blocking Visual Shortcuts in Audio-Video Generation

The paper investigates how joint audio–video generation models learn to associate sound with visual appearance rather than the underlying causal event, a problem termed the visual shortcut. By constructing a structural causal model where audio is independent of video appearance, the authors demonstrate that cross‑attention and shared latent approaches fail when appearance‑event correlations are broken, and that common‑cause routing does not solve the issue. They propose blocking the shortcut via interventions on nuisance variables, proving that counterfactual invariance is necessary and sufficient for identifying the causal predictor, and validate this approach on synthetic and real datasets, including a pretrained video‑to‑audio generator.

By Jian Xu, Delu Zeng, John Paisley, Qibin Zhao

diffusion

arXiv Machine Learning

1d ago

Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs

arXiv:2609.36798v1 Announce Type: cross Abstract: Omni-modal large language models (LLMs) are expected to answer a question using the modality it explicitly refers to. However, existing training para...

By Yueran Ma, Ronghao Lin

llms nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Sep 1

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

arXiv:2608.29996v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve strong performance across many multimodal tasks; however, they often exploit spurious object-background...

By Aditi Sarker, Nazreen Shah, Rafi Ibn Sultan, Rhongho Jang, Dongxiao Zhu, Prashant Khanduri

llms multimodal benchmarks safety

Hugging Face Trending Papers

Aug 30

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

Large Vision-Language Models (LVLMs) achieve strong performance across many multimodal tasks; however, they often exploit spurious object-background correlations, resulting in predictions driven by co...

llms multimodal benchmarks safety

arXiv Machine Learning

Aug 12

Causal Variational Deep Embedding: A Family of Interventional Generators for Confounded Images

arXiv:2606. 21806v2 Announce Type: replace Abstract: Deep generative models reproduce the observational distribution of their training data, inheriting any spurious associations it contains.

By Jingyuan Chen, Kangrui Ruan, Junzhe Zhang

rag benchmarks

arXiv AI

Sep 21

Towards the Vision-Sound-Language-Action Paradigm: The HEAR Framework for Sound-Centric Manipulation

arXiv:2603.16086v2 Announce Type: replace-cross Abstract: While recent Vision-Language-Action (VLA) models have begun to incorporate audio, they typically treat sound as static pre-execution prompts...

By Chang Nie, Tianchen Deng, Guangming Wang, Zhe Liu, Hesheng Wang

rag agents robotics multimodal benchmarks

One email a morning, machine-written

Related stories

Common Cause, Not Cross-Attention: Blocking Visual Shortcuts in Audio-Video Generation

Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

Causal Variational Deep Embedding: A Family of Interventional Generators for Confounded Images

Towards the Vision-Sound-Language-Action Paradigm: The HEAR Framework for Sound-Centric Manipulation