arXiv Computer Vision By Niange Yu, Ye Tian, Biaolong Chen, Miao Lu, Aixi Zhang, Hao Jiang, Yunhai Tong, Pipei Huang

Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation

Read the original on arXiv Computer Vision →

The paper tackles two main issues in multi‑subject video generation—uncontrollable fidelity strength and semantic drift—by exploiting intrinsic attention patterns in Diffusion Transformers. It introduces an Intrinsic Spatial Grounding Map (ISGM) that accurately locates reference subjects and a Dual‑phase Intrinsic Attention Leveraging (DIAL) framework that uses ISGM during both training and inference. DIAL guides attention in low‑noise stages for precise fidelity control and builds preference pairs in high‑noise stages for reinforcement learning, resulting in superior identity consistency and controllable fidelity on the OpenS2V‑Eval benchmark.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

Hugging Face Trending Papers
Sep 10

Harnessing Intrinsic Subject-Aware Attention for Controllable Multi-Subject Video Generation

The paper tackles two main issues in multi-subject video generation—uncontrollable fidelity strength and semantic drift—by studying Diffusion Transformers (DiTs). It discovers that certain attention blocks naturally create an Intrinsic Spatial Grounding Map (ISGM) that accurately locates reference subjects. Leveraging this insight, the authors introduce Dual-phase Intrinsic Attention Leveraging (DIAL), which uses ISGM during low-noise stages to control fidelity strength without retraining and during high-noise stages to generate preference pairs for reinforcement learning, thereby anchoring attention and reducing semantic drift. Experiments on the OpenS2V-Eval benchmark show that DIAL outperforms baseline models, improving identity consistency and enabling controllable fidelity strength.

arXiv Computer Vision
Sep 7

Step Back to Move Forward: Reflection-Aware Preference Optimization for Visual Generation

The paper introduces Reflection-Aware GRPO (RA‑GRPO), a reinforcement‑learning framework that aligns diffusion generative models with human preferences. It uses Diffusion Reflection to correct intermediate sampling paths by reversing the diffusion process, and Counterfactual Path Synthesis to embed these corrected trajectories into the policy, avoiding extra inference cost. Experiments on text‑to‑image and text‑to‑video models show RA‑GRPO outperforms existing methods, reducing reward hacking and improving generalization while remaining architecture‑agnostic.

By Junlong Wu, Jiuzhou Lin, Jia Sun, Boheng Zhang, Huaiqing Wang, Dewen Fan, Houde Liu, Qianqian Gan, Fan Yang, Tingting Gao
arXiv AI
Jun 12

ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding

arXiv:2506. 01274v2 Announce Type: replace-cross Abstract: Recent progress in Large Multi-modal Models (LMMs) has enabled effective vision-language reasoning, yet the ability to video understanding remains constrained by suboptimal frame selection strategies, albeit with the rapid development of video-specialized LMMs.

By Hosu Lee, Junho Kim, Hyunjun Kim, Yong Man Ro
arXiv AI
Aug 28

CounterVid: Counterfactual Video Generation for Mitigating Action and Temporal Hallucinations in Video-Language Models

CounterVid introduces a scalable counterfactual video generation framework that creates videos differing only in actions or temporal structure while keeping scene context intact. The approach uses multimodal LLMs for action proposals and diffusion models for editing, producing a synthetic dataset of ~26k preference pairs for action recognition and sequence ordering. With the MixDPO optimization method, the authors demonstrate significant improvements in action recognition and temporal ordering on Qwen2.5‑VL and InternVL3 backbones, while maintaining overall video understanding.

By Tobia Poppi, Burak Uzkent, Amanmeet Garg, Lucas Porto, Garin Kessler, Yezhou Yang, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara, Florian Schiffers
arXiv Computer Vision
Sep 10

VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning

VideoTIR introduces a reinforcement‑learning approach to improve long‑video understanding by encouraging multimodal large language models to use comprehensive multi‑level toolkits efficiently. It combines Zero‑RL and SFT cold‑starting strategies to help models retrieve and focus on meaningful video segments, images, and regions, thereby reducing hallucinations. The method includes Toolkit Action Grouped Policy Optimization (TAGPO) to streamline tool‑calling and a sandbox‑based trajectory synthesis framework for high‑quality data, achieving strong results on three long‑video QA benchmarks.

By Zhe Gao, Shiyu Shen, Taifeng Chai, Weinong Wang, Haotian Xu, Xing Wu, Wenbin Li, Qi Fan, Yang Gao, Dacheng Tao