arXiv AI

Aligning Large Vision-Language Models at Test Time: A Trajectory-Guided Structured Sampling Approach

arXiv:2608. 03204v1 Announce Type: cross Abstract: Post-training reinforcement learning (RL) algorithms are commonly used to align large vision-language models (LVLMs) with human intent and the requirements of visual reasoning tasks.

arXiv AI

Jun 16

UniT: Unified Multimodal Chain-of-Thought Test-time Scaling

arXiv:2602. 12279v2 Announce Type: replace-cross Abstract: Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.

By Leon Liangyu Chen, Haoyu Ma, Zhipeng Fan, Ziqi Huang, Animesh Sinha, Xiaoliang Dai, Jialiang Wang, Zecheng He, Jianwei Yang, Chunyuan Li, Junzhe Sun, Chu Wang, Serena Yeung-Levy, Felix Juefei-Xu

llms agents multimodal

arXiv AI

Jul 28

CADER: Confidence-Aware Dynamic Evidence Reasoning for Long-Video Understanding

arXiv:2607. 24582v1 Announce Type: cross Abstract: Long-video understanding increasingly relies on large vision-language models and tool-augmented reasoning, but most systems apply the same inference procedure to every example regardless of difficulty.

By Jinlong Yang, Wenhao Zhang, Kuanwei Lin, Sijie Cheng

llms agents multimodal benchmarks

arXiv AI

Jun 2

Pause and Think: A Dataset and Benchmark for Video-Grounded Assistive Action Suggestion

arXiv:2606. 00616v1 Announce Type: cross Abstract: Recent Vision-Language Models (VLMs) struggle with grounded reasoning, temporal consistency, and context aware planning in videos.

By Shivam Singh, Saptarshi Majumdar, Pratik Prabhanjan, Zicheng Liu, Emad Barsoum

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 16

Think Less, Act Early: Reinforced Latent Reasoning with Early Exit in Vision-Language-Action Models

arXiv:2606. 15099v1 Announce Type: cross Abstract: Existing Vision-Language-Action (VLA) models predominantly rely on explicit Chain-of-Thought (CoT) reasoning to bridge perception and action.

By Dianqiao Lei, Lianlei Shan

reinforcement-learning robotics multimodal benchmarks safety

arXiv AI

1d ago

Deep Thought Alignment: Trajectory-Level Latent Distillation for Video Reasoning

arXiv:2608. 16316v1 Announce Type: cross Abstract: Large Multimodal Models (LMMs) for video reasoning have long been hindered by the high computational cost of processing vast amounts of visual information.

By Ao Shen, Yongheng Zhang, Yinghui Li, Manning Wang, Di Yin, Xing Sun

efficiency multimodal benchmarks safety

arXiv AI

Jun 17

See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RL

arXiv:2606. 17678v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images, indicating ineffective utilization of visual evidence during inference.

By Yilian Liu, Sicong Leng, Guoshun Nan, Junyi Zhu, Jiayu Huang, Minghao Sun, Xuancheng Zhu, Yisong Chen, Zexian Wei, Xiaofeng Tao

llms reinforcement-learning fine-tuning multimodal benchmarks safety

Hugging Face Trending Papers

Jun 23

PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought

Pointing-based visual grounding requires models to precisely locate target objects by deciphering complex spatial relationships between the visual scene and pointing gestures. Traditional methods typically encode input images into static feature representations and perform reasoning primarily within the linguistic domain, often overlooking the rich perceptual cues and explicit spatial geometry inherent in images.

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 16

DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning

arXiv:2606. 15160v1 Announce Type: cross Abstract: Reasoning capabilities of multimodal large language models (MLLMs) have improved considerably in recent years.

By David Huang, Lianlei Shan

llms diffusion reinforcement-learning efficiency multimodal benchmarks

arXiv AI

1d ago

Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

arXiv:2608. 15869v1 Announce Type: cross Abstract: Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments.

By Xiaoyu Zhu, Xinke Deng, Suresh Taddewadikar, Arnab Kumar Mondal, Zhongyu Jiang, Ian Fasel, Joerg Liebelt

llms rag robotics fine-tuning multimodal

arXiv AI

Jun 12

ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding

arXiv:2506. 01274v2 Announce Type: replace-cross Abstract: Recent progress in Large Multi-modal Models (LMMs) has enabled effective vision-language reasoning, yet the ability to video understanding remains constrained by suboptimal frame selection strategies, albeit with the rapid development of video-specialized LMMs.

By Hosu Lee, Junho Kim, Hyunjun Kim, Yong Man Ro

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jun 30

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

arXiv:2606. 29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs).

By Peng, Lee, Yin Zhang, Yanglin Zhang, Haonan Wu, Zishan Liu, Ruoxi Zang, Xin Zhu, Jiayin Zheng, Jian Yao, Zefeng Ji, Fei Ma

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jun 17

ThinkJEPA: Empowering Latent World Models with Large Vision-Language Reasoning Model

arXiv:2603. 22281v2 Announce Type: replace-cross Abstract: Recent progress in latent world models (e.

By Haichao Zhang, Yijiang Li, Shwai He, Tushar Nagarajan, Mingfei Chen, Jianglin Lu, Ang Li, Yun Fu

llms robotics multimodal safety

Related stories