arXiv AI

Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents

arXiv:2606. 30185v1 Announce Type: new Abstract: Improving vision-language models (VLMs) on visual reasoning typically requires retraining or hand-designed prompts and tools.

Hugging Face Trending Papers

Jun 29

Dynamo: Dynamic Skill-Tool Evolution for Vision-Language Agents

Improving vision-language models (VLMs) on visual reasoning typically requires retraining or hand-designed prompts and tools. We present Dynamo, a training-free framework that adapts a frozen VLM without any weight updates.

llms agents reinforcement-learning multimodal benchmarks

arXiv AI

Jun 11

AVIS: Adaptive Test-Time Scaling for Vision-Language Models

arXiv:2606. 11576v1 Announce Type: cross Abstract: Modern Vision-Language Models (VLMs) benefit from chain-of-thought prompting and test-time scaling, but these gains often come with prohibitive inference cost due to large visual contexts and long decoding chains.

By Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni, Hakki Can Karaimer, Hue Nguyen, Iqbal Mohomed, Michael Brudno, Alex Levinshtein, Konstantinos G. Derpanis, Babak Taati, Radek Grzeszczuk

llms reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen

llms computer-vision robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 19

Vero: An Open RL Recipe for General Visual Reasoning

arXiv:2604. 04917v3 Announce Type: replace-cross Abstract: What does it take to build a visual reasoner that works across charts, science, spatial understanding, and open-ended tasks?

By Gabriel Sarch, Linrong Cai, Qunzhong Wang, Haoyang Wu, Danqi Chen, Zhuang Liu

llms reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Jun 4

Dive into the Scene: Breaking the Perceptual Bottleneck in Vision-Language Decision Making via Focus Plan Generation

arXiv:2606. 04046v1 Announce Type: cross Abstract: In embodied vision-language decision making tasks such as robotic manipulation and navigation, Vision-Language and Vision-Language-Action Models (VLMs & VLAs) are powerful tools with different benefits: VLMs are better at long-term planning, while VLAs are better at reactive control.

By Boyuan Xiao, Bohong Chen, Yumeng Li, Ji Feng, Yao-Xiang Ding, Kun Zhou

robotics multimodal benchmarks safety

arXiv AI

Jun 3

MUSE: A Unified Agentic Harness for MLLMs

arXiv:2606. 03005v1 Announce Type: cross Abstract: Despite rapid progress, multimodal large language models (MLLMs) still fail on tasks that humans solve effortlessly, such as navigating a grid maze from a screenshot or selecting the correct puzzle piece.

By Jianglin Lu, Hailing Wang, Xu Ma, Qihua Dong, Mingyuan Zhang, Yizhou Wang, Yun Fu

llms agents multimodal benchmarks

arXiv AI

2d ago

Evolve Vision-Language-Action Model into an Agent with On-the-fly Tool-use

arXiv:2608. 14047v1 Announce Type: cross Abstract: This paper integrates end-to-end Visual-Language-Action (VLA) models with agentic tool-use to propose Agentic Robot with Tool-use (ART).

By Yi Ding, Yanzhao Yu, Xili Dai, Xianbiao Qi, Peiwen Sun, Xueqian Wang, Xiangyu Yue, Jianan Wang

agents robotics multimodal

arXiv AI

Aug 5

SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them

arXiv:2607. 27703v2 Announce Type: replace Abstract: Vision-language models (VLMs) are increasingly used in embodied agents to interpret visual inputs, reason about spatial relationships, and make task-level decisions based on that reasoning.

By Yang Zhou, Zixuan Huang, Sunzhu Li, Zhuo Yang, Chen Zhang, Shunian Chen, Caijun Yan, Jianyao Xu, Shunyu Liu, Weijie Fu, Peiliang Li, Xiaozhi Chen, Yuxiang Cai

llms agents computer-vision reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Jun 9

NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoning

arXiv:2602. 21172v3 Announce Type: replace Abstract: Vision-Language-Action (VLA) models are advancing autonomous driving by replacing modular pipelines with unified end-to-end architectures.

By Ishaan Rawal, Shubh Gupta, Yihan Hu, Wei Zhan

llms agents robotics fine-tuning multimodal safety

arXiv AI

Jul 1

Can VLMs Reason Robustly? A Neuro-Symbolic Investigation

arXiv:2603. 23867v2 Announce Type: replace-cross Abstract: Vision-Language Models (VLMs) have been applied to a wide range of reasoning tasks, yet it remains unclear whether they can reason robustly under distribution shifts.

By Weixin Chen, Antonio Vergari, Han Zhao

llms fine-tuning multimodal

arXiv AI

Jun 4

Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning

arXiv:2605. 14054v2 Announce Type: replace Abstract: Achieving robust perception-reasoning synergy is a central goal for advanced Vision-Language Models (VLMs).

By Haozhe Wang, Qixin Xu, Changpeng Wang, Taofeng Xue, Chong Peng, Wenhu Chen, Fangzhen Lin

llms agents reinforcement-learning multimodal

arXiv Machine Learning

Jun 11

Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering

arXiv:2606. 12299v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically similar instructions can induce drastically different behaviors, while some capabilities may not be elicitable through prompting alone.

By Hyun Joe Jeong, Gokul Swamy, Andrea Bajcsy

llms robotics fine-tuning multimodal

Related stories