Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

9,470 stories · RSS feed

arXiv AI

Jun 2

The Image Reconstruction Game: Drawing Common Ground Through Iterative Multimodal Dialogue

arXiv:2606. 01901v1 Announce Type: cross Abstract: We introduce the Image Reconstruction Game, a fully automated benchmark in which a vision-language model issues corrective instructions to an image generator across multiple turns, making accumulated common ground directly observable as a rendered image.

By Sherzod Hakimov, Mattia D'Agostini, Ivan Samodelkin, David Schlangen

llms multimodal benchmarks

arXiv AI

Jun 2

MMG2Skill: Can Agents Distill In-the-Wild Guides into Self-Evolving Skills?

arXiv:2606. 01993v1 Announce Type: cross Abstract: Abundant procedural knowledge on the Web holds great potential for helping agents solve long-horizon tasks.

By Xinyu Che, Junqi Xiong, Yunfei Ge, Xinping Lei, Shihao Li, Hang Yan, Han Li, Yuanxing Zhang, Zhiqi Bai, Jinhua Hao, Ming Sun, Han Li, Jiaheng Liu

llms agents multimodal benchmarks

arXiv AI

Jun 2

Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluation

arXiv:2310. 15676v2 Announce Type: replace-cross Abstract: Multi-modal 3D Intelligence has gained considerable attention due to its wide applications in autonomous driving and world simulation, etc.

By Yinjie Lei, Zixuan Wang, Feng Chen, Guoqing Wang, Peng Wang, Yang Yang

agents robotics multimodal benchmarks

arXiv AI

Jun 2

Catch-Only-One: Non-Transferable Examples for Model-Specific Authorization

arXiv:2510. 10982v2 Announce Type: replace-cross Abstract: Recent AI regulations increasingly emphasize the need for mechanisms that preserve the utility of data for AI innovation while preventing misuse, particularly by enforcing purpose limitation in downstream AI applications.

By Zihan Wang, Zhiyong Ma, Zhongkui Ma, Shuofeng Liu, Akide Liu, Derui Wang, Minhui Xue, Guangdong Bai

llms multimodal benchmarks safety

arXiv AI

Jun 2

TrafficRAG: A Multimodal RAG Framework for Traffic Accident Liability Determination

arXiv:2606. 01737v1 Announce Type: new Abstract: Traffic accident liability analysis is a critical yet challenging task in intelligent transportation and legal assistance.

By Xu Li, Zedong Fu, Xinyi Li, Xun Han

llms rag multimodal

arXiv AI

Jun 2

Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Support

arXiv:2606. 00991v1 Announce Type: new Abstract: Transportation systems management and operations (TSMO) increasingly depends on timely interpretation of heterogeneous data, from various sensor streams, incident reports, traveler feedback, and visual observations.

By Siyan Li, Zehao Wang, Jiachen Li, Kanok Boriboonsomsin, Matthew J. Barth, Guoyuan Wu

llms efficiency multimodal benchmarks

arXiv AI

Jun 2

Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation

arXiv:2606. 00095v1 Announce Type: cross Abstract: Vision-Language Navigation (VLN) enables embodied agents to reach target locations in unseen environments by following language instructions.

By Kailing Li, Tianwen Qian, Lijin Yang, Yuqian Fu, Jingyu Gong, Xiaoling Wang, Liang He

llms agents robotics multimodal benchmarks

arXiv AI

Jun 2

Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization

arXiv:2604. 08324v3 Announce Type: replace-cross Abstract: Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) through combinatorial search over symbolic structures.

By Benjamin L\'eger, Kazem Meidani, Christian Gagn\'e

diffusion multimodal safety

arXiv AI

Jun 2

Prior Availability in Industrial Visual Sim-to-Real: A Review of CAD-Guided and CAD-Unavailable Regimes

arXiv:2605. 30581v2 Announce Type: replace-cross Abstract: Industrial visual sim-to-real is often described as transferring from synthetic images to real images, but industrial deployment usually involves a broader mismatch between available evidence and required decisions.

By Chenxi Tao, Seung-Kyum Choi

computer-vision robotics multimodal benchmarks

arXiv Machine Learning

Jun 2

Escaping the Mode Lottery: Multi-Response Training Improves Language Model Generalization

arXiv:2606. 00544v1 Announce Type: new Abstract: Modern language-model fine-tuning typically pairs each prompt with a single response, even though many prompts admit multiple valid completions.

By Hasan Amin, Kian Ahrabian, Ming Yin, Rajiv Khanna

llms fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 2

LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models

arXiv:2606. 00573v1 Announce Type: new Abstract: Vision-language models (VLMs) deliver strong multimodal reasoning capabilities, but their large computational cost and high parameter counts make deployment challenging on resource-constrained devices.

By Haiyu Wang, Yutong Wang, Leshu Li, Yihui Ren, Sai Qian Zhang

llms efficiency multimodal

arXiv AI

Jun 2

Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTS

arXiv:2605. 30748v2 Announce Type: replace-cross Abstract: We present Chatterbox-Flash, a zero-shot text-to-speech model obtained by fine-tuning a pretrained autoregressive TTS decoder into a block-diffusion decoder, enabling parallel token generation within each block while retaining block-by-block streaming.

By Deokjin Seo, Gangin Park, Kihyun Nam

diffusion fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 2

TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning

arXiv:2606. 01591v1 Announce Type: cross Abstract: The TimeLogic Challenge evaluates formal temporal-logic reasoning over video - 16 operators (before, after, until, since, always, co-occur, ordering, ...

By Ali Alavi

llms nlp multimodal benchmarks

arXiv Machine Learning

Jun 2

Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioning

arXiv:2606. 00851v1 Announce Type: cross Abstract: Empathetic spoken dialogue systems must infer a user's emotional state to respond appropriately, yet everyday speech often carries weak, neutral, or ambiguous affective cues.

By Sukru Samet Dindar, Riki Shimizu, Xilin Jiang, Nima Mesgarani

multimodal safety

arXiv Machine Learning

Jun 2

Modeling Robotics Dataset Construction as an Artifact-Based Build Process

arXiv:2606. 00162v1 Announce Type: cross Abstract: Robotic systems generate large volumes of multimodal sensor data, but converting ROS bag recordings into machine learning datasets is often handled by ad hoc sequential scripts, creating engineering overhead and slow iteration cycles.

By Leon Pohl, Lukas Beer, George Sebastian, Mirko Maehlisch

robotics multimodal

arXiv Machine Learning

Jun 2

PaintBench: Deterministic Evaluation of Precise Visual Editing

arXiv:2606. 00188v1 Announce Type: cross Abstract: While current multimodal models are proficient at open-ended visual editing, executing precise single-answer edits remains an important obstacle.

By Kai Xu, Ellis Brown, Shrikar Madhu, Rob Fergus, He He, Saining Xie

robotics multimodal benchmarks safety

arXiv AI

Jun 2

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention

arXiv:2601. 21444v2 Announce Type: replace-cross Abstract: The efficiency of long-video inference remains a critical bottleneck, mainly due to the dense computation in the prefill stage of Large Multimodal Models (LMMs).

By Yuxiang Huang, Mingye Li, Xu Han, Chaojun Xiao, Weilin Zhao, Ao Sun, Ziqi Yuan, Hao Zhou, Fandong Meng, Zhiyuan Liu

rag multimodal

arXiv Machine Learning

Jun 2

Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs

arXiv:2606. 00813v1 Announce Type: cross Abstract: Safety alignment in LLMs does not improve monotonically across model generations.

By Subhadip Mitra

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 2

Feature Alignment Determines Fusion Strategy: A Comparative Study of Cross-Attention and Concatenation in Multimodal Learning

arXiv:2606. 01207v1 Announce Type: cross Abstract: The choice between cross-attention and concatenation for multimodal fusion remains governed by practitioner intuition rather than principled understanding.

By Zhiqiang Zhou, Xuezhen Xie

llms multimodal safety

arXiv Machine Learning

Jun 2

Simple Recipe Works: Vision-Language-Action Models are Natural Continual Learners with Reinforcement Learning

arXiv:2603. 11653v2 Announce Type: replace Abstract: Continual Reinforcement Learning (CRL) for Vision-Language-Action (VLA) models is a promising direction toward self-improving embodied agents that can adapt in openended, evolving environments.

By Jiaheng Hu, Jay Shim, Chen Tang, Yoonchang Sung, Bo Liu, Peter Stone, Roberto Martin-Martin

agents reinforcement-learning robotics fine-tuning multimodal benchmarks

Get the daily briefing