Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,022 stories · RSS feed

arXiv AI

Jul 15

Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

arXiv:2509. 22415v3 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved strong vision-language performance, yet their token-level visual evidence remains difficult to inspect.

By Jiawei Liang, Jianjie Huang, Ruoyu Chen, Xianghao Jiao, Siyuan Liang, Shiming Liu, Xiaochun Cao

llms multimodal

arXiv AI

Jul 15

A Learning-Rate-Gated Failure of GRPO in a Small Language and Vision-Language Model Web Agent: A Controlled Null and Its Mechanism

arXiv:2607. 12640v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards, and Group Relative Policy Optimization (GRPO) in particular, is now run routinely on a supervised checkpoint in the hope of producing a stronger agent.

By Chengguang Gan, Zhixi Cai, Yunhao Liang, Hanjun Wei, Shiwen Ni, Qinghao Zhang

llms rag agents reinforcement-learning multimodal

arXiv AI

Jul 15

1D-Bench: A Benchmark for Iterative UI Code Generation with Visual Feedback in Real-World

arXiv:2602. 18548v2 Announce Type: replace-cross Abstract: Design-to-code translates high-fidelity UI designs into executable front-end implementations, but progress remains hard to compare due to inconsistent datasets, toolchains, and evaluation protocols.

By Qiao Xu, Yipeng Yu, Chengxiao Feng, Xu Liu

reinforcement-learning multimodal benchmarks

arXiv AI

Jul 15

The Computational Basis of Confidence in Large Language Models

arXiv:2607. 12447v1 Announce Type: cross Abstract: Reliable confidence -- the probability that a model's own answer is correct -- is essential for the trustworthy deployment of language models.

By Dharshan Kumaran, Viorica Patraucean, Maks Ovsanikov, Petar Veli\v{c}kovi\'c, Nathaniel Daw

llms multimodal

arXiv AI

Jul 15

Hallo4D: Multi-Modal Hallucination Mitigation for Consistent Spatio-Temporal Generation

arXiv:2607. 12752v1 Announce Type: cross Abstract: While recent advances in 3D generation have enabled impressive visual synthesis, existing methods often rely on 2D diffusion supervision without explicit mechanisms for geometric consistency, leading to spatial hallucinations such as duplicated structures and misaligned geometry.

By Hongbo Wang, Huaibo Huang, Jie Cao, Jin Liu, Haoyang Tong, Ran He

llms diffusion efficiency multimodal safety

arXiv AI

Jul 15

Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?

arXiv:2607. 12787v1 Announce Type: new Abstract: Recent advances in multimodal large language models (MLLMs) have significantly improved the performance of multimodal emotion recognition (MER) and enabled interpretable description generation by jointly modeling video, audio, and language, etc.

By Kaiwen Zheng, Junchen Fu, Wenhao Deng, Hu Han, Joemon M. Jose, Xuri Ge

llms robotics efficiency multimodal benchmarks safety

arXiv Machine Learning

Jul 15

Hierarchical Synthetic Tabular Data Generation: A Hybrid Top-Down and Bottom-Up Framework

arXiv:2605. 28198v2 Announce Type: replace Abstract: Existing approaches for synthetic tabular data generation are based on either purely generative models or LLMs, both of which struggle with data heterogeneity, logical consistency, rare-event coverage, and robustness in low-data regimes.

By Junfeng Nie, Alvin Jin, Xiaohui Chen

llms multimodal benchmarks safety

arXiv AI

Jul 15

FormalAnalyticGeo: A Neural-Symbolic Based Framework for Multimodal Analytic Geometry Problem Generation

arXiv:2607. 12982v1 Announce Type: new Abstract: Math reasoning has achieved significant progress with the rapid advancement of Multimodal Large Language Models (MLLMs), however analytic geometry remains largely underexplored, primarily due to the scarcity of annotated samples.

By Ruoran Xu, Wending Gao, Qiufeng Wang

llms multimodal

arXiv AI

Jul 15

Visual Access Boundaries in Vision-Language Model Reasoning

arXiv:2607. 12815v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting is widely used as a test-time scaling strategy for Vision-Language Models (VLMs), but it remains unclear what is extended when VLMs generate longer reasoning traces.

By Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi, Kai Yamashita, Masahiro Suzuki, Yutaka Matsuo

llms multimodal

arXiv Machine Learning

Jul 15

ANGLE: Angular Neural Generative Learning via Engression

arXiv:2607. 12833v1 Announce Type: cross Abstract: Circular data, representing angles or directions, are frequently encountered in computer vision, biology, geology, and meteorology.

By Rajdeep Pathak, Archi Roy, Tanujit Chakraborty

agents computer-vision multimodal

arXiv Machine Learning

Jul 15

SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learning

arXiv:2607. 12042v1 Announce Type: cross Abstract: Visual generation is increasingly ubiquitous in diverse domains, from text-to-image/video synthesis to multimodal interactive creation.

By Jinxiu Liu, Jianru Li, Tanqing Kuang, Xuanming Liu, Kangfu Mei, Yandong Wen, Weiyang Liu

llms agents diffusion fine-tuning multimodal benchmarks

arXiv AI

Jul 15

Enabling 24-hour Agricultural Robotics: Unsupervised Day-to-Night Cross-Modal Image Translation for Nighttime Visual Navigation

arXiv:2607. 12065v1 Announce Type: cross Abstract: While visual navigation has been extensively studied in agricultural robotics, most existing systems assume daytime conditions.

By Robel Mamo, Rajitha de Silva, Grzegorz Cielniak, Taeyeong Choi

agents computer-vision robotics multimodal benchmarks

arXiv Machine Learning

Jul 15

What Does a Temporal Benchmark Score Measure? Decomposing Channel Use in Video VLM Evaluation

arXiv:2607. 12304v1 Announce Type: cross Abstract: A score on a temporal video question answering benchmark is meant to measure that a model has temporal understanding, but it conflates two questions.

By Farrukh Rahman

nlp multimodal benchmarks

arXiv Machine Learning

Jul 15

Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors

arXiv:2512. 15748v2 Announce Type: replace Abstract: Visual Species Recognition (VSR) is a fundamental task in scientific disciplines that require species-level identification, including ecology, palynology, evolutionary biology, systematics, and phylogenetics.

By Tian Liu, Anwesha Basu, James Caverlee, Shu Kong

llms multimodal benchmarks

arXiv Machine Learning

Jul 15

Spectral Diffusion Processes

arXiv:2209. 14125v3 Announce Type: replace-cross Abstract: Diffusion models have proven to be a flexible and effective framework for modelling probability distributions on finite-dimensional spaces.

By Angus Phillips, Thomas Seror, Michael Hutchinson, Valentin De Bortoli, Arnaud Doucet, Emile Mathieu

diffusion multimodal

arXiv Machine Learning

Jul 15

Auditing Data Leakage in Whole-Slide Image Multimodal Benchmarks

arXiv:2607. 12278v1 Announce Type: cross Abstract: Recent vision-language models (VLMs) for computational pathology report striking zero-shot performance on whole-slide image (WSI) visual question answering (VQA) benchmarks.

By Wenhao Zhang, Zhongliang Zhou, John Kang, Sheng Li

llms nlp multimodal benchmarks

arXiv AI

Jul 15

Navigating the Mirage: A Dual-Path Agentic Framework for Robust Misleading Chart Question Answering

arXiv:2603. 28583v2 Announce Type: replace-cross Abstract: Despite the success of Vision-Language Models (VLMs), misleading charts remain a significant challenge due to their deceptive visual structures and distorted data representations.

By Yanjie Zhang, Yafei Li, Rui Sheng, Zixin Chen, Yanna Lin, Huamin Qu, Lei Chen, Yushi Sun

llms agents computer-vision nlp fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jul 15

Text-Aided Multi-Modal Panoptic Symbol Spotting for CAD Floor Plan Drawings

arXiv:2607. 12678v1 Announce Type: cross Abstract: Computer-Aided Design (CAD) floor plan drawings contain both graphical primitives and textual annotations, which provide complementary geometric and semantic cues for intelligent design understanding.

By Yan Gong, Bohao Li, Bowen Du, Junchen Ye

multimodal benchmarks safety

arXiv Machine Learning

Jul 15

APPLV: Adaptive Planner Parameter Learning from Vision-Language-Action Model

arXiv:2603. 08862v2 Announce Type: replace-cross Abstract: Autonomous navigation in highly constrained environments remains challenging for mobile robots.

By Yuanjie Lu, Beichen Wang, Zhengqi Wu, Yang Li, Xiaomin Lin, Chengzhi Mao, Xuesu Xiao

llms agents reinforcement-learning robotics fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jul 15

Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning

arXiv:2607. 12112v1 Announce Type: cross Abstract: Federated fine-tuning of Multimodal Large Language Models (MLLMs) across distributed networks enables privacy-sensitive adaptation to evolving data streams, yet a fundamental obstacle prevents robust deployment in dynamic environments: catastrophic forgetting, wherein sequential task updates erase previously acquired knowledge across visual, linguistic, and cross-modal representations.

By Jing Liu, Chenxuanyin Zou, Jiayang Ren, Gaoyun Fang, Chengfang Li, Yan Wang, Zhenchao Ma, Bo Hu

llms rag agents fine-tuning multimodal benchmarks safety

Get the daily briefing