arXiv Machine Learning

Multimodal Evaluator Preference Collapse: Cross-Modal Coupling in Self-Evolving Agents

arXiv:2606. 16682v3 Announce Type: replace Abstract: When AI agents use language models to evaluate their own outputs in a feedback loop, systematic biases emerge.

arXiv Machine Learning

Jun 16

Multimodal Evaluator Preference Collapse: Cross-Modal Contagion in Self-Evolving Agents

arXiv:2606. 16682v1 Announce Type: new Abstract: When AI agents use language models to evaluate their own outputs in a feedback loop, systematic biases emerge.

By Zewen Liu

llms agents multimodal

arXiv AI

Jul 1

Calibrating the Evaluator: Does Probability Calibration Mitigate Preference Coupling in LLM Agent Feedback Loops?

arXiv:2606. 31371v1 Announce Type: cross Abstract: When large language model (LLM) agents adapt their behavior through evaluator feedback, systematic evaluator biases propagate into the agent's learned strategy distribution - a phenomenon termed evaluator preference coupling.

By Zewen Liu

llms agents

arXiv AI

Jul 28

Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization

arXiv:2607. 24354v1 Announce Type: new Abstract: Automatic prompt optimization (APO) has been widely adopted to adapt vision-language models (VLMs) to downstream tasks without weight updates, yielding promising results.

By Haoyue Liu, Xiaoyu Ma, Ye Chen, Yuexian Zou, Xiaoying Tang

llms multimodal benchmarks

arXiv Machine Learning

Jun 30

A Diagnostic Framework and Multi-Evaluator Audit of Evaluator-Driven Preference Dynamics in Self-Adapting LLM Agents

arXiv:2606. 29719v1 Announce Type: new Abstract: Measurements of proprietary LLM evaluators can become invalid within weeks -- we document one case and provide the diagnostic framework to detect it.

By Liu Zewen

llms agents multimodal benchmarks

arXiv Machine Learning

Jun 25

Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models

arXiv:2606. 26079v1 Announce Type: cross Abstract: Standard benchmarks for multimodal large language models (MLLMs) score each item on one canonical ordering and miss whether order-irrelevant shuffling changes the answer, a baseline reliability property called for by emerging AI evaluation guidelines.

By Akshay Paruchuri, Sanmi Koyejo, Ehsan Adeli

llms multimodal benchmarks safety

arXiv Machine Learning

Jun 30

The Contagion Tensor: A Framework for Measuring Output-Distribution Coupling in Multi-Agent LLM Systems -- and Auditing the Claims It Enables

arXiv:2606. 28839v1 Announce Type: new Abstract: We introduce the Contagion Tensor, a measurement framework for quantifying how large language model (LLM) output distributions couple across modalities, agents, and time steps.

By Zewen Liu

llms agents benchmarks

arXiv Machine Learning

Jul 22

Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning

arXiv:2607. 18615v1 Announce Type: cross Abstract: Machine unlearning for vision-language models (VLMs) remains underexplored.

By Zijie Liu, Jinhao Duan, Gaowen Liu, Sijia Liu, Tianlong Chen

llms multimodal

arXiv AI

Jun 9

IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generation

arXiv:2606. 09169v1 Announce Type: new Abstract: In recent years, unified multimodal models (UMMs) have emerged to support both understanding and generation within a single framework.

By Lingyi Meng, Zecong Tang, Haoran Li, Tengju Ru, Zhejun Cui, Weitong Lian, Qi Kang, Hangshuo Cao, Yichen Zhu, Yechi Liu, Kaixuan Wang, Yu-Jie Yuan, Chunwei Wang, Yu Zhang, Bo Dai

multimodal benchmarks safety

arXiv Machine Learning

Aug 6

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

arXiv:2608. 05000v1 Announce Type: cross Abstract: Vision offers a critical axis for advancing foundation models, driving a shift towards natively unified multimodal pretraining.

By Junlin Han, Shengbang Tong, David Fan, Minghao Chen, Philip Torr, Filippos Kokkinos, Mike Lewis

nlp multimodal safety

arXiv AI

Aug 5

In-Context Collapse in Vision-Language Models and How to Mitigate it?

arXiv:2608. 02830v1 Announce Type: cross Abstract: Many-shot in-context learning (ICL) lets vision-language models (VLMs) adapt from image--label demonstrations without weight updates, and is widely assumed to improve as more demonstrations are supplied.

By Mohammad Rostami

llms computer-vision multimodal benchmarks

arXiv AI

Aug 7

C$^3$PO: Evaluating Cross-Modal Composition and Counterfactual Performance in Omnimodal Models

arXiv:2608. 05381v1 Announce Type: new Abstract: Current Multimodal Large Language Models (MLLMs) can process diverse sensory inputs, yet their reasoning remains heavily biased toward a dominant modality, resulting in brittle cross-modal reasoning.

By Swapnanil Mukherjee, Agyeya Negi, Tanuja Ganu, Ponnurangam Kumaraguru

llms multimodal benchmarks

arXiv AI

Jul 13

GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs

arXiv:2507. 18043v2 Announce Type: replace-cross Abstract: Inference-time steering methods offer a lightweight alternative to fine-tuning large language models (LLMs) and vision-language models (VLMs) by modifying internal activations at test time without updating model weights.

By Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, Mohit Bansal

llms fine-tuning multimodal safety

Related stories