Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

8,869 stories · RSS feed

arXiv AI

Jul 28

Earnings25: A Comprehensive 500-Hour Speech Benchmark for Finance

arXiv:2607. 23813v1 Announce Type: cross Abstract: We introduce Earnings25, a finance-domain benchmark for evaluating automatic speech recognition (ASR) on English-language earnings calls under realistic conditions.

By Denglin Jiang, Haoran Zhou, Anshul Wadhawan, Brendan Fahy, Vinay Ramesh, David Weisberg, Dmitriy Derkachevskiy, Helen Sheehan, Srivas Prasad, Michele Franceschini

multimodal benchmarks

arXiv AI

Jul 28

ARdena: Scenario-driven control of real-time LLM agents

arXiv:2607. 22651v1 Announce Type: new Abstract: Large language models (LLMs) have enabled increasingly capable conversational agents, but reliably controlling their behavior in real-time interactive environments remains a significant challenge.

By Luka Borozan, Domagoj Matijevi\'c

llms agents robotics fine-tuning multimodal safety

arXiv Machine Learning

Jul 28

Compressing LLMs with MoP: Mixture of Pruners

arXiv:2602. 06127v2 Announce Type: replace Abstract: The high computational demands of Large Language Models (LLMs) motivate methods that reduce parameter count and accelerate inference.

By Bruno Lopes Yamamoto, Lucas Lauton de Alcantara, Victor Zacarias, Leandro Giusti Mugnaini, Keith Ando Ogawa, Lucas Pellicer, Rosimeire Pereira Costa, Edson Bollis, Anna Helena Reali Costa, Artur Jordao

llms fine-tuning efficiency multimodal

arXiv AI

Jul 28

AI-generated Images Challenge Visual Trust in High-risk Scenarios

arXiv:2607. 22745v1 Announce Type: cross Abstract: Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and personal reputation.

By Yi-Zhi Wang, Yichen Xiao, Linan Yue, Weibo Gao, Yichao Du, Pengfei Fang, Shimin Di, Min-Ling Zhang

llms diffusion multimodal benchmarks safety

arXiv AI

Jul 28

Myopia Prevention and Control 3.0: Artificial Intelligence--Driven Risk Stratification, Proactive Monitoring, and Personalized Intervention

arXiv:2607. 24187v1 Announce Type: new Abstract: The convergence of artificial intelligence (AI), digital sensing, and ubiquitous computing has created an unprecedented opportunity to transform myopia prevention from a reactive, population-based model into a proactive, precision-driven one.

By Tieniu Wang, Cangzhu Huang, Qianhui Li

multimodal data-engineering

arXiv AI

Jul 28

Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification

arXiv:2607. 24683v1 Announce Type: cross Abstract: Multi-modal classification leverages complementary information across diverse data sources to enhance predictive performance.

By Francisco Mena, Dino Ienco, Roberto Interdonato, Cassio F. Dantas, Simon Besnard

multimodal benchmarks safety

arXiv AI

Jul 28

Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training

arXiv:2512. 24125v3 Announce Type: replace-cross Abstract: General-purpose robotic systems operating in open-world environments must achieve both broad generalization and high-precision action execution, a combination that remains challenging for existing Vision-Language-Action (VLA) models.

By Yi Liu, Sukai Wang, Dafeng Wei, Xiaowei Cai, Linqing Zhong, Jiange Yang, Guanghui Ren, Jinyu Zhang, Maoqing Yao, Chuankang Li, Xindong He, Liliang Chen, Jianlan Luo

llms nlp robotics multimodal benchmarks

arXiv AI

Jul 28

CADER: Confidence-Aware Dynamic Evidence Reasoning for Long-Video Understanding

arXiv:2607. 24582v1 Announce Type: cross Abstract: Long-video understanding increasingly relies on large vision-language models and tool-augmented reasoning, but most systems apply the same inference procedure to every example regardless of difficulty.

By Jinlong Yang, Wenhao Zhang, Kuanwei Lin, Sijie Cheng

llms agents multimodal benchmarks

arXiv AI

Jul 28

Voice-Driven Semantic Perception for UAV-Assisted Emergency Networks

arXiv:2602. 17394v2 Announce Type: replace-cross Abstract: Unmanned Aerial Vehicle (UAV)-assisted networks are increasingly foreseen as a promising approach for emergency response, providing rapid, flexible, and resilient communications in environments where terrestrial infrastructure is degraded or unavailable.

By Nuno Saavedra, Pedro Ribeiro, Andr\'e Coelho, Rui Campos

llms nlp multimodal

arXiv Machine Learning

Jul 28

Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer

arXiv:2605. 07407v2 Announce Type: replace Abstract: We show that information can be transferred post-hoc across independently trained health foundation models (FMs), each pretrained on ~20M minutes of wearable sensor data from ~172K participants, by aligning their data-dependent coordinate systems.

By Gajendra Katuwal, Advait Koparkar, Salar Abbaspourazad, Anshuman Mishra, Sarvesh Kirthivasan

rag multimodal safety

arXiv Machine Learning

Jul 28

To Erase, or Not to Erase: Robust Training-Free Concept Erasure with Preservation aware Adaptive Ranked Subspace Expansion

arXiv:2607. 23492v1 Announce Type: cross Abstract: Concept erasure techniques (CETs) edit text-to-image diffusion models to erase undesired targets such as NSFW content or copyrighted styles, while preserving model utility on benign concepts.

By Shaswati Saha, Rajasekhar Anguluri, Manas Gaur

llms diffusion multimodal

arXiv AI

Jul 28

MulRobBench: A Decision-Level Benchmark for Safe and Security-Policy-Compliant Multimodal UAV Agents

arXiv:2607. 23870v1 Announce Type: cross Abstract: Smart-city airspace is transforming Uncrewed Aerial Vehicles (UAVs) from passive sensing platforms into cyber-physical decision makers that must follow operational rules under degraded observations and ambiguous language.

By Belal S. Alsinglawi, Weizheng Wang, Junyi Wu, Yi Jiang, Lianhai Lin, Merouane Debbah, Izzat Alsmadi

agents multimodal benchmarks safety

arXiv AI

Jul 28

EgoPlay: Event-Triggered Video Editing for Egocentric Streams

arXiv:2607. 24560v1 Announce Type: cross Abstract: We introduce EgoPlay, an event-triggered video-to-video editor for egocentric streams, obtained by fine-tuning a pretrained V2V diffusion transformer on event-conditioned data built primarily from Ego4D.

By Jinjie Mai, Gordon Guocheng Qian, Willi Menapace, Arpit Sahni, Chaoyang Wang, Ashkan Mirzaei, Runjia Li, Sergey Tulyakov, Bernard Ghanem, Peter Wonka, Rameen Abdal

llms diffusion fine-tuning multimodal benchmarks

arXiv AI

Jul 28

scMIR: a vision-language foundation model for single-cell light microscopy image representation

arXiv:2607. 22712v1 Announce Type: cross Abstract: Single-cell light microscopy images have become an important data source for characterizing cell phenotypes, but their complexity and heterogeneity pose challenges to high-throughput automated analysis.

By Yifan Shang (Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China, College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Jiahui Tan (College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Xiangxiang Zeng (College of Computer Science and Electronic Engineering, Hunan University, Changsha, China), Renjie Zhou (Department of Biomedical Engineering, The Chinese University of Hong Kong, Hong Kong, China)

fine-tuning efficiency multimodal benchmarks safety

arXiv Machine Learning

Jul 28

Self-Boosting Vision-Language Models with Noisy Student On-Policy Self-Distillation

arXiv:2607. 23125v1 Announce Type: new Abstract: Post-training enables vision-language models (VLMs) to understand human instructions and perform various downstream tasks.

By Shuai Wang, Daoan Zhang, Zhe Tang, Hao Cheng, Jiaheng Wei

llms reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Jul 28

VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference

arXiv:2512. 01031v2 Announce Type: replace-cross Abstract: Vision-Language-Action models (VLAs) are becoming increasingly capable across diverse robotic tasks.

By Jiaming Tang, Yufei Sun, Yilong Zhao, Shang Yang, Yujun Lin, Zhuoyang Zhang, James Hou, Yao Lu, Zhijian Liu, Song Han

robotics efficiency multimodal benchmarks

arXiv AI

Jul 28

WCM: World-Cognition Model for Generalizable Human-Robot Interaction

arXiv:2607. 22999v1 Announce Type: cross Abstract: Language agents can now interact fluently with users in software, but robots still struggle to bring comparable interaction to physical tasks.

By Yuzhen Chen, KC Zhou

agents robotics fine-tuning multimodal

arXiv AI

Jul 28

What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features

arXiv:2607. 23271v1 Announce Type: cross Abstract: Contrastive vision-language models such as CLIP map semantically opposite phrases (e.

By Chen-Yi Lu, Yueh-Shao Chen, Somali Chaterji

llms rag diffusion fine-tuning multimodal safety

arXiv AI

Jul 28

An empirical investigation into the properties of standard word embeddings

arXiv:2607. 23675v1 Announce Type: cross Abstract: The embedding of word sequences into continuous vector spaces has been one of the most important developments in Natural Language Processing in the recent past.

By Salomon Kabongo

rag nlp multimodal

arXiv AI

Jul 28

How Well Can AI Generate Backlogs from App Mockups?

arXiv:2607. 22902v1 Announce Type: new Abstract: Creating sprint backlogs requires considerable effort, as items such as epics, user stories, and tasks can be missed or inconsistently specified.

By Andrea Lezcano Airaldi, Lourdes Romera, Walid Maalej

llms multimodal

Get the daily briefing