Multimodal models

Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.

4,626 stories · RSS feed

arXiv AI
Jun 9

Self-Evolving Scientific Agent Discovers Generalizable Physically-Reasoned Fluid Control

arXiv:2606. 08405v1 Announce Type: new Abstract: While data-intensive deep reinforcement learning can optimize complex control policies, scientific discovery in physical systems fundamentally requires an interpretable chain of reasoning that connects physical evidence to structured control architectures.

By Boai Sun, Wenjin Guo, Zongmin Yu, Liu Yang
arXiv AI
Jun 9

Robust-U1: Can MLLMs Self-Recover Corrupted Visual Content for Robust Understanding?

arXiv:2606. 08063v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in visual understanding, yet their performance degrades significantly under real-world visual corruptions.

By Jiaqi Tang, Jianmin Chen, Youyang Zhai, Wei Wei, Runtao Liu, Mengjie Zhao, Xiangyu Wu, Qingfa Xiao, Qifeng Chen
arXiv AI
Jun 9

Blockchain Infrastructure for Intelligent Cyber--Physical--Social Systems:Post-Quantum Security, Interoperability, and Trustworthy Data Economies in the Era of Embodied AI

arXiv:2606. 06895v1 Announce Type: cross Abstract: The deployment of embodied artificial intelligence via world-model-based robotics presents a transformative opportunity for blockchain infrastructure, establishing urgent demand for trustworthy data provenance, cross-organizational governance, and incentive-compatible sharing across decentralized ecosystems.

By Song Guo, Huawei Huang, Dongping Liu, Aoyu Zhang, Luyao Zhang
arXiv AI
Jun 9

DIYHealth Suite: Dataset, Model, and Benchmark for Health Management at Home

arXiv:2606. 07542v1 Announce Type: cross Abstract: Generative AI is reshaping healthcare, yet most existing advances rely on hospital-grade devices, which limits their accessibility and potential for health management outside clinical settings.

By Changshuo Liu, Junran Wu, Zhongle Xie, Wenqiao Zhang, Kaiping Zheng, Jiaqi Zhu, Qingpeng Cai, Ooi Gene Anne, Marcus Chun Jin Tan, Jianwei Yin, James Wei Luen Yip, Beng Chin Ooi
arXiv AI
Jun 9

Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER)

arXiv:2606. 07608v1 Announce Type: cross Abstract: We present a systematic study of fine-tuning OpenAI's Whisper large-v3 for Swiss German ASR, using 1,367 hours of broadcast speech paired with Standard German subtitles as weak supervision.

By Felix Akeret
arXiv AI
Jun 9

MemoVAD: Resource-Efficient Video Anomaly Detection via Dynamic Semantic Memory in Edge Computing Scenarios

arXiv:2606. 07669v1 Announce Type: cross Abstract: Deploying Video Anomaly Detection (VAD) in real-world surveillance faces a fundamental tension between the demand for high-level semantics to ensure effectiveness and the limited computational resources of edge devices.

By Guo Li, Jiandian Zeng, Yang Li, Zihao Peng, Ke Chen, Tian Wang
arXiv AI
Jun 9

IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generation

arXiv:2606. 09169v1 Announce Type: new Abstract: In recent years, unified multimodal models (UMMs) have emerged to support both understanding and generation within a single framework.

By Lingyi Meng, Zecong Tang, Haoran Li, Tengju Ru, Zhejun Cui, Weitong Lian, Qi Kang, Hangshuo Cao, Yichen Zhu, Yechi Liu, Kaixuan Wang, Yu-Jie Yuan, Chunwei Wang, Yu Zhang, Bo Dai
arXiv Machine Learning
Jun 9

Conditional Normalizing Flows for Forward and Backward Joint State and Parameter Estimation

arXiv:2601. 07013v2 Announce Type: replace-cross Abstract: Traditional filtering algorithms for state estimation -- such as classical Kalman filtering, unscented Kalman filtering, and particle filters -- show performance degradation when applied to nonlinear systems whose uncertainty follows arbitrary non-Gaussian, and potentially multi-modal distributions.

By Luke S. Lagunowich, Guoxiang Grayson Tong, Daniele E. Schiavazzi