The paper introduces PIVOT, a dual-level learning framework designed to improve visually-grounded multimodal reasoning in large vision-language models. PIVOT employs a self‑calibrated experience replay mechanism to selectively reuse valuable visual reasoning trajectories, and a vision‑guided advantage allocation scheme that assigns extra rewards to tokens with strong visual support. Experiments on multiple benchmarks show that PIVOT enhances the multimodal reasoning performance of these models.
By Xinxin Song, Siyuan Li, Tingxiong Xiao, Jinli Suo
Evolutionary Ensemble Search (EES) is a framework that builds machine‑learning procedures through expert‑guided program evolution. A specialized council interprets task evidence and experimental results to generate structured search directions, which an orchestrator assigns to execution specialists and an evolutionary engine. The engine selects parents, diagnoses errors, and creates descendants via code mutation, pipeline edits, and crossover, with each child evaluated on its own validation evidence. Population archives preserve useful alternatives, and compatible predictions compete in a validation‑gated ensemble stage. Search adapts through parent‑relative operator credit, session memory, and lessons retrieved across runs. The system achieved medal‑threshold artifacts on 19 of 22 tasks (86.36 %) with 11 gold, five silver, and three bronze outcomes across diverse modalities.
By Juan P. Madrigal-Cianci, Eshan Chordia
MUSE is a new benchmark designed to evaluate large vision‑language models on artistic image understanding within situated educational contexts. It separates image annotation from question generation, offering twelve tasks that cover visual perception, semantic and affective interpretation, cultural understanding, and compositional reasoning across diverse artistic images from Singaporean, Southeast Asian, and Western traditions. The benchmark reveals significant gaps in model performance, especially in affective interpretation and compositional reasoning, and highlights common failure modes for trustworthy educational multimodal systems.
By Luyao Zhu, Xun Wei Yee, Wei Li, Mun Thye Mak, Wee Siong Ng
Lexara-RF introduces reference‑free metrics for evaluating conversational visual analytics agents that generate visualizations and natural‑language explanations. The framework uses only the prompt, data, and model response to score outputs, applying 13 metrics derived from visualization design theory and Gricean principles as consistency, intent‑alignment, and design validity checks. In tests against a human‑rated corpus, Lexara‑RF matches reference‑based methods, outperforms surface‑similarity NLG baselines, and accurately identifies structurally grounded failures.
By Srishti Palani, Vidya Setlur
The paper introduces Generalist‑Specialist Mixture‑of‑Experts (GS‑MoE), a two‑branch architecture that combines a cross‑modal generalist model with modality‑specific specialists through domain‑constrained feature fusion. GS‑MoE improves detection of rare pathologies in multimodal medical imaging, achieving significant per‑class F1 gains and outperforming dense and specialist‑only MoE baselines while using about 53% fewer active parameters at inference. The study demonstrates that balancing cross‑modal shared representations with expert routing can enhance performance on low‑prevalence conditions.
By Johannes Kaiser, Florian Braunmiller, Daniel R\"uckert, Georgios Kaissis
CompArt introduces a new approach to aesthetic alignment in text-to-image generation by using the Principles of Art (PoA) such as Balance, Rhythm, and Emphasis to define explicit compositional constraints. The authors create a large dataset of 80,032 WikiArt images, each annotated with PoA analyses generated by a multimodal LLM, and present ArtDapter, a lightweight adapter that steers a pretrained diffusion model along ten PoA dimensions while preserving semantic fidelity. Experiments demonstrate that CompArt outperforms strong baselines in adhering to PoA controls under a dual evaluation protocol.
By Zhe Jin, Tat-Seng Chua
SurgRAW introduces a multi‑agent, chain‑of‑thought workflow for robotic surgical video analysis, leveraging a new SurgCoTBench benchmark with 14,256 QA pairs across five surgical tasks. The system uses an orchestrator to split scene understanding into two reasoning streams, panel‑discussion‑style collaboration among task‑specific agents, and retrieval‑augmented generation to incorporate surgical knowledge. Experiments show SurgRAW outperforms mainstream vision‑language models and a supervised baseline by 14.61% accuracy.
By Chang Han Low, Ziyue Wang, Tianyi Zhang, Zhu Zhuo, Zhitao Zeng, Evangelos B. Mazomenos, Yueming Jin
The paper introduces ThoughtMed-1M, a large-scale medical visual question answering dataset built from de‑identified images and clinician‑generated commentaries, designed to capture structured clinical reasoning and image‑text alignment. Using this dataset, the authors train FOLTMed, a foundational large language model that achieves state‑of‑the‑art performance on 42 medical VQA benchmarks, with a macro accuracy of 85.4% and improved factuality and similarity metrics over existing models.
By Lingxuan Hou, Yuhua Xie, Yue Hu, Yan Zhuang, Junqi Li, Chengzhi Xia, Binh Phu Nguyen, Abubakar Siddique, Minh Nguyen, Yao Hou, Yanju Bao, Kexin Liu, Ke Chen, Jianjun Sun, Zeqi Li, Trung Nguyen, Jiangli Lin
CapMem is a new benchmark for evaluating caption-based episodic memory in egocentric video. It contains 75 videos (33.7 hours total) and 1,000 multiple-choice questions across 16 scenarios, designed to test the Episodic Memory Video Caption QA task. Experiments show that using captions as memory outperforms direct VideoQA on long videos, and a caption-guided retrieve-and-verify approach further boosts accuracy.
By Dingli Liang, Yiqiao Xie, Yukai Huang, Zhaokai Wang, Weitong Cai, Guangwen Feng, Jifei Song, Zhensong Zhang, Hang Zhang
The thesis presents AI frameworks that accelerate crystalline materials discovery by tackling both crystal property prediction and crystal structure generation. It introduces CrysXPP, CrysGNN, and CrysMMNet for efficient, data‑sparse property prediction using graph autoencoding, self‑supervised pretraining, and multimodal learning. For generation, TGDMat is a text‑guided diffusion model that jointly learns lattice parameters, atomic types, and coordinates, enabling valid, stable, and conditionally generated periodic materials.
By Kishalay Das
The paper presents Real‑Time EXPO‑FT, a reinforcement learning framework that fine‑tunes large Vision‑Language‑Action models for real‑time robotic control. It separates slow, expressive action generation from fast, reactive edits, allowing a lightweight policy to adjust actions based on the latest observation. Experiments on the Kinetix benchmark and four dynamic real‑world tasks show that Real‑Time EXPO‑FT achieves superior performance, improving policy success rates from 42% to 97% with only ten minutes of online data and no human intervention.
By Perry Dong, Kuo-Han Hung, Dorsa Sadigh, Chelsea Finn
The paper introduces a metric interaction framework for robotic manipulation that explicitly models object- and scene-level interactions in Cartesian space. It uses Interaction‑Centric Tokens (ICTs) to represent end‑effector trajectories relative to objects and a Metric Action Interaction Field (MAIF) to attend to scene point‑cloud features for geometry‑conditioned action corrections. Experiments show modest but consistent improvements across several benchmarks, including LIBERO, RoboTwin 2.0, and real‑world tasks.
By Lijie Wang, Zheng Lu, Yiming Wang, Heyang Yu, Kenghou Hoi, Bowen Hu, Di Cui, Tianyu Xin, Haoran Liao, Wanqi Zhong, Xingjie Fan, Yizhao Xu, Ziliang Wang, Fei Gao, Yiming Li
arXiv:2601.11262v2 Announce Type: replace-cross
Abstract: Music Cover Retrieval, also known as Version Identification, aims to recognize distinct renditions of the same underlying musical work, a tas...
By Joanne Affolter, Benjamin Martin, Elena V. Epure, Gabriel Meseguer-Brocal, Fr\'ed\'eric Kaplan
arXiv:2609.18533v1 Announce Type: new
Abstract: Automatic speech recognition (ASR) systems exhibit unequal error rates across speaker groups, motivating interventions on their internal representation...
By Nicolas Bourrel, Abderrahmane Issam, Gerasimos Spanakis
arXiv:2609.18516v1 Announce Type: new
Abstract: While Large Language Models excel in natural language processing, efficiently extending their capabilities to spoken input remains a significant challe...
By Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis
SynAgent is a framework that uses large language model agents to run autonomous experiments while building an explicit, revisable understanding of the synthesis process. Unlike traditional black‑box optimizers, SynAgent generates analysis skills on the fly and reasons multimodally over data such as X‑ray diffraction patterns and electron micrographs. In an 18‑experiment campaign on LiCoO₂ thin‑film deposition, SynAgent produced highly crystalline films and uncovered a sharp temperature threshold and optimal growth window (650–690 °C) for crystallization.
By Izumi Takahara, Kazunori Nishio, Akira Aiba, Shigeru Kobayashi, Takao Nakajima, Taro Hitosugi, Teruyasu Mizoguchi
The paper introduces a 3D-CLIP encoder trained with structured hard negatives to improve vision‑language alignment for text‑to‑CT generation. This encoder drives a latent diffusion model that operates directly in 3D latent space, eliminating spatial artifacts from super‑resolution pipelines. Experiments on the CT‑RATE dataset show state‑of‑the‑art image fidelity and factual correctness across 18 pathological conditions, with lower inference time and GPU memory usage than competing methods.
By Daniele Molino, Camillo Maria Caruso, Filippo Ruffini, Paolo Soda, Valerio Guarrasi
PACT‑WAM is a world‑action model that simultaneously predicts a 16‑step action trajectory and its corresponding visual forecast for robot manipulation. It uses a hierarchical history encoder that compresses past observations into fewer tokens, reducing processing cost by 75% compared to dense encoding. The model’s shared flow module updates action and visual states jointly, and a TiTok‑VAE decoder reconstructs multi‑view future images, which are then used by a vision‑language component (Proposal Review) to improve execution‑prefix selection and proposal rejection, boosting success rates on several benchmarks.
By Yushan Liu, Jingjing Fan, Shoujie Li, Yifan Xie, Xiao-Ping Zhang, Wenbo Ding
MINT (Multimodal Imaging-to-Speech Knowledge Transfer) is a three-stage framework that transfers MRI-derived biomarkers to speech representations for early Alzheimer’s screening. An MRI teacher creates a compact embedding space for CN‑versus‑MCI classification, and a residual projection head aligns speech features to this space using a geometric loss, allowing imaging‑free inference. Experiments on ADNI‑4 show that aligned speech matches speech baselines, while multimodal fusion outperforms MRI alone, and ablations highlight dropout regularization and self‑supervised pretraining as key design choices.
By Vrushank Ahire, Yogesh Kumar, Anouck Girard, M. A. Ganaie
AMIGO (Agentic Multi-Image Grounding Oracle Benchmark) is a long-horizon evaluation framework for vision‑language models that tests hidden‑target identification across galleries of visually similar images. The benchmark requires a model to ask a sequence of attribute‑focused Yes/No questions, receiving Yes/No/Unsure feedback and penalizing invalid actions with Skip, thereby stressing question selection under uncertainty, constraint tracking, and fine‑grained discrimination. Using the Guess My Preferred Dress task, the study shows that final‑answer accuracy alone overstates performance, as models may guess correctly without verified evidence, waste turns, or violate the protocol, highlighting the need for combined visual discrimination, informative questioning, and robust protocol adherence.
By Min Wang, Ata Mahjoubfar