Xiaomi-CocktailASR-1 is an LLM-based end‑to‑end target‑speaker ASR architecture that uses reference speech as voiceprint prompts to transcribe a target speaker directly from mixed audio, eliminating the need for speech separation. It retains competitive single‑speaker performance comparable to mainstream ASR models and introduces a negative sample rejection capability, outputting empty text when the target speaker is absent. The system also supports a Chain‑of‑Thought reasoning mode and achieves state‑of‑the‑art performance on synthetic and real‑world multispeaker benchmarks, effectively addressing the cocktail party problem with a unified architecture.
By Yiru Zhang, Hang Su, Lichun Fan, Ying Zeng, Chang Liu, Yifeng Wang, Yuquan Liang, Tao Li, Lian Li, Wenhao Yang, Jian Luan, Cong Zou, Heng Qu
The paper introduces a multimodal in‑context learning framework that uses contrastive demonstration modeling to align large language models’ responses with the required reasoning paths. By contrasting suboptimal and better responses and incorporating a response‑conditioned retrieval mechanism, the method explicitly guides models beyond surface imitation. Experiments on various multimodal tasks, especially visual question answering, show consistent performance gains.
By Mingbo Yang, Wenqiang Wang, Zhaolu Kang, Peng Chen, Yannan Chen, Sunshang Wang, Yan Xiao
The paper introduces FedMVLA, a modality‑decoupled federated learning framework designed for privacy‑preserving embodied intelligence in 6G networks. It addresses the unique challenges of vision‑language‑action models by employing modality‑aware federated aggregation, privacy allocation, and communication compression, along with a precision‑critical action transport slice. A case study on federated robotic manipulation demonstrates significant gains in task success, scalability, and uplink payload reduction compared to standard FedAvg.
By Zhuodong Liu, Xiangyu Li, Chunhong Yuan, Hongyang Du, Bodong Shang, Qingqing Wu, Tony Q. S. Quek, Mohsen Guizani
The paper presents a method for generating synthetic plankton images conditioned on taxonomic labels to address the long‑tailed nature of automated plankton imaging datasets. A CLIP encoder is fine‑tuned on a large plankton corpus using a ranked contrastive objective that accommodates deep, ragged taxonomies, and then frozen to guide a parameter‑efficient diffusion transformer. The quality of the synthetic samples is evaluated both for distributional fidelity and for their usefulness in training downstream classifiers.
By Daniela Ivanova, Ozgu Goksu, Nicolas Pugeault
The paper introduces an end‑to‑end framework for testing image‑triggered command injection on computer‑use agents (CUAs). It demonstrates that a local visual patch can cause verifiable environmental effects through the entire pipeline—from screenshot input, through vision‑language‑model generation, action parsing, to execution—by training and deploying patches on GitHub Pages and a CSDN clone. Across five open‑source or publicly available GUI‑agent or VLM backends, the study reports 84.5% T‑ASR, 47.0% TAPR, and 20.3% E2E‑ASR success rates, with trajectory analysis revealing that some attacks first execute malicious terminal commands before continuing the original task.
By Zhihao Liu, Hongyu Sun, Zhiyuan Fu, Xiaonan Duan, Jice Wang, Shangru Zhao, Weizhi Meng, Wuxin Yang, Yangfan Zhou, Yuqing Zhang
arXiv:2609. 09404v1 Announce Type: cross Abstract: Agentic AI frameworks let a language model plan, keep memory, and call tools that reach real files, mail, and services.
By Viet K. Nguyen, Mohammad I. Husain
RiVaT‑Fuse introduces a reliability‑calibrated variational tensor fusion framework for multimodal image‑metadata prediction, treating fusion as a sample‑wise latent‑state estimation rather than simple aggregation. It replaces scalar modality confidence with matrix‑valued trust geometry, decomposes interactions into additive, multiplicative, and relational components, and couples the latent state with conditional robustness and structured multi‑task prediction. On an image‑level benchmark, RiVaT‑Fuse outperforms direct representation‑level baselines and improves probability and label stability under perturbation.
By Yingfan Xu, Tieming Liu, Ye Liang, Taiping Liu
The paper critiques current optimization-based methods for balancing modalities in Multimodal Sentiment Analysis, arguing they overpromise and underdeliver. It introduces a unified evaluation framework that tests gradient- and loss-based balancing strategies, provides a theoretical diagnosis showing these methods conflate fitting speed with discriminative contribution, and proposes a research agenda for held‑out discriminative modality valuation. Experiments on CMU‑MOSI and CMU‑MOSEI demonstrate that no strategy consistently outperforms Late Concatenation, performance is highly sensitive to hyperparameters, and ratio calibration does not yield reliable gains, highlighting that loss is not utility and gradients are not importance.
By Ioanna Kaffeza, Efthymios Georgiou, Alexandros Potamianos
MultiHuSE is a multimodal dataset featuring 2,407 high‑definition videos of 50 diverse actors delivering 1,463 text samples in four psychological humour styles—affiliative, aggressive, self‑enhancing, and self‑deprecating—plus neutral content. Each text is performed by multiple actors, allowing analysis of expressive diversity, and a subset includes emotion annotations. Baseline experiments show that multimodal fusion improves humour style classification accuracy over unimodal approaches, especially for affiliative humour.
By Mary Ogbuka Kenneth, Foaad Khosmood, Abbas Edalat
The paper introduces methods to improve speech recognition for multilingual video transcription using Whisper-based tools, targeting cross‑cultural understanding. It reports an average transcription error rate of 30% across seven languages, which can be lowered to 20% with modest fine‑tuning. The authors also release associated speech and metadata to aid community refinement of these techniques.
By Michael Picheny
The paper investigates whether Vision‑Language Models (VLMs) can understand visual persuasiveness by evaluating image‑message pairs that humans consistently judge as persuasive. It introduces Visual Persuasive Factors (VPFs), a taxonomy from cognitive psychology, to quantify visual cues influencing persuasive judgments. Empirical analysis shows VLMs tend to over‑predict persuasiveness, partially reproducing human patterns but often generating false positives, and that VPF‑guided interventions can improve performance only when properly framed.
By Gyuwon Park, Hyounghun Kim
The paper introduces a densely labeled coastal dataset of over 1,000 images from Oahu, Hawaii, featuring 18 semantic classes and 7,400 annotated instances. Seven modern vision‑language models were evaluated using text‑to‑mask, mask‑to‑mask, and mask‑to‑text alignment experiments. Results show that broad landscape classes are recognized more accurately than conventional object and coastal classes, with coastal concepts posing the greatest challenge; however, performance differences are not solely due to environmental context, and alternative textual labels improve recognition of several coastal concepts.
By Seth Knoop, Chad R. Samuelson, Gabriel R. Slade, Brady Moon, Joshua G. Mangelson
CLIP-RD introduces a relational distillation framework for efficient CLIP knowledge distillation, featuring Vertical Relational Distillation (VRD) and Cross Relational Distillation (XRD). VRD aligns intra‑modal similarity distributions between teacher and student, while XRD aligns cross‑modal similarity distributions to enforce bidirectional symmetry. This joint modeling of multidirectional relational structures improves the student’s embedding geometry, yielding a 1.8%p performance gain over CLIP‑KD across various architectures, tasks, and corruption settings with minimal training‑time overhead.
By Jeannie Chung, Hanna Jang, Ingyeong Yang, Uiwon Hwang, Jaehyeong Sim
V‑Retrver is an evidence‑driven retrieval framework that treats universal multimodal retrieval as an agentic reasoning process grounded in visual inspection. It allows multimodal large language models to selectively acquire visual evidence through external tools, alternating between hypothesis generation and targeted visual verification. The approach is trained with a curriculum that blends supervised activation, rejection‑based refinement, and reinforcement learning, achieving an average 23.0% improvement in retrieval accuracy across multiple benchmarks.
By Dongyang Chen, Chaoyang Wang, Dezhao Su, Xi Xiao, Zeyu Zhang, Jing Xiong, Qing Li, Yuzhang Shang, Shichao Kan
ABACUS is a 3‑billion‑parameter vision‑language model that simultaneously tackles object counting, crowd counting, referring‑expression counting, and count‑faithful image generation. It introduces density‑aware adaptive zooming with an objectness map, a boundary‑aware count policy trained via GRPO to avoid over‑ or under‑counting at crop edges, and a cycle‑consistent GRPO strategy that scores generated images for count accuracy and aesthetic quality without external critics. The model sets new state‑of‑the‑art performance on seven benchmarks, outperforming both specialized and larger generalist models.
By Anindya Mondal, Sauradip Nag, Anjan Dutta
The paper reports an entry to the EgoLongQA track of the Wearable‑AI Challenge at ECCV 2026 that achieved first place in the ≤2 B parameter division with a score of 0.8279 on the held‑out test set. The system is a single 2 B vision‑language model that answers multiple‑choice questions about ten‑minute egocentric videos in one greedy forward pass, obtained by distilling the junior perception module of a tool‑using agentic pipeline into a small student using teacher traces that answered correctly. By pruning the multilingual embedding table from 248,320 to 143,469 rows, the authors reduce the model to 1.9985 B parameters while maintaining identical logits on retained rows, enabling the model to reach 89 % of the accuracy of the large agentic pipeline with only 1.1 % of its parameters.
whyItMatters":"The work demonstrates that a highly efficient, sub‑2 B parameter vision‑language model can match the performance of a much larger agentic pipeline on long‑video perception tasks, showing the feasibility of deploying such models in resource‑constrained settings."
By Logesh Kumar Umapathi
The paper introduces SurgicalRoomAgent, a voice‑interactive multi‑agent system for smart operating rooms that leverages large language models to understand natural language, control devices, record intraoperative events, and generate surgical reports. Its layered architecture includes a voice interaction pipeline (wake, ASR, turn detection, agent reasoning, TTS) and an agent core (skill registry, task planner, device manager). Three key technologies—KV Cache prefix warming, streaming partial JSON parsing with early parallel task execution, and progressive skill prompt disclosure—reduce latency and maximize context efficiency, enabling real‑time operation within a 16,384‑token limit.
By Tianxiang Zhou
The paper presents a lightweight vision‑language encoder paired with a compact multi‑layer perceptron to estimate flare combustion efficiency from low‑cost thermal video. The trained model is deployed via a user‑friendly GUI that overlays efficiency values on each frame, tracks real‑time trends, visualizes distribution across the video, and exports CSV reports. In a six‑month trial, the system maintained 99% uptime with under 15 minutes of weekly maintenance.
By Afeefa Azam, Iyyakutti Iyappan Ganapathi, Fares Ossama Abdelhafez, Divya Velayudhan, Maregu Assefa Habtie, Hamad Karki, Khalid Yousef Al Awadhi, Naoufel Werghi
The paper investigates using Vision Language Models (VLMs) to accelerate verification and validation (V&V) of classification models by automatically detecting systematic errors. It introduces a VLM-based error slice detection (ESD) method that groups and labels errors, demonstrating its ability to identify perturbations in a non-military dataset and to cluster images by surroundings in a military context. The study highlights challenges such as underrepresentation of defence data in VLM training and limited contextual diversity, and suggests that while fully automated V&V is not yet feasible, VLMs could speed up the process in the future.
By Dieuwertje Alblas, Alma M. Liezenga, Jan Erik van Woerden, Fedor Taggenbrock, Dalia Aljawaheri, Klamer Schutte
HALDETECT is a system developed for the English hallucination-detection track of ImageEval 2026, where the task is to identify the single visually grounded statement among three culturally plausible options. The approach treats the problem as a contrastive decision, outputs the answer before an explanation, and bases reasoning on colour/texture, shape/form, and context. The best model fine‑tunes Qwen2.5‑VL‑7B‑Instruct with 4‑bit QLoRA, freezes the vision encoder, and achieves a Contrastive Instability score of 0.035 on the test set, placing third among eight teams.
By Syed Mohaiminul Hoque, Md Sakhawat Hossain