Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2609.24576v1 Announce Type: cross Abstract: Modern Vision-Language Navigation (VLN) models rely mostly on pre-trained large Vision-Language Models (VLMs) to predict navigation actions. While th...
arXiv:2509.22650v3 Announce Type: replace Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...
arXiv:2606.07338v2 Announce Type: replace Abstract: Vision-language driving models increasingly use reasoning supervision to bridge perception, prediction, and planning, but existing driving rational...
arXiv:2608.10706v3 Announce Type: replace Abstract: Recent vision-language models demonstrate impressive general visual understanding, yet their art interpretation remains shallow: they describe surf...
arXiv:2609.07780v2 Announce Type: replace Abstract: Automated drone surveillance has become increasingly important for public safety, critical infrastructure protection,and restricted airspace monito...
M3GA-Wild is a new benchmark for multi-modal, multi-session ground-to-aerial place recognition in forests, combining synchronized RGB imagery and LiDAR from ground traversals with high‑resolution aerial imagery and multi‑altitude LiDAR over 370 hectares. The dataset includes accurate geo‑referenced 6‑DoF poses and spans 36 km of forest traversals, enabling systematic evaluation of visual, LiDAR, cross‑modal, and multi‑modal methods. Baseline experiments show LiDAR outperforms vision‑only approaches under severe viewpoint changes, while current multi‑modal fusion offers limited gains due to poor cross‑modal alignment, highlighting challenges in cross‑platform localisation and domain gaps.
BrainIAC is a unified framework for 3D brain lesion segmentation that handles heterogeneous MRI modalities and adapts online during interactive segmentation. It combines a multi‑modal backbone trained with zero‑filling and random modality dropping, 3D interactive prompts that default to fully automatic predictions, and a two‑stage online adaptation guided by pseudo‑labels and a Click‑Centered Gaussian loss. Experiments on seven MRI datasets show that the components work synergistically, outperforming existing methods and generalizing to unseen modalities and pathologies.
The paper introduces a free, label‑free visual evidence signal that improves fine‑grained vision‑language reasoning. By selecting image crops that maximize the model’s answer distribution peak, the method locates answer‑bearing regions without training or annotations, boosting accuracy from 70 % to 85 %. The evidence gap also complements model confidence, enabling better correctness prediction and error flagging.
DeCo introduces an efficient decouple-to-couple learning framework for multi-task visual grounding, addressing conflicts between localization and segmentation tasks. It first applies Task-aware Semantic Decoupling (TSD) to separate shared visual cues into task-specific features guided by salient words, then uses Hybrid Prior Coupling (HPC) to merge sentence-level semantic priors with mask-derived spatial priors for improved grounding. Experiments across multiple natural and remote sensing datasets show that DeCo achieves state‑of‑the‑art performance while requiring only lightweight trainable parameters on a frozen multimodal encoder.
SPHQuant introduces a rotation‑free spherical weight‑only quantization framework for Vision‑Language Models, decomposing 8‑dimensional weight vectors into sign, radius, and a positive unit direction. By isolating outlier magnitudes in the radius and allocating extra precision there, it mitigates accuracy loss at extreme low bit‑widths. The method also employs a compact positive‑direction codebook with angular fine‑tuning and a hardware‑friendly GEMV kernel, achieving state‑of‑the‑art performance while boosting decode throughput by 30.3% on RTX A6000 compared to QTIP.
CIG-MAE is a self‑supervised framework for WiFi‑based human action recognition that uses a cross‑modal masked autoencoder to reconstruct both amplitude and phase of Channel State Information. It introduces an adaptive, information‑guided masking strategy that focuses on high‑density time‑frequency regions and employs a Barlow Twins regularizer to align cross‑modal representations without negative samples. Experiments on three public datasets show that CIG‑MAE outperforms state‑of‑the‑art SSL methods and even surpasses a fully supervised baseline, highlighting its data efficiency, robustness, and generalization.
The CXR‑LT 2026 Challenge introduces a multi‑center, long‑tailed chest X‑ray classification benchmark with over 145,000 radiologist‑annotated images from PadChest and NIH datasets. It defines two core tasks: robust multi‑label classification on 30 known classes and open‑world generalization to 6 unseen rare disease classes. The paper outlines data collection, annotation, solution strategies, and evaluates performance across head‑vs‑tail, calibration, and cross‑center gaps, noting that vision‑language models improve in‑distribution and zero‑shot performance but rare‑finding detection under multi‑center shift remains difficult.
MinCU is a new benchmark for grounded minimal‑change understanding that presents pairs of near‑identical images differing by a single atomic variation in object category, attribute, count, or spatial position. Models are evaluated on their ability to describe the change, localize the changed region, and identify the changed entity. The authors also introduce SG‑ISA, a structured autoregressive method that decomposes the task into a Think‑Locate‑Describe sequence, showing that fine‑tuning with SG‑ISA improves both grounding accuracy and description quality while reducing reasoning‑token overhead.
The study developed a multimodal ultrasound and clinical data model to predict microvascular invasion (MVI) preoperatively in hepatocellular carcinoma (HCC). Using data from 489 patients across eight centers, the model combined B-mode ultrasound, color Doppler flow imaging, dynamic contrast-enhanced ultrasound, and clinical information, achieving an AUC of 0.8953 in external validation. Dynamic contrast-enhanced ultrasound contributed the most predictive power, while other modalities and clinical data added complementary value.
LIBERO-VPro is a benchmark designed to assess the closed‑loop visual robustness of robotic foundation models by systematically perturbing visual inputs during task execution. It spans four dimensions—Visual Evidence Degradation, Camera Staleness, Visual Source Consistency, and Task‑Relevant Scene Variation—across 12 challenge categories, 96 settings, and 3,296 task‑condition cases. Evaluations on six models over 196,000 simulated episodes and 200 real‑world rollouts show that high nominal performance can hide significant weaknesses in visual grounding, adaptation, and sensitivity to stale or missing observations.
The study reports that agitation—an early, subtle sign of distress—can be detected in autistic youth using multimodal wearable sensors. By fusing upper‑body movement, wrist‑worn physiological data, and vocalizations through pretrained foundation models, the authors achieved an AUC of 0.724 at the clinician‑annotated onset of agitation, with audio contributing the most signal. The approach works across 15 participants, showing that individualized agitation is detectable even before the onset and without needing a separate model per child.
arXiv:2609.24579v1 Announce Type: new Abstract: Event logs arise in a wide range of real-world processes, capturing not only event activities and timestamps but also multi-modal contextual informatio...
arXiv:2609.24718v1 Announce Type: new Abstract: While a centralized approach involving patient consent to collect and analyze data centrally would theoretically offer the best data quality and predic...
arXiv:2609.22094v1 Announce Type: cross Abstract: Content moderation systems traditionally entangle multimodal understanding with policy-specific classification, requiring full pipeline retraining fo...
arXiv:2609.22149v1 Announce Type: cross Abstract: Multimodal synthetic datasets combine structured attributes with free text, but are often evaluated separately. Such metrics can remain high after ta...