Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2608.24010v1 Announce Type: new Abstract: Supervised proxy-based deep cross-modal hashing has become the dominant paradigm for large-scale retrieval. However, prevalent methods model class prox...
arXiv:2608.24563v1 Announce Type: new Abstract: Imaging factor disentanglement in text-to-image generation aims to independently control image acquisition properties such as types of camera lenses, s...
The paper introduces Variance‑Guided Spatial Attention Fusion (VG‑SAF), a method for robust end‑to‑end driving that fuses camera and LiDAR data while handling asymmetric sensor degradation. VG‑SAF uses a physically grounded augmentor to generate dense reliability masks, modality‑specific experts to predict per‑pixel reliability scales, and a hybrid attention mechanism that gates unreliable cells and balances modalities. The approach also includes a Laplace uncertainty head to signal severe or combined sensor failures, and demonstrates improved closed‑loop robustness on the CARLA Longest6 benchmark across various degradation scenarios.
arXiv:2608.24281v1 Announce Type: new Abstract: Reducing annotation requirements remains a key challenge in developing robust medical object detectors. To address this, Vision-Language (VL) object de...
arXiv:2608.24430v1 Announce Type: new Abstract: Responsible deployment of face verification systems requires more than accurate decisions: systems should also provide interpretable and auditable evid...
arXiv:2604.09057v3 Announce Type: replace Abstract: Audio-video (AV) generation has recently made strong progress in perceptual quality and multimodal coherence, yet generating content with plausible...
arXiv:2506.12006v4 Announce Type: replace-cross Abstract: The cross-Modality Domain Adaptation (crossMoDA) challenge series, initiated in 2021 in conjunction with the International Conference on Medi...
arXiv:2608.23853v1 Announce Type: new Abstract: The interpretation of endoscopic imagery in ulcerative colitis is complex and subjective, with variability in human assessment and subtle mucosal infla...
arXiv:2605.31351v2 Announce Type: replace-cross Abstract: AI-based Visually Impaired Assistance (VIA) remains challenging, largely due to the high cost of human evaluation. The VLM-as-a-Judge paradig...
The paper investigates whether joint language‑audio embedding models encode human perceptual timbre semantics. It evaluates several state‑of‑the‑art models, finding that LAION‑CLAP aligns best with human‑perceived timbre across instrumental sounds and descriptor‑conditioned audio effects, yet the overall alignment remains limited. The study also notes that reverb‑induced timbre semantics are more consistently captured than equalization‑induced ones.
arXiv:2608.22872v2 Announce Type: new Abstract: Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline...
The paper introduces ICA, an evidence‑centric framework that represents information from web‑tool interactions as stable, rendered snapshots, enabling comparison across trajectories. It proposes Information‑Aware Credit Assignment, a post‑hoc reward propagation technique that estimates turn‑level utility from rollout success and assigns dense rewards to steps that provide high‑utility information. When combined with GSPO, ICA consistently improves performance on several web‑search benchmarks such as BrowseComp, GAIA, Xbench‑DS, and Seal‑0.
ForceFlow is a force-aware reactive framework that uses flow matching to improve contact-rich manipulation. It fuses force signals asymmetrically, treats force as a global regulator, and employs a joint prediction paradigm to couple force and motion. The approach splits tasks into a vision-dominant localization stage and a touch-dominant execution stage, using a Vision-to-Force handover to separate spatial generalization from contact regulation.
arXiv:2607.19261v4 Announce Type: replace-cross Abstract: Whole-slide image (WSI) diagnosis requires identifying diagnostically relevant regions, examining them across magnifications, and integrating...
arXiv:2509.13577v3 Announce Type: replace-cross Abstract: Trustworthy trajectory prediction grounds autonomous vehicle (AV) safety, yet deployed models inevitably face out-of-distribution (OOD) scene...
ReactBench is a benchmark designed to evaluate the structural reasoning abilities of multimodal large language models (MLLMs) using chemical reaction diagrams. The dataset contains 1,618 expert‑annotated question‑answer pairs that test reasoning across four hierarchical task dimensions, from simple endpoint counting to complex topological analysis. Evaluation of 24 MLLMs shows a performance gap of more than 30% between anchor‑based tasks and holistic structural reasoning tasks, indicating that current models struggle with reasoning over branching, converging, and cyclic structures.
PinSieve is a production system that selectively serves vision‑language models (VLMs) for enterprise content‑quality triage, operating only on the grey‑zone cases that lightweight models cannot resolve. The deployed VLM Serving Agent filters 2.05× more non‑actionable items, improves review productivity by 25.7%, cuts operating costs by 16.2%, and delivers signals the same day instead of the next. A governed memory flywheel with selective feedback, audit sampling, and a bounded proposal‑verifier loop further reduces false‑negative rates from 17.73% to 13.29% over six months, while a reasoning review agent audits teacher‑generated rationales for keep/repair/drop decisions. whyItMatters":"The system demonstrates how selective VLM serving and governed feedback loops can substantially improve efficiency, cost, and accuracy in enterprise AI content‑quality pipelines."
The paper introduces a self‑evolving scientific agent that uses large language models and iterative code generation to build interpretable, physically‑reasoned white‑box controllers for complex systems. The agent deploys candidate controllers in simulations, diagnoses dynamic behavior from multimodal evidence, and refines source code until a robust policy is achieved. Applied to a nonlinear fluid‑structure interaction problem—a two‑joint dogfish swimmer navigating an unsteady wake—the agent autonomously designs a controller that consistently reaches targets across a wide range of conditions without retraining.
The paper introduces a controlled evaluation framework for interactive visual grounding in large vision-language models (LVLMs), examining how varying amounts of initial target information and dialogue affect performance. Experiments across four visual contexts and interaction protocols show that current LVLMs lag behind human baselines, especially when no initial description is given and information must be gathered through questions. The study also finds that LVLMs are poorly calibrated, often overestimating confidence, and that interactive grounding remains a significant challenge requiring visual matching, information seeking, and synthesis.
The paper investigates the blending ratio used in few‑shot adaptation of vision‑language models, which combines a zero‑shot text prototype with the mean of labeled image features. It shows that the theoretically optimal ratio—derived from a closed‑form mean‑squared error minimizer—does not align with the ratio that actually maximizes performance, falling short by an average of 8.5 points. Moreover, a leave‑one‑out estimate on the support set achieves near‑oracle performance, and validation‑free linear probes outperform even oracle‑tuned blends, indicating that the hyperparameter can be set near‑optimally without external validation data.