Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2609.12181v1 Announce Type: new Abstract: Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate p...
RoES is a Rotational Equivariant Selective-frequency fusion network that dynamically separates low- and high-frequency components of infrared-visible images. It uses a trainable rotation-enhanced updater to decouple frequencies, then fuses them with a dual-branch module: a rotation-equivariant Mamba for low-frequency structural dependencies and a polar spectral attention Dual-Fourier block for high-frequency detail refinement. Experiments show RoES outperforms existing methods in fusion quality and downstream object detection, offering a robust multimodal fusion solution.
arXiv:2504.04903v3 Announce Type: replace Abstract: We present Lunima-OmniLV (abbreviated as OmniLV), a universal multimodal multi-task framework for low-level vision that addresses over 100 sub-task...
ChitraMiti introduces a synthetic benchmark of 12,874 Bengali planar geometry problems with structured 15‑attribute descriptions, alongside a complementary set of 500 textbook diagrams. Using a three‑phase protocol that tests diagram‑only, diagram‑plus‑description, and description‑only inputs, the study finds that description‑only performance matches diagram‑plus‑description performance across several VLMs, yet models still struggle with cross‑modal verification. Fine‑tuning on ChitraMiti improves results on both datasets, though a gap remains compared to the best zero‑shot model.
The study investigates how adding ballistocardiography (BCG) signals to photoplethysmography (PPG) and arterial pressure wave (APW) signals affects the estimation of cardiovascular biomarkers. Using a unified whole‑body cardiovascular model, synthetic PPG, APW, and BCG signals were generated and analyzed with neural posterior estimation and simulation‑based inference. Results show that incorporating BCG significantly improves estimation accuracy, converting multimodal posterior distributions from PPG/APW alone into unimodal distributions, thereby enhancing the reliability of cardiovascular biomarker estimation.
AnchorVLN is an open‑vocabulary vision‑language navigation system that separates semantic proposals from geometric metrics. It uses a VLM to generate semantics while a geometry module supplies reliable metric quantities such as range and bearing, all within a Model Context Protocol server. The system achieves 64.4% on instruction following and improves object‑reference accuracy, reducing median center error from 3.37 m to 2.48 m.
arXiv:2609.12078v1 Announce Type: new Abstract: Objects in post-fire environments often undergo irreversible physical transformations that change their geometry, material state, and visual appearance...
arXiv:2609.12818v1 Announce Type: new Abstract: Long video understanding often behaves like a visual needle-in-a-haystack problem: query-relevant evidence is sparsely distributed across long temporal...
arXiv:2601.20433v4 Announce Type: replace Abstract: Deepfake detection is a widely researched topic that is crucial for combating the spread of malicious content, with existing methods mainly modelin...
arXiv:2609.06729v2 Announce Type: replace Abstract: Accurate 3D brain tumour segmentation from multi-modal Magnetic Resonance Imaging (MRI) is essential for clinical diagnosis and treatment planning....
The paper introduces a style-aware paraphrasing method for text anonymization that leverages pretrained large language models to build compact stylistic profiles from minimal samples and rewrite text to suppress identifiable style markers while preserving meaning. It demonstrates that this approach reduces authorship attribution F1 scores by 60‑70% on blog and review datasets, outperforming both differential privacy‑based and non‑DP baselines, and maintains content quality and readability.
The paper introduces Observation‑Anchored Selective Assimilation (OASA) for forecasting tumor‑state proxies in post‑treatment glioma patients using serial MRI observations. OASA anchors the patient‑specific state with an intermediate observation and selectively updates it via a tiered rule and voxel‑wise soft gate, outperforming baseline methods in Dice score at certain thresholds. The approach is validated on 120 patient triplets and the code is publicly released.
Semantically Aligned Gradient-Driven Context-Preserving Image Editing (IABEdit) is a model‑agnostic framework that embeds differentiable semantic verification into the training of generative image editors. By using a frozen vision‑language model to extract spatially‑aware descriptors from ground‑truth edits and a trainable aligner to reproduce them from generated outputs, the residual becomes a gradient that teaches the generator both what to edit and where, without adding inference‑time VLM cost. IABEdit is compatible with various backbones (e.g., U‑Net in Stable Diffusion and MMDiT in FLUX) and improves structural fidelity on MagicBrush, achieves state‑of‑the‑art instruction adherence on RealEdit and EMU Edit, and outperforms the proprietary Gemini agent on the D‑LORD surveillance benchmark under heavy occlusion. "whyItMatters":"IABEdit demonstrates that incorporating semantic verification during training can produce more accurate, well‑localized edits and outperform existing methods even in challenging surveillance scenarios, as shown by its superior metrics and human/GPT‑4o evaluations."
arXiv:2609.13012v1 Announce Type: new Abstract: Vision-language models retain a substantial amount of pixel-decodable visual content in their visual key-value cache. We show, in our setting, that thi...
The paper introduces a dual cross‑attention deep learning framework for automated grading of Cervical Intraepithelial Neoplasia (CIN) and prediction of Swede scores, using a newly released BUET Multi‑Center Colposcopy Dataset. The architecture fuses paired multimodal cervigrams and employs a custom composite loss to handle class imbalance, achieving 71.85% accuracy and 86.23% AUC‑ROC for three‑class CIN grading, and AUC‑ROC values between 75.7% and 88.4% for individual Swede score components. The total predicted Swede Score has a mean absolute error of 1.489, indicating potential for AI‑assisted colposcopy screening in resource‑limited settings.
arXiv:2609.12081v1 Announce Type: cross Abstract: Mobile manipulation requires perceptual evidence at different spatial scales for base motion and arm control, while the two action modalities remain...
Robion is a new serving and management system designed to run Vision‑Language‑Action (VLA) models on multi‑GPU edge servers for robot factories. It splits the VLM and ADiT stages within a single GPU, shares streams across multiple models, and prioritizes requests by remaining SLO time, enabling high robot load while meeting strict latency requirements. In experiments, Robion achieves 6.7× higher robot load than vLLM‑Omni and 1.5× higher than a monolithic pipeline, and can serve 64 robots on a 4‑GPU server with 98% SLO attainment.
The study presents an explainable multimodal deep‑learning framework that combines a 3D CNN for T1‑weighted MRI with a feedforward network for harmonized clinical and demographic data to diagnose Alzheimer’s disease. Using 6,479 ADNI records and 1,703 OASIS‑3 records, the authors compare various model configurations on three‑way and pairwise diagnostic tasks, finding that performance and explanations vary by task, modality, fusion strategy, and cohort. SHAP and Integrated Gradients consistently highlight the MMSE score as the most influential tabular feature, while CAM‑based explanations differ across model setups and cohorts, indicating that explainability is not a stable property under cohort shift.
ReVisIT is a train‑free framework that turns retrieved image‑label pairs into units of visual thought, combining structured class definitions, multimodal retrieval, and alternating user/assistant injection before joint decoding. On several benchmarks—including Fast Open MiniImageNet, Bongard‑OpenWorld, and the newly released MAAC‑Bench—ReVisIT achieves performance comparable to or surpassing large, trained models while using far fewer parameters. The approach demonstrates that high‑quality retrieval and a simple turns layer can provide a universal performance boost across diverse multimodal tasks.
The paper introduces an Early Intervention (EI) framework for multimodal medical image classification that addresses two key challenges: limited exploitation of complementary multimodal information and scarcity of labeled data for Vision Foundation Models (VFMs). EI treats one modality as the target and uses high‑level semantic tokens from other modalities as intervention tokens to guide the target’s embedding early in the process. The authors also propose Mixture of varied‑rank LoRAs (MoR) for efficient VFM adaptation, and demonstrate the method’s effectiveness on retinal, skin, and knee medical image datasets.