Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2608.22916v1 Announce Type: new Abstract: Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unc...
arXiv:2608.22419v1 Announce Type: cross Abstract: Query-based Vision-Language-Action (VLA) models offer low-latency inference that is attractive for bimanual robotic manipulation, but we observe that...
arXiv:2606.04588v2 Announce Type: replace Abstract: Multimodal large language models have made rapid progress in video understanding, yet existing benchmarks largely rely on simple prompts and provid...
The paper introduces TimeCatch, a benchmark that evaluates temporal consistency in vision‑language models (VLMs) by treating temporal grounding as an anomaly detection problem. Temporal anomalies are created by swapping consecutive frames, while frame‑level anomalies involve replacing a frame with Gaussian noise. Across synthetic and real‑world datasets, VLMs reliably detect and localize frame‑level anomalies but perform near chance on temporal anomaly detection, whereas humans excel at both tasks.
Mol-JEPA is a scalable multimodal framework that learns molecular world models by using modality masking instead of suboptimal perturbations. It incorporates diverse data such as molecular structures, cellular phenotypes, binding affinities, ADMET profiles, quantum chemistry simulations, and other drug‑discovery information. Benchmarks show that the representations it learns perform strongly, highlighting the benefit of embedding biochemical context via latent‑space prediction.
arXiv:2608.21796v1 Announce Type: cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) aims to answer queries that necessitate reasoning over external knowledge sources beyond the visua...
arXiv:2608.23329v1 Announce Type: cross Abstract: Open-world video understanding often requires a model to locate sparse visual evidence and acquire external knowledge that is absent from the video a...
The Cultural Moment Benchmark (CMB) evaluates video cultural reasoning in Southeast Asia by testing three distinct abilities: naming a cultural concept, visually recognizing it in a video, and locating its sub‑events in time. It contains 306 expert‑curated concepts from seven countries across five categories, with each concept assessed through three stages that use semantic‑similarity distractors, unlabeled video moments, and free‑form temporal localization. Experiments on six vision‑language models reveal varied failure modes, limited cascading between abilities, and differing impacts of audio and subtitles, while a human study shows even experts struggle with concepts from neighboring countries.
The paper introduces PM4Bench, a multimodal, multilingual, multi-task benchmark built on a strictly parallel 10‑language corpus, allowing fair cross‑lingual comparison of Large Vision‑Language Models (LVLMs). It also proposes a vision setting that embeds textual inputs directly into images to better mimic real deployment scenarios. Experiments show OCR performance drives cross‑lingual gaps, leading to an OCR‑centric GRPO training strategy that improves multilingual VQA and reduces disparities without costly supervision.
arXiv:2608.23354v1 Announce Type: cross Abstract: Autonomous indoor navigation requires both semantic understanding and precise geometric control. We propose OptiSight, a hybrid framework that combin...
arXiv:2605.17610v2 Announce Type: replace-cross Abstract: The rapid growth of online video platforms and AI-generated content has made reliable video guardrails a key challenge for safety and real-wo...
arXiv:2511.17681v2 Announce Type: replace Abstract: Referring Multi-Object Tracking (RMOT) extends conventional multi-object tracking (MOT) by introducing natural language references for multi-modal...
arXiv:2608.21577v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in high-stakes domains where fairness is a critical safety requirement. In practic...
arXiv:2608.22872v1 Announce Type: new Abstract: Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline...
arXiv:2608.23363v1 Announce Type: cross Abstract: Audio-visual deepfake detection is an actively studied topic, where one of the main challenges is to develop detectors able to generalize across deep...
arXiv:2608.22485v1 Announce Type: new Abstract: Current visual tokenizers in Multimodal Large Language Models (MLLMs) predominantly rely on patch-based partitioning, which causes severe semantic mixt...
arXiv:2608.23518v1 Announce Type: new Abstract: Vision-Language Models (VLMs) achieve strong performance in visual reasoning tasks, but it remains unclear whether they understand visual relations, or...
arXiv:2608.22857v1 Announce Type: new Abstract: Vision-language models (VLMs) frequently fail at visual change reasoning, even when their vision encoders contain sufficient information. We observe th...
arXiv:2608.15404v2 Announce Type: replace Abstract: Concept Bottleneck Models (CBMs) are designed to make visual classification interpretable by expressing predictions through human-understandable co...
arXiv:2608.21937v1 Announce Type: new Abstract: Incremental Object Detection (IOD) aims to enable detectors to continuously learn novel categories while preserving previously acquired knowledge. Howe...