Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
Multimodal Large Language Models (MLLMs) are increasingly deployed as multi-step agents, where explicit reasoning supports task decomposition and tool coordination but also accumulates self-generated...
Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquatic-waste datasets provide limited geographic cove...
Referring Expression Segmentation (RES) aims to generate a pixel-level mask for the object specified by a language expression. Recent methods based on multimodal large language models (MLLMs) often re...
Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottlenecked by two fundamental issues: current data s...
Large language models (LLMs) and vision-language models (VLMs) are expanding the range of behaviors that can be represented in agent-based simulations, but many contemporary platforms are difficult to...
Spatial multi-omics technologies jointly profile gene expression, surface proteins, and histology at each tissue spot, yet most spatial domain discovery methods provide only cluster assignments, witho...
arXiv:2608.20840v1 Announce Type: cross Abstract: Recent advances in multimodal retrieval have improved the ability to retrieve information from visually rich documents such as PDFs and reports. Howe...
TLive-Omni is an omni‑modal understanding model designed for e‑commerce live streaming, integrating image, video, audio, and text inputs into a unified representation. It introduces Per‑vGrid for timestamped token organization, a three‑stage supervised training pipeline, and a Faithful‑RFT reinforcement fine‑tuning stage to enhance answer faithfulness and expression quality. The model is supported by a scenario‑oriented capability taxonomy and a compact data production engine that generates training signals for tasks such as speech recognition, product visual grounding, and omni‑modal QA, achieving strong performance on live‑commerce benchmarks and good generalization to general tasks.
ReFrame is a training‑free framework that enhances safety alignment for multimodal large language models at test time. It uses two lightweight agents: one generates risk and utility evidence, and the other rewrites prompts and routes images to create a safe proxy before invoking the deployed MLLM. Experiments show that ReFrame improves jailbreak defense, safety awareness, and reduces over‑sensitivity while maintaining multimodal utility.
The paper investigates how visual presentation affects vision‑language models (VLMs) on the SPaRC spatial planning benchmark. By adding lightweight input‑side scaffolds that keep the visual modality but make spatial structure clearer, the authors achieve up to a 34.0‑percentage‑point accuracy boost across multiple VLMs, and an additional 4.6 points when combined with GRPO training. Analyses reveal that these improvements stem mainly from reduced grounding errors, while rule‑based reasoning remains difficult, highlighting visual presentation as a key determinant of whether VLM benchmarks test grounded perception, downstream reasoning, or both.
arXiv:2608.21247v1 Announce Type: new Abstract: Token compression has become a key technique for reducing the inference cost of large foundation models, with approaches such as token pruning and KV-c...
TH-GNN is a heterogeneous temporal graph neural network designed to detect shilling attacks generated by large language model (LLM) agents. It combines a two‑layer Heterogeneous Graph Transformer with per‑type and per‑relation attention, learnable sinusoidal temporal encodings, cross‑modal attention that fuses user embeddings with frozen RoBERTa representations of reviews and item descriptions, and a GRU that models log inter‑arrival times. Across five attack families and four benchmark datasets, TH‑GNN achieves a grand‑mean F1 score of 0.870, surpassing the best text‑only baseline on Agent4SR attacks by 10.9 percentage points and 11.5 percentage points at the lowest injection rate.
The paper presents a two‑stage early‑warning system for detecting lithium‑ion battery thermal runaway under mechanical abuse. Stage I estimates localized thermal instability from infrared hotspot dynamics, achieving an ROC‑AUC of 0.945. Stage II fuses this instability score with mechanical, electrical, thermal, and image‑intensity features, reaching an ROC‑AUC of 0.908 and providing a 14.8‑frame mean lead time before voltage‑based detection.
The paper introduces CARD, a diagnostic tool that steers activations along one mental-state axis while measuring predictions on another, to investigate how vision‑language models use internal representations of beliefs, knowledge, and intentions. Using CARD on open‑weight VLMs evaluated on the Relay Chain cooperative grid‑world benchmark, the authors uncover a routing failure: models do not incorporate belief representations into their next action predictions, leaving partner information unused.
arXiv:2608. 20810v1 Announce Type: cross Abstract: Multimodal information systems increasingly route generated visual content back through the same vision-language index that informed its production, so the output must remain retrievable by the queries it was meant to serve.
The paper introduces a multi‑modal traffic sign detection framework that fuses camera and LiDAR data using an Intensity‑Aware Deformable Fusion module to align retro‑reflective LiDAR cues with visual features. It also presents a dual motion‑model tracker to handle non‑linear perspective changes and a semantic attribute classification pipeline that estimates occlusion, readability, sign embeddedness, and road relevance. Evaluated on a dataset covering more than 60 countries and 2,500 hours of driving, the system achieves an Object Miss Ratio of 0.49% across 221,068 sequences, indicating strong global generalization for autonomous driving.
Re$^3$Cap introduces a retrieval‑guided refinement strategy for image captioning that leverages multi‑modal retrieval as a reasoning signal. The method, built on a Caption Refinement Suggester and a Caption Quality Assessor, detects hallucinations and omissions to produce more accurate and detailed captions without extra annotations. Experiments show it surpasses supervised fine‑tuning and improves relation reasoning by 8.64% on the COCO‑LN500 benchmark.
arXiv:2608.20944v1 Announce Type: new Abstract: Multimodal object detection in remote sensing faces challenges due to semantic heterogeneity and modality-specific noise interference. To this end, we...
arXiv:2608.21030v1 Announce Type: cross Abstract: Video multimodal large language models have advanced significantly, yet fine-grained motion-temporal understanding remains fragile. The core bottlene...
arXiv:2511.17886v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) have achieved remarkable success across multimodal tasks, yet their substantial computational demands hinder ef...