Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2608.23172v1 Announce Type: new Abstract: Large-scale vision-language models (VLMs) have demonstrated remarkable versatility across a wide range of multimodal tasks. However, understanding humo...
arXiv:2608.23224v1 Announce Type: cross Abstract: Retrieval can efficiently and effectively augment a frozen vision--language--action (VLA) policy without retraining, yet retrieved text becomes a con...
arXiv:2608.21440v1 Announce Type: cross Abstract: Vision-language-action (VLA) models have advanced end-to-end autonomous driving by leveraging foundation models for semantic reasoning and long-tail...
The paper introduces MemeMind, a large-scale dataset for detecting harmful memes that includes a detailed taxonomy and Chain-of-Thought reasoning annotations. It also proposes MemeGuard, a multimodal framework that uses a three-stage training strategy to improve visual understanding, reasoning, and discrimination of harmful content. Experiments show MemeGuard surpasses current state-of-the-art methods on MemeMind, advancing detection accuracy and interpretability.
arXiv:2608.22916v1 Announce Type: new Abstract: Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unc...
arXiv:2608.21808v1 Announce Type: new Abstract: Multimodal Retrieval-Augmented Generation (RAG) with visual citation is crucial for ensuring the traceability and verifiability of MLLMs. However, curr...
arXiv:2608.23268v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical reasoning. Parameter-le...
arXiv:2608.22337v1 Announce Type: cross Abstract: Speech-guided referring video object segmentation aims to recover the mask tracks of objects specified by a spoken motion description. Here, speech c...
arXiv:2608.21714v1 Announce Type: new Abstract: Recent page-image retrievers such as ColPali have improved retrieval over visually rich documents, yet little is known about how they behave in cross-l...
arXiv:2608.21606v1 Announce Type: new Abstract: Machine unlearning aims to remove the influence of targeted training data from a model while preserving its remaining capabilities, but evaluating whet...
The paper introduces CloudEdgeVLA, a cloud‑edge policy for Vision‑Language‑Action models that treats temporal misalignment as a representation‑learning problem. It encodes delayed observations into slowly varying task features on the cloud while a lightweight edge head fuses the latest cloud feature with current local vision. Experiments on four LIBERO suites show that CloudEdgeVLA retains 63.8–78.0% success under a 40‑step delay window, far outperforming VLASH and single‑path baselines.
arXiv:2608.21941v1 Announce Type: new Abstract: Accurate assessment of patients in intensive care units (ICUs) is essential for timely clinical intervention and improved patient outcomes. Multimodal...
arXiv:2608.22869v1 Announce Type: cross Abstract: While Vision-Language-Action (VLA) models have leveraged internet-scale pretraining and task-focused finetuning to achieve strong performance on long...
arXiv:2605.17949v2 Announce Type: replace Abstract: Remote sensing vision-language models (RS-VLMs) commonly employ a pretrained vision encoder and a projection module to map image features into the...
arXiv:2608.22950v1 Announce Type: new Abstract: Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquat...
SynerMedGen is a unified framework that aligns medical multimodal understanding with generation tasks through task alignment. It introduces three generation‑aligned understanding tasks and a two‑stage training strategy that transfers representations learned during understanding to medical image synthesis. The model achieves strong zero‑shot performance on 22 synthesis tasks and outperforms state‑of‑the‑art specialized and unified models when combined with generation training, supported by a new 1M‑sample SynerMed dataset.
The paper presents a multimodal 3D patch-based U‑Net for generating pseudo‑CT images from non‑attenuation‑corrected PET (NAC‑PET), MRI, and 2D topograms. It employs separate PET and MR encoders, multi‑scale feature fusion, and FiLM‑based topogram conditioning at the bottleneck to integrate complementary cross‑modal information while reducing dependence on precise voxel‑wise correspondence. The authors participated in the BIC‑MAC Challenge and released their final model on GitHub.
The paper introduces CORA (Counterfactual, Observable Redundancy Audit), a protocol for auditing website redundancy by measuring repetition load, normal-use tax, and failure-domain recovery reserve. Each audit run records screenshots, stable element identities, and task traces, while a versioned vision‑language model generates annotations that are validated and released only if they meet calibrated criteria. Experiments on a transparent mechanistic testbed show that CORA’s factorized representation separates reserve from normal-use tax and predicts perturbed success more accurately than scalar-load baselines, but it withholds automated scores when instruments fail to meet release requirements, indicating that CORA is an auditable candidate procedure for the studied benchmark rather than a universal standard.
The paper introduces VERDICT, a method for validating optical chemical structure recognition (OCSR) outputs by leveraging agreement among multiple recognizers rather than pixel‑space re‑rendering. On 263 ACS journal images, agreement achieved an AUROC of 0.916, far surpassing the 0.547 AUROC of re‑rendering similarity. VERDICT was applied to PMC Open Access, yielding over 6,000 high‑precision structure labels, and is integrated into SES AI’s Molecular Universe platform for image‑based molecular search.
The paper introduces two minimal simulation foundations—SD-AgentFoundry-2D and SD-AgentFoundry-3D—for educational and rapid prototyping use with large language models (LLMs) and vision-language models (VLMs). SD-AgentFoundry-2D offers a 2D multi‑agent environment where LLM agents move, communicate, and react to local events such as fire, while SD-AgentFoundry-3D provides a 3D digital‑twin setting where a VLM interprets first‑person images to generate natural‑language movement instructions. Both frameworks run locally on macOS, Windows, and Linux, are intentionally lightweight, and are open for modification rather than being finished applications.