Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2608.24680v1 Announce Type: new Abstract: Video games provide a scalable source of training data for video world models, offering diverse environments, complex interactions, and abundant in-the...
arXiv:2608.24053v1 Announce Type: new Abstract: Universal multimodal embeddings are becoming a core component of modern AI systems, enabling heterogeneous content to be represented in a shared space...
RT-NeuS is a neuro‑symbolic framework for long‑form video question answering that retains the accuracy and formal guarantees of temporal‑logic‑guided methods while dramatically reducing inference latency. It achieves this by using coarse‑to‑fine adaptive sampling to focus on query‑relevant frames and batched proposition detection with KV‑cache reuse, enabling all propositions to be evaluated in a single forward pass. Experiments on LongVideoBench, Video‑MME, and MLVU show up to a 13× speed‑up on an NVIDIA H200 GPU while matching or surpassing prior neuro‑symbolic accuracy.
arXiv:2608.23723v1 Announce Type: new Abstract: Few-shot anomaly detection (FSAD) has recently benefited from vision-language models such as CLIP, which enable anomaly de?tection by aligning visual f...
arXiv:2608.24354v1 Announce Type: cross Abstract: MLLMs are increasingly deployed in user-facing applications, yet they inherit backdoor risks from the pipelines used to construct them: triggers may...
arXiv:2608.24674v1 Announce Type: new Abstract: Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal...
MolEmb is a lightweight framework that adapts multimodal large language models (MLLMs) to serve as general molecular embedding models. By aligning molecular profiles with textual descriptions in a shared embedding space using a bidirectional contrastive objective, MolEmb produces embeddings conditioned on both a molecular profile and a natural‑language semantic context. The model performs competitively on molecular property prediction and enables cross‑modal molecule‑text retrieval, while the newly introduced MolCAR benchmark demonstrates that context‑aware molecular embedding is largely a data property of the supervision.
The paper evaluates the use of large language models (LLMs) as judges for assessing conversational voice agents, comparing human judgments with GPT‑4.1 and GPT‑5 across telecom and retail interactions. It examines agreement, metric‑level correlations, and consistency across three evaluation configurations (p0, p1, p2) to determine how reliably LLMs can judge conversational quality and safety. The study finds that LLM‑based evaluation can be effective but its reliability varies by metric and configuration, suggesting a hybrid approach where LLMs handle scalable assessment while humans focus on metrics requiring contextual interpretation.
The paper evaluates the tablet‑2 long‑term memory engine on multilingual text benchmarks and cross‑lingual photo retrieval without lexical matching. Tablet‑2 achieves high accuracy on LongMemEval‑S (95.7%) and moderate accuracy on BEAM‑1M (67.5%), with minimal variance across runs. In multimodal tests, it outperforms BM25 on image‑cell recall and shows significant language‑dependent performance gaps, especially for low‑resource languages.
The paper surveys the evolution of smart glasses from simple capture devices to first‑person intelligence platforms that integrate human perception, context, and action. It introduces a unified framework that formalizes data flow, hardware capabilities, and seven foundational capabilities, and presents an L0‑L5 hierarchy for capture to embodied action. The study also maps nine application scenes, proposes a nine‑dimensional deployment framework, and outlines an evidence ladder for evaluation and trustworthiness.
The paper introduces DoublesEval, a diagnostic framework that uses professional doubles badminton to test visual‑language models’ ability to reason about dynamic multi‑agent interactions. It decomposes rallies into key moments and evaluates models across four dimensions—atomic recognition, intra‑segment composite understanding, cross‑segment causal reasoning, and high‑level tactical abstraction—highlighting specific reasoning failures. The authors also propose TacticCheck, a lightweight consistency checker that improves performance without retraining the models, yet significant gaps remain in tactical reasoning.
PinSieve is a production system that selectively serves vision‑language models (VLMs) for enterprise content‑quality triage, operating only on the grey‑zone cases that lightweight models cannot resolve. The deployed VLM Serving Agent filters 2.05× more non‑actionable items, improves review productivity by 25.7%, cuts operating costs by 16.2%, and delivers signals the same day instead of the next. A governed memory flywheel with selective feedback, audit sampling, and a bounded proposal‑verifier loop further reduces false‑negative rates from 17.73% to 13.29% over six months, while a reasoning review agent audits teacher‑generated rationales for keep/repair/drop decisions. whyItMatters":"The system demonstrates how selective VLM serving and governed feedback loops can substantially improve efficiency, cost, and accuracy in enterprise AI content‑quality pipelines."
The paper reviews the evolution of object‑counting techniques from class‑specific density regression to open‑vocabulary, foundation‑model‑backed counters that can handle visual and textual prompts. It highlights that current evaluation relies on a few saturated benchmarks, leading to models exploiting statistical regularities rather than true generalization. The authors propose a five‑axis taxonomy and audit the literature across domains such as microscopy, remote sensing, crowd counting, and agriculture, identifying six structural contradictions and outlining a roadmap for robust, multimodal evaluation protocols.
arXiv:2601.16520v2 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable progress in visual recognition and semantic understanding, yet precise comp...
arXiv:2608.24430v1 Announce Type: new Abstract: Responsible deployment of face verification systems requires more than accurate decisions: systems should also provide interpretable and auditable evid...
Luce is a 3D representation that unifies geometry and physically based rendering (PBR) materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material‑aware latent space, which a rectified‑flow transformer generates from a single image conditioned on multi‑layer features from a pretrained image encoder. The latent decodes into relightable PBR Gaussians and an optional textured mesh with a tangent‑space normal map, achieving state‑of‑the‑art single‑image‑to‑3D generation on Toys4K and improving CLIP image‑alignment scores on a benchmark of AI‑generated images.
arXiv:2608.24281v1 Announce Type: new Abstract: Reducing annotation requirements remains a key challenge in developing robust medical object detectors. To address this, Vision-Language (VL) object de...
The paper investigates the blending ratio used in few‑shot adaptation of vision‑language models, which combines a zero‑shot text prototype with the mean of labeled image features. It shows that the theoretically optimal ratio—derived from a closed‑form mean‑squared error minimizer—does not align with the ratio that actually maximizes performance, falling short by an average of 8.5 points. Moreover, a leave‑one‑out estimate on the support set achieves near‑oracle performance, and validation‑free linear probes outperform even oracle‑tuned blends, indicating that the hyperparameter can be set near‑optimally without external validation data.
The paper introduces Tlow, a flow-based item tokenizer that transforms raw semantic embeddings into a latent space following a standard normal distribution, enabling independent tokenization and simplifying distributional complexity. Tlow incorporates codebook guidance to align token embeddings with the codebook space, producing semantically clear token IDs. Experiments on four public datasets and an online multi‑modal retrieval task on WeChat show that Tlow improves recommendation performance and increases user click‑through rates by over 10%.
arXiv:2608.23921v1 Announce Type: new Abstract: Recent Vision-Language Models encode high-resolution images into long visual token sequences, incurring prohibitive prefill costs. To compress them, ex...