Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2608.30428v1 Announce Type: cross Abstract: Strategic deception by LLM and VLM agents has emerged as a central AI alignment and safety concern. Social-deduction games (where each player holds a...
arXiv:2608.29239v1 Announce Type: new Abstract: Low-resource ASR remains difficult because scarce transcripts provide limited supervised evidence for target-side generation. To address this gap, we p...
arXiv:2605.14040v2 Announce Type: replace Abstract: Trackable improvement in multimodal physics reasoning rests on a training-and-evaluation system that is itself rarely verified: the corpora a model...
arXiv:2508.13073v3 Announce Type: replace-cross Abstract: Robotic manipulation, a key frontier in robotics and embodied AI, requires precise motor control and multimodal understanding, yet traditiona...
arXiv:2608.29208v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models, built upon Vision-Language Models (VLMs), have significantly enhanced robotic capabilities by leveraging interne...
TEMPO is a unified model that adds temporally‑grounded capabilities to large audio‑language models, enabling timestamping of events, speakers, and sounds in audio, speech, and music. It introduces a supervised fine‑tuning stage featuring atomic timestamp tokens, a time‑aware projector with sinusoidal encodings, and a distance‑aware Gaussian loss, trained via a synthetic‑to‑real curriculum. Additionally, TEMPO employs reinforcement learning (GRPO) as a refinement step, and achieves state‑of‑the‑art performance on a benchmark of 10K samples across five timestamping tasks, surpassing Audio Flamingo Next and Qwen3‑Omni.
HERBIOME is a modular, end‑to‑end pipeline that automates the digitization of herbarium labels. It combines YOLOv8 for component detection, CRAFT Hezar for word‑level text localization, a fine‑tuned TrOCR model for mixed handwritten and printed text recognition, and GPT‑4o Mini for structuring metadata into standardized fields. Evaluation on 450 French specimens shows high surface similarity (MWS ≈ 0.616) and moderate semantic accuracy (SMA ≈ 0.442), with taxonomic fields identified as the main challenge.
The paper introduces a framework that combines world models, which generate concrete visual rollouts of possible futures, with multimodal large language models (MLLMs) that perform abstract reasoning. It proposes a controlled concrete reasoning approach and a new training method called Privileged‑Future On‑Policy Self‑Distillation (PF‑OPSD), which uses ground‑truth future videos as privileged teacher context during training while the student model never sees true futures at test time. Experiments on two human‑verified benchmarks, VRQABench and OpenWorldQA, show that PF‑OPSD improves performance by about 10–11% over baselines and enhances robustness to noisy or conflicting rollouts.
FRAMEWORKERS is a task‑centric, multi‑agent framework designed for end‑to‑end AI‑generated video production. It uses a central Director to dynamically manage a task stack and an Assistant to execute tasks within a shared Workspace, leveraging modular sub‑agents that can be added without redesigning the workflow. The system is fine‑tuned with supervised learning and policy optimization, outperforming existing LLM planners and fixed pipelines in routing accuracy, failure recovery, and overall video quality.
GarmentWeaver is a new framework for multimodal sewing pattern generation that uses a schema‑aware approach to construct compact hierarchical targets. By activating garment‑relevant structural branches and building on a pretrained vision‑language model, it predicts executable sewing patterns in a structured manner. Experiments show that GarmentWeaver produces more accurate, executable patterns and yields better simulation results than strong baselines.
The paper introduces NavMCP, a scaffolding framework that couples vision‑language models (VLMs) with navigation foundation models (NFMs) to enable long‑horizon physical‑world agents. NavMCP orchestrates three communication channels—intent, observation, and memory—to allow the VLM to decide what evidence to seek and the NFM to ground semantic sub‑goals into closed‑loop navigation, without retraining either model. The approach achieves state‑of‑the‑art results on several embodied question‑answering benchmarks and significantly outperforms episodic interfaces on the Unitree Go2 robot as task horizons lengthen.
RLG-TPV introduces a multimodal Tri-Perspective View framework that fuses camera, radar, and training‑time LiDAR data for 3D object detection. It uses radar and LiDAR to guide a ray‑deformable attention lift, refining depth distributions and providing geometric supervision for side and front planes, while radar cross‑section awareness spreads evidence spatially. On nuScenes, the method attains 0.4981 mAP and 0.5959 NDS, improving orientation and velocity accuracy by about 32 % and 31 % over the CRN baseline.
CogEvol is a family of models designed to generate learning environments—structured JSON slides or interactive HTML pages—from course briefs in a single pass. In production, it completes a slide in a median of 17 seconds and an interactive page in 59 seconds, vastly faster than multi‑turn agent scaffolding. The system enforces reliability through a production‑grounded data pipeline that produces over 53,000 verified SFT samples and a hybrid rule‑plus‑VLM reward that guides GRPO‑based reinforcement learning, with a 27B model achieving high quality scores while using far fewer parameters than flagship coding models.
arXiv:2608.31106v1 Announce Type: new Abstract: Recent video generators often omit audio or synthesize it in a separate stage, limiting reciprocal modeling of visual dynamics and acoustic events. We...
arXiv:2608.30163v1 Announce Type: cross Abstract: Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, ex...
The study shows that jailbreak robustness in language models is highly sensitive to operational-state changes. Even minor alterations to system prompts, not intended to affect safety, can dramatically shift attack success rates across seven aligned models and three jailbreak methods. The authors link these variations to changes in hidden representations along a refusal-related axis, which can predict jailbreak outcomes.
arXiv:2604.07146v3 Announce Type: replace Abstract: Knowledge-based visual question answering (KB-VQA) requires vision-language models to understand images and use external knowledge, especially for...
The paper surveys Multimodal Code Intelligence, focusing on tasks where code is generated, edited, refined, or reasoned about under visually grounded inputs such as screenshots, charts, and videos. It categorizes the field by the role of code—rendered artifact, editable structure, intermediate reasoning trace, or executable tool interface—and organizes benchmarks into four domains: Graphical User Interface, Scientific Visualization, Structured Graphics, and Frontier Tasks and Frameworks. The authors argue that reliable evaluation must include evidence of semantics and interaction beyond visual fidelity, and propose four verification-centered research directions to advance the field toward evidence-grounded executable systems.
arXiv:2608.30935v1 Announce Type: cross Abstract: Embodied navigation requires agents to translate heterogeneous goals and visual observations into actions across tasks, environments, and robot embod...
arXiv:2608.29924v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) are prone to hallucinations: they fluently describe objects, attributes, and scenes that are not in the image. W...