Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2608.21099v1 Announce Type: cross Abstract: Multi-modal object detection is essential for robust scene understanding in challenging conditions, including low-light and adverse environments. Rec...
arXiv:2608.20969v1 Announce Type: new Abstract: Multimodal clinical AI is limited by weakly aligned inputs and the absence of domain-specific interpretable representations, particularly when learning...
arXiv:2608.20905v1 Announce Type: new Abstract: Face-to-face audiovisual interaction is central to human communication, conveying rich emotional and social cues. However, existing multimodal dialogue...
ClawSentry is an open‑source, framework‑agnostic security supervision gateway designed to protect autonomous large language model (LLM) agents from progressive risks that can arise at four points in the agent control loop: skill admission, invocation‑time intent, execution‑time effect, and post‑action consequence. It introduces a multi‑tier decision engine—deterministic L1, rule‑anchored L2, and read‑only L3—alongside a First‑Use Skill Package Review (FSPR) and an Agent Harness Protocol (AHP) that applies a single policy across multiple agent runtimes without modifying their internals. Evaluation on SkillInject and the SkillsSafety benchmark shows that ClawSentry significantly reduces contextual adversarial skill risk (ASR) while maintaining high task success rates (TSR).
arXiv:2608.20999v1 Announce Type: new Abstract: Multimodal LLMs apply the language model interface to visual inputs, where ordinal regression tasks such as age estimation, image quality assessment, a...
arXiv:2608.15698v2 Announce Type: replace Abstract: Visual document retrieval is a critical component of multimodal retrieval-augmented generation, aiming to identify query-relevant pages from docume...
The paper introduces a multi‑modal traffic sign detection framework that fuses camera and LiDAR data using an Intensity‑Aware Deformable Fusion module to align retro‑reflective LiDAR cues with visual features. It also presents a dual motion‑model tracker to handle non‑linear perspective changes and a semantic attribute classification pipeline that estimates occlusion, readability, sign embeddedness, and road relevance. Evaluated on a dataset covering more than 60 countries and 2,500 hours of driving, the system achieves an Object Miss Ratio of 0.49% across 221,068 sequences, indicating strong global generalization for autonomous driving.
arXiv:2608.20890v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models have emerged as a powerful paradigm for end-to-end autonomous driving by jointly integrating perception, reasoning,...
The paper investigates how vision‑language models (VLMs) can be used for training‑free video anomaly detection (VAD) by answering questions about video segments. It shows that the way a VLM’s output is converted into an anomaly score—either by taking only the most likely answer (generated readout) or by using the full probability distribution (probability readout)—has a significant impact on ranking performance. Across four large VLMs, the probability readout consistently outperforms the generated readout, achieving 5–13 point gains in AUROC or AP, because the generated readout compresses the rank order into only a few distinct scores.
arXiv:2608.20720v1 Announce Type: new Abstract: Open-world 3D affordance grounding requires localizing functional object parts in 3D given free-form language queries. Existing methods typically assum...
arXiv:2608.20394v1 Announce Type: cross Abstract: Industry pipelines that turn speech into supervised fine-tuning (SFT) data via multi-stage refinement are increasingly adopted but, to our knowledge,...
arXiv:2608.20868v1 Announce Type: cross Abstract: Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information ext...
The paper introduces CRATE, a two‑stage vision‑language model framework that evaluates mobile agents by reasoning about each step’s consequences and aggregating this evidence to assess task completion. It also presents CRATE‑S, an extension that evaluates operational safety. Experiments show CRATE and CRATE‑S outperform existing benchmarks, achieving high F1‑scores on AndroidWorld and MobileRisk datasets.
arXiv:2608.20984v1 Announce Type: cross Abstract: Narratives are central to how social communication is framed, making their detection critical for understanding and analysing public discourse. Prior...
SPARC (Single-Pass Adaptive Risk Calibration) is a Bayesian‑conformal uncertainty layer for human motion forecasting that adds an analytic epistemic scale to a deterministic MLP backbone’s Gaussian covariance. The scale, κ_t(x), inflates the covariance without altering its correlation structure, enabling 95% marginal prediction tubes with finite‑sample validity via split conformal calibration. Across nine dataset‑protocol blocks, SPARC outperforms baselines on NLL and a combined MPJPE+NLL metric while maintaining competitive point accuracy and efficient calibrated tubes.
BC-Bench is a new benchmark for assessing agentic engineering systems on the domain‑specific language AL used in Microsoft Dynamics 365 Business Central. It contains 101 manually curated tasks drawn from real Microsoft production repositories and evaluates not only code generation but also test creation and multimodal problem statements. The study shows that performance gaps between models are larger than those between different agent harnesses, and that gains on general‑purpose benchmarks do not reliably transfer to the AL domain.
The paper introduces the Generative Embedding Benchmark (GEB), which evaluates how much content from an embedding can be recovered by a decoder that only has access to the frozen embedding and a question, without the original image or intermediate features. GEB uses a curated visual‑question‑answering dataset with 1,800 development and 900 test items covering natural images, scene text, and visual documents. Experiments on seven public embedding models show that visual‑only scores range from 28.25 to 33.21, while joint image‑question encoding boosts scores up to 65.56, revealing that generative readout uncovers information bottlenecks not captured by traditional separability‑based benchmarks.
arXiv:2608.21140v1 Announce Type: cross Abstract: Reliable spatial understanding is an important prerequisite for future medical vision-language systems that aim to support radiological report genera...
TRACE is a new framework that uses agentic Large Language Models to automatically enrich e-commerce product catalogs with missing or buried attributes. It employs a ScoutAgent to gather multimodal evidence from merchant catalogs, syndicated feeds, and web search, and a JudgeAgent to verify and publish the proposed attribute values. In offline evaluation, TRACE achieved 98.2% accuracy with 74.7% coverage, and in production it increased enrichment coverage by 90.4% and boosted checkout conversion by 0.48%.
The paper introduces OraRL, a reinforcement learning framework that leverages annotations as oracle rollouts to improve sample efficiency and scalability for video multimodal large language models (MLLMs). By decoupling advantage estimation and employing sign‑balanced pruning, OraRL achieves faster training and better performance across multiple video‑perception benchmarks compared to existing methods. The approach scales from 0.8B to 9B parameters and handles up to 100k prompts, delivering significant gains in temporal mIoU, tracking accuracy, segmentation, and spatial‑intelligence metrics.