Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
arXiv:2608.24535v1 Announce Type: new Abstract: Data visualizations are widely used for communicating information, but they are also vulnerable to intentional manipulations that induce misleading int...
IterCAD introduces an iterative program‑repair framework for generating executable parametric CAD code from dimension‑annotated orthographic drawings. Unlike one‑shot vision‑language models, IterCAD repeatedly evaluates the current CAD output, identifies discrepancies with the target views, and decides whether to revise the code or stop refining. The authors build a structured revise‑or‑stop dataset (IterCAD‑RS) and a three‑stage training pipeline—initial generation, revision learning, and multi‑turn reinforcement learning—to enable this progressive correction, achieving higher code executability and geometric fidelity on the CADExpert benchmark.
The paper reviews the evolution of object‑counting techniques from class‑specific density regression to open‑vocabulary, foundation‑model‑backed counters that can handle visual and textual prompts. It highlights that current evaluation relies on a few saturated benchmarks, leading to models exploiting statistical regularities rather than true generalization. The authors propose a five‑axis taxonomy and audit the literature across domains such as microscopy, remote sensing, crowd counting, and agriculture, identifying six structural contradictions and outlining a roadmap for robust, multimodal evaluation protocols.
arXiv:2608.24354v1 Announce Type: cross Abstract: MLLMs are increasingly deployed in user-facing applications, yet they inherit backdoor risks from the pipelines used to construct them: triggers may...
The paper introduces the Boot-and-Feedback (BooF) framework, which facilitates collaboration between multimodal large language models (MLLMs) and expert vision models for breast ultrasound (BUS) diagnosis. In the Boot Stage, the MLLM is guided by the BI-RADS lexicon and preliminary expert predictions to generate reliable textual descriptions, while the Feedback Stage fuses these descriptions with visual features using an Attention-Gated Cross-Modality Fusion Module, allowing the expert model to incorporate textual insights and filter out noise. Experiments on multiple BUS datasets show that BooF improves both diagnostic accuracy and interpretability compared to existing methods.
arXiv:2608.24563v1 Announce Type: new Abstract: Imaging factor disentanglement in text-to-image generation aims to independently control image acquisition properties such as types of camera lenses, s...
arXiv:2608.24138v1 Announce Type: new Abstract: Large vision-language models have shown strong progress in UI-to-code generation, yet their test-time self-evolution remains unstable. We first identif...
The paper introduces a self‑evolving scientific agent that uses large language models and iterative code generation to build interpretable, physically‑reasoned white‑box controllers for complex systems. The agent deploys candidate controllers in simulations, diagnoses dynamic behavior from multimodal evidence, and refines source code until a robust policy is achieved. Applied to a nonlinear fluid‑structure interaction problem—a two‑joint dogfish swimmer navigating an unsteady wake—the agent autonomously designs a controller that consistently reaches targets across a wide range of conditions without retraining.
arXiv:2608.24688v1 Announce Type: new Abstract: Precision oncology necessitates a longitudinal model of patient state that captures cancer evolution and treatment over time, integrating multimodal ob...
arXiv:2602.06268v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, p...
NeuronGuard is a fine‑tuning defense for large language models that hardens them against both jailbreak and neuron‑level attacks. It redistributes safety signals across many neurons by identifying safety‑critical ones with per‑layer linear classifiers, enforcing refusal behavior when those neurons are ablated, and applying KL‑divergence regularization for consistency. A randomized gradient projection preserves task performance, and the authors provide a formal guarantee that NeuronGuard lowers the attack success rate upper bound, with experiments showing near‑zero success rates across multiple models and attack strategies.
The paper investigates the blending ratio used in few‑shot adaptation of vision‑language models, which combines a zero‑shot text prototype with the mean of labeled image features. It shows that the theoretically optimal ratio—derived from a closed‑form mean‑squared error minimizer—does not align with the ratio that actually maximizes performance, falling short by an average of 8.5 points. Moreover, a leave‑one‑out estimate on the support set achieves near‑oracle performance, and validation‑free linear probes outperform even oracle‑tuned blends, indicating that the hyperparameter can be set near‑optimally without external validation data.
arXiv:2603.00188v2 Announce Type: replace-cross Abstract: Training-free KV cache compression is essential for deploying vision-language GUI agents under memory and latency constraints, yet existing m...
The paper evaluates the tablet‑2 long‑term memory engine on multilingual text benchmarks and cross‑lingual photo retrieval without lexical matching. Tablet‑2 achieves high accuracy on LongMemEval‑S (95.7%) and moderate accuracy on BEAM‑1M (67.5%), with minimal variance across runs. In multimodal tests, it outperforms BM25 on image‑cell recall and shows significant language‑dependent performance gaps, especially for low‑resource languages.
arXiv:2608.24281v1 Announce Type: new Abstract: Reducing annotation requirements remains a key challenge in developing robust medical object detectors. To address this, Vision-Language (VL) object de...
The paper surveys the evolution of smart glasses from simple capture devices to first‑person intelligence platforms that integrate human perception, context, and action. It introduces a unified framework that formalizes data flow, hardware capabilities, and seven foundational capabilities, and presents an L0‑L5 hierarchy for capture to embodied action. The study also maps nine application scenes, proposes a nine‑dimensional deployment framework, and outlines an evidence ladder for evaluation and trustworthiness.
The paper introduces DoublesEval, a diagnostic framework that uses professional doubles badminton to test visual‑language models’ ability to reason about dynamic multi‑agent interactions. It decomposes rallies into key moments and evaluates models across four dimensions—atomic recognition, intra‑segment composite understanding, cross‑segment causal reasoning, and high‑level tactical abstraction—highlighting specific reasoning failures. The authors also propose TacticCheck, a lightweight consistency checker that improves performance without retraining the models, yet significant gaps remain in tactical reasoning.
The paper introduces Giraffe, a new mapping architecture that converts hidden text token representations into visual embeddings for graphic design tasks. It uses a single [IMG] token per image and two shallow MLP blocks—one for training and one for inference—to compress and expand embeddings, trained with six loss functions. The approach achieves strong performance in both image‑to‑design and text‑to‑design generation while remaining lightweight.
Web agents that act from rendered pixels avoid the fragility and heavy token cost of reading a page's HTML or accessibility tree, but training them depends on large amounts of high-quality interaction...
We present LAION-BVD, a large-scale open video dataset for multimodal learning, which contains 1.3B platform-specific video URLs collected from CommonCrawl. From these, we download 80M videos with a t...