Vision-language models, speech and cross-modal systems that read, look and listen in the same forward pass.
Robotic foundation models achieve impressive performance on standard manipulation benchmarks, yet these evaluations typically assume clean, timely, and consistent visual observations throughout execut...
Deep Imbalanced Regression (DIR) addresses a common failure mode of regression models: target distributions are highly non-uniform, causing models to perform best in densely populated target regions e...
Continual learning, the ability to learn from sequential experience while retaining and adapting prior knowledge, is central to intelligent systems operating in changing environments. However, convent...
Autonomous endoscopic navigation requires the policy model to predict actions from texture-poor monocular observations, make safe control decisions, and retain evidence of lesions after they leave the...
Visual search on large e-commerce catalogs must serve both "similarity" queries that ask for items resembling an uploaded image and "modifier" queries that comprise an image and text describing a desi...
The paper introduces Jarvis, an offline, edge‑deployable voice assistant designed for autonomous racecars. It combines speech recognition, synthesis, and a lightweight text‑to‑command classifier fine‑tuned from the Mistral 7B model to provide high‑level behavioral commands. Experiments show 97.63 % intent recognition accuracy with an average latency of 1.39 s, outperforming larger online‑hosted models and enabling quick response times for time‑critical driving tasks.
The paper introduces a benchmarking framework that evaluates open-weight Vision‑Language Models (VLMs) for face recognition by treating explanation quality as a core metric. It defines two key criteria for explanations—relevance, meaning reliance on identity‑stable facial features, and faithfulness, meaning alignment with the visible image content without hallucinations. Using this framework, the authors benchmark several VLM families, jointly assessing face verification accuracy and explanation quality, and find that current models still exhibit shortcomings in their explanations, underscoring the importance of explanation metrics for a complete performance assessment.
The paper presents a configurable multi‑stage vision pipeline for Farmer.Chat, a farm advisory service that processes farmer‑submitted crop photos. It splits the task into a quality gate, a crop detector, and a disease/pest detector, offering two routes: a single fine‑tuned vision‑language model and a set of small specialist models. The new pipeline improves crop accuracy to 95.41% and provides a fast MobileNetV3 quality gate, while retaining the ability to request better photos and answer all queries in one call.
PRIME introduces a perception feedback mechanism for Vision‑Language‑Action models in autonomous driving, conditioning perceptual queries on a Situational Memory that aggregates past perception, reasoning, navigation goals, and predicted behaviors via cross‑attention. This approach adds only 29.7 M parameters (0.41 % of a 7.3 B‑parameter base model) and enables intent‑driven perceptual attention at minimal computational cost. On the Bench2Drive closed‑loop benchmark, PRIME achieves a state‑of‑the‑art Driving Score of 82.47 and a Success Rate of 60.00 %, outperforming prior models such as ORION.
The paper introduces an end‑to‑end framework for personalized audio‑driven facial motion that operates in real time without look‑ahead. It combines a causal multi‑resolution motion tokenizer, which captures both global temporal context and fine articulatory details, with a multi‑modal style retriever that pulls stylistic priors from arbitrary reference footage using ongoing audio and motion queries. This approach allows high‑fidelity, identity‑consistent animation from just a few casually recorded clips, outperforming existing methods in lip‑sync accuracy, identity consistency, and perceived realism while maintaining real‑time streaming constraints.
The paper introduces a multimodal reasoning framework for cross‑domain visual question answering in Printed Circuit Board Assembly (PCBA) inspection, converting diverse data sources into a unified instruction format and generating verified reasoning traces. It proposes Task‑Aware Group Relative Policy Optimization (GRPO) that uses semantic, distance‑aware, and format rewards to improve choice‑based and counting tasks beyond exact‑match supervision. During inference, the system applies semantic consistency correction, self‑consistency voting, and multi‑model arbitration, achieving an overall score of 83.24 on the PCBA Standard‑to‑Real Grand Challenge leaderboard.
ProTracer is a training‑free framework that uses Vision‑Language Models (VLMs) together with proprioceptive signals to analyze robot manipulation failures. It performs binary failure detection, categorization, explanation generation, and introduces failure onset localization—identifying the earliest moment a robot deviates from a valid trajectory leading to failure. The method leverages proprioceptive dynamics to pinpoint informative action boundaries and converts robot‑state signals into natural‑language descriptions for joint multimodal reasoning, achieving strong performance on both conventional failure diagnosis and the new failure onset localization task.
The study compares temporal deep learning models—Bidirectional LSTM, Temporal Convolutional Network, and Transformer—for physiological emotion recognition using two multimodal wearable datasets, WESAD and EmoWear. Experiments evaluate wrist-only, chest-only, and multimodal sensor configurations with participant-independent leave-one-subject-out cross-validation, and also explore ensembles, sensor ablation, sampling frequency, and saliency analysis. Results show that the best architecture varies by dataset, multimodal sensing consistently outperforms single-site configurations, and a 4 Hz sampling rate offers a cost-effective operating point.
The study evaluates the robustness of medical vision‑language models for tuberculosis screening on chest X‑rays by testing them across multiple datasets, prompts, and evaluation settings. Three specialized models (BioMedCLIP, CheXficient, MedSigLIP) and a general OpenCLIP model were audited on 12,200 images, producing 244,000 model–image–prompt scores. Results show that no model consistently outperforms others across all cohorts and reliability criteria, with prompt changes and control group composition significantly affecting AUROC, and that high training‑set performance does not reliably transfer to external cohorts.
The paper introduces OverThink, a slowdown attack that forces reasoning language models (RLMs) to produce many more reasoning tokens while still giving correct answers. By injecting decoy reasoning problems—such as Markov decision processes, language translation, or graphic comprehension—into the model’s context, attackers can dramatically increase token generation (up to 46× on SQuAD and 17× on coding agents). The study evaluates the attack on both proprietary and open-source RLMs across multiple datasets, explores multimodal and coding‑agent variants, and tests several defenses, concluding that defending against OverThink is challenging and that newer RLMs are even more vulnerable due to higher per‑token costs and increased reasoning token usage.
MemeTAG introduces a dual‑objective framework for classifying harmful memes by combining keyword generation from a pretrained Vision‑Language Model with an Aggregated Tag Inference Network (ATIN) that condenses these keywords into a rich semantic embedding. The embedding is used as a target for an auxiliary reconstruction loss, encouraging deep alignment between visual and textual features. This approach, along with a three‑stage training strategy, achieves new state‑of‑the‑art results on the HarMeme, Hateful Memes Challenge, and PrideMM datasets.
The paper introduces MultiHU‑TD, an interpretable framework for multifeature hyperspectral unmixing that employs tensor decomposition and incorporates the abundance sum‑to‑one constraint via an ADMM algorithm. It extends previous models by adding mathematical morphology and neighborhood patch analysis, and provides detailed mathematical, physical, and graphical interpretations linked to the extended linear mixing model. Experiments on real hyperspectral images demonstrate the model’s interpretability and effectiveness, with code released on GitHub.
The paper introduces SceneLM, a vision‑language model that maintains an open‑vocabulary 3D scene map using only a structured text list of objects as persistent memory. The model updates this textual map by adding, editing, and removing objects for each input image, learning the process through supervision tasks and an automatic annotation pipeline. Evaluations on language‑grounded retrieval and localization benchmarks show competitive performance with traditional mapping systems while producing a 6‑12× more compact representation, and the model can run online on an edge device such as a quadruped robot.
VeriFuse is a bounded arbitration framework that integrates vision‑language models (VLMs) into vehicle‑infrastructure cooperative 3D perception. Each agent first generates independent detections, then VeriFuse creates a unified candidate pool of geometric proposals and cross‑source hypotheses. A frozen VLM selects among three actions—SELECT, REFINE, or REJECT—to resolve ambiguity and produce final 3D detections, achieving strong AP50/AP70 scores on the DAIR‑V2X dataset while keeping vehicle‑side BEV AP50 drop minimal under delay.
The paper describes Transsion Speech Team’s submission to Task 1 of the MLC‑SLM 2026 Challenge, aiming at speaker‑attributed transcription for multilingual conversational speech. Their cascaded framework includes a DiariZen‑based speaker diarization module, a Qwen3‑Omni‑based long‑form multilingual ASR module with CTC alignment for precise timestamps, and a fusion module that merges diarization and transcription outputs into speaker‑attributed STM results. On the official evaluation set, the system achieved a tcpMER of 15.41% and secured second place among all participants.