Event-Causal RAG (EC‑RAG) is a lightweight retrieval‑augmented framework designed for reasoning over ultra‑long and streaming videos. It segments video streams into semantically complete events using a dual visual‑audio sentinel mechanism, representing each event as a State‑Event‑State (SES) structure that captures pre‑event, event, and post‑event states. During question answering, bidirectional graph retrieval accesses relevant predecessor and successor events from a dual vector‑graph memory, and answers are generated using both this structured memory and the corresponding video evidence. The authors also introduce ECV‑1H, an hour‑scale long‑video QA benchmark with over 150 hours of untrimmed video and 1,251 human‑annotated QA pairs, where EC‑RAG achieves significant accuracy gains across multiple video foundation models while maintaining efficient streaming memory usage on a single RTX 5090 GPU.
By Peizheng Yan, Yu Zhao, Liang Xie, Juntong Qi, Mingming Wang, Erwei Yin
The article reviews 55 machine‑learning studies on autism spectrum disorder (ASD) published between 2017 and 2023, focusing on how ML can aid early diagnosis and treatment. It finds that supervised learning dominates current research, while deep learning is gaining traction as data volumes grow. The review highlights the need for models that fuse complex data—such as genetic, clinical, wearable, and biometric sources—to improve diagnostic accuracy and enable continuous, non‑intrusive monitoring.
By Rafael Mu\~noz-Terol, Jes\'us Peral, Sandra Amador, David Gil
The paper introduces HN-CLIP, a new objective for dense-caption retrieval that adapts similarity margins per negative example using the text encoder’s own geometry. By adding a detached caption‑similarity matrix to the negative logits, HN‑CLIP addresses the issue of near‑duplicate captions that cause premature loss saturation in InfoNCE training. Experiments on four benchmarks show that HN‑CLIP outperforms leading methods by 2.4–4.3 R@1, trains 2.4× faster than GOAL and 5.4× faster than StructXLIP, and achieves the best full‑data baseline with only 20% of the training data.
By Haoyue Liu, Ye Chen, Zhichao Wang, Xiaoying Tang
The paper introduces PAFIR, a Personalized and Adaptive Feature selection framework that treats feature selection as a reinforcement learning problem over longitudinal multimodal health data. PAFIR jointly models structural dependencies among assessment variables and temporal dynamics in wearable-derived physical activity data, learning adaptive selection policies across repeated study visits using reward signals from sparse fall incidence outcomes. Applied to the Physio Feedback Exercise Program (PEER) trial, PAFIR outperforms state‑of‑the‑art baselines by capturing longitudinal and structural patterns of feature relevance, enabling dynamic, subject‑specific feature selection for more timely fall prevention strategies.
By Chang Liu, Ladda Thiamwong, Yanjie Fu, Rui Xie
EgoMemReason is a new benchmark for week‑long egocentric video understanding that focuses on memory‑driven reasoning rather than simple perception tasks. It tests three memory types—entity, event, and behavior—across 500 questions, each requiring evidence from an average of 5.1 video segments and 25.9 hours of backtracking. Evaluation of 17 models shows that even the best achieves only 39.6% accuracy, highlighting the difficulty of long‑horizon memory in multimodal systems.
By Ziyang Wang, Yue Zhang, Shoubin Yu, Ce Zhang, Zengqi Zhao, Jaehong Yoon, Hyunji Lee, Gedas Bertasius, Mohit Bansal
arXiv:2608. 18339v1 Announce Type: cross Abstract: Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during inference.
By Qi Yu, Zhichen Zeng, Katherine Tieu, Xiyuan Yang, Ruizhong Qiu, Yuchen Yan, Lihui Liu, Yanjun Zhao, Lingjie Chen, Jingrui He, Hanghang Tong
DentAgent is an evidence‑centric multi‑agent framework designed for multimodal dental reasoning. It coordinates five specialized agents that process domain knowledge, radiographs, intraoral photographs, and 3D dental data, converting observations into structured evidence records. The Evidence Blackboard tracks coverage, gaps, and conflicts before generating responses, and the system outperforms senior specialists by 17.3 percentage points on multi‑label diagnosis across four benchmarks.
By Zijie Meng, Xiwei Dai, Yixuan Tang, Jin Hao, Yang Feng, Fudong Zhu, Xiaoqiang Liu, Shaosheng Cao, Zuozhu Liu
The paper discusses the "embodiment gap" in robot foundation models, highlighting that while models can generalize across tasks, additional work is often needed to deploy them on specific robot bodies. It surveys what components can be reused across different robot embodiments and what must be implemented anew, mapping existing methods along axes of shared structure and adaptation stage. The authors propose a reporting framework to better assess adaptation efforts and identify remaining challenges for cross-embodiment learning.
By Yukiyasu Domae, Keisuke Shirai, Hanbit Oh, Ryoichi Nakajo, Tomohiro Motoda, Koshi Makihara, Masaki Murooka, Takuma Yagi, Yoshiaki Bando, Ryo Hanai
The paper discusses how recent progress in deep learning has improved Natural Language Processing and Vision‑Language Models, but also introduced new vulnerabilities, particularly backdoor attacks that threaten security. It explores methods for analyzing, detecting, and designing such attacks in both NLP and VLMs. Additionally, it proposes efficient multimodal representation techniques aimed at clinical and medical imaging applications.
By Weimin Lyu
The paper introduces TTSD‑FAR, a test‑time self‑distillation framework that adapts large video‑language models to missing‑modality scenarios in emotion recognition. A frozen teacher trained on complete modalities guides a low‑rank student, while Fisher‑Anchored Restoration monitors Fisher information to prevent drift and restore the student when distribution shifts occur. Experiments on MELD, DFEW, and BAH with up to 50% missing modalities show TTSD‑FAR consistently outperforms entropy‑based adaptation, retrieval‑augmented generation, and perplexity‑based generation, maintaining performance over long adaptation horizons.
By Muhammad Haseeb Aslam, Alessandro Koerich, Marco Pedersoli, Ali Etemad, Eric Granger
GS‑VLA introduces a lightweight, plug‑and‑play framework that uses a 4 M‑parameter 3D‑Gaussian canonicalizer to adapt frozen Vision‑Language‑Action (VLA) policies to viewpoint shifts without retraining the policy. By treating viewpoint changes as a localized novel‑view synthesis problem under a locality assumption, the method normalizes observations through a scene‑ and policy‑independent disocclusion task. Experiments on the LIBERO benchmark demonstrate that GS‑VLA recovers a large portion of performance lost due to camera displacement, improving results across different policy architectures, unseen task suites, and perturbation scales.
whyItMatters":"The approach offers a computationally efficient alternative to costly fine‑tuning or generative augmentation, enabling robust VLA deployment in real‑world settings where camera configurations may vary."
By Yechan Park, HyunJin Kim
The paper evaluates whether audio‑language models can use multimodal clinical context to improve dysarthric speech recognition. Using a benchmark built on the Speech Accessibility Project dataset, the authors test diagnosis labels, clinician ratings, and detailed clinical descriptions as prompts for nine models. They find that these prompts yield negligible or negative effects on word error rate, though fine‑tuning with LoRA and mixed prompt formats reduces WER by 52% and benefits certain subgroups such as Down syndrome and mild‑severity speakers.
By Pehu\'en Moure, Niclas Pokel, Bilal Bounajma, Yingqiang Gao, Roman Boehringer, Longbiao Cheng, Shih-Chii Liu
The review examines how Large Language Models (LLMs) enhance nanophotonics design by providing semantic interfaces, code generation, and tool orchestration. It traces the evolution from classical neural networks to transformer-based models and categorizes LLM applications into surrogate models that map structure to spectrum and agentic systems that generate code and orchestrate simulations for closed-loop optimization. The article also highlights potential cross-disciplinary uses of LLMs in materials science and wireless communications, and envisions future multimodal foundation models that actively collaborate in autonomous scientific discovery.
By Huanshu Zhang, Kegeng Tang, Lei Kang, Sawyer D. Campbell, Zihao Wang, Douglas H. Werner
MAVEN is a hierarchical framework for evaluating whether multimodal content aligns with macro‑societal values such as peace, justice, and freedom. It organizes values into six primary dimensions and 72 secondary indicators, enabling multi‑level quantitative scoring. The authors build a human‑verified multimodal benchmark, a soft‑match metric, and propose efficient evaluator optimization techniques, demonstrating that a compact 2B evaluator performs comparably to larger models and approaches state‑of‑the‑art closed‑source VLMs.
By Zijuan Zhao, Zheren Fu, Hou Xia, Licheng Zhang, Yi Liu, Zhendong Mao
Learning-based global point cloud registration has achieved remarkable progress, yet its reliance on geometric representations makes existing methods sensitive to variations in point density, scan pattern, viewpoint, and sensor characteristics. We propose CVSD-Reg, a robust global LiDAR registration framework that distills visual semantic priors from a vision foundation model into LiDAR representations.
State-of-the-art vision-language-action (VLA) models such as $π_{0.5}$ exhibit strong semantic understanding, instruction following and task behavior. However, when deployed on new robots, even minor...
Split-conformal prediction provides marginal coverage under exchangeability and is increasingly used as an abstention layer for zero-shot vision-language models (VLMs). We audit this practice under de...
The paper introduces Harness Continual Learning (HCL), a paradigm where an agent’s state evolves through prompts, memories, tools, skills, and routing rules while keeping the foundation model frozen. HCL defines harness-level forgetting and proposes guarded harness evolution with a Continual Optimizer and Evaluator to balance improvement, retention, and validity. Experiments across textual reasoning, multimodal perception, and open‑world interaction show over 10% performance gains and demonstrate explicit control over the stability–plasticity trade‑off.
The paper investigates how Vision Language Models (VLMs) can be fooled by tiny, human‑imperceptible changes to images. It introduces a gradient‑based attack that targets only the vision encoder, reducing computational cost while still effectively disrupting both untargeted and targeted multimodal interpretations. Experiments on open‑source VLMs such as Qwen2.5‑VL, Granite‑Vision, FastVLM, and Phi‑3.5‑Vision demonstrate that these small perturbations can dramatically alter the models’ textual outputs.
The paper examines how multilingual medical adaptation affects the internal representations of Whisper ASR models by performing layer‑wise encoder analysis. It compares several adaptation strategies—zero‑shot decoding, English‑only fine‑tuning, German‑only diagnostic fine‑tuning, two‑stage EN→EN+DE continuation, and direct EN+DE fine‑tuning—across different Whisper sizes, finding that fine‑tuning improves performance but the best model varies by setting. Layer‑wise results show that English medical fine‑tuning drives the main encoder shift, while multilingual continuation largely preserves the adapted representation space, with domain and language information remaining recoverable across layers.
"whyItMatters":"The study provides insight into how multilingual medical adaptation reshapes Whisper’s internal representations, guiding the selection of model sizes and fine‑tuning strategies for improved MedASR performance."