The paper presents a semester-long deployment of the VideoPoints platform, featuring a retrieval‑augmented chatbot that answers questions using only the active course’s lecture materials and provides clickable, timestamped citations. Across 833 interactions, 70.5% of messages included citations, no cross‑course references were made, and the bot declined to answer when no lecture evidence matched. The study also shows that the system improves correct‑lecture retrieval by 6.3 percentage points over dense‑only retrieval on the EduVidQA benchmark.
By S M Masrur Ahmed, Jaspal Subhlok
The paper investigates how Vision‑Language‑Action (VLA) models can generalise across different driving environments and camera setups. It introduces a multi‑dataset training strategy and an auxiliary objective called BEV‑Forcing, which injects bird‑eye‑view spatial information into the VLA backbone to improve both in‑distribution and out‑of‑distribution performance on a limited number of camera rigs. The authors observe that while BEV‑Forcing helps when training data is scarce, its advantage diminishes as the number of training embodiments grows, suggesting that scaling diversity may reduce the impact of such auxiliary tasks.
By Caio Azevedo, Stefano Sabatini, Sascha Hornauer, Fabien Moutarde
The paper explores using generative models to translate RGB UAV images into synthetic infrared (IR) images for training vehicle detectors in domains where real IR data is scarce. Various translators—supervised GANs, ControlNet-based diffusion models, and LoRA-ed foundation models—were trained on paired RGB-IR datasets and applied to unseen target datasets to generate synthetic IR data. The synthetic IR images, especially those produced by Stable Diffusion 3.5 with ControlNet, significantly improved detection performance on unseen IR test sets, outperforming RGB and grayscale baselines and narrowing the gap to real IR data.
By Thijs A. Eker, Ella P. Fokkinga, Jan Erik van Woerden, Elfi I. S. Hofmeijer, Sebastiaan P. Snel, Klamer Schutte, Friso G. Heslinga
The study presents a multimodal BrainAGE framework that fuses structural T1-weighted MRI with DeepCBV maps—vascular information synthesized from non‑contrast MRI—to estimate brain age more accurately. Using two separate 3D convolutional neural networks, the combined model achieved a mean absolute error of 3.95 years on cognitively normal controls, outperforming single‑modality models. Saliency analyses showed MRI highlighted white matter and cortical atrophy, while DeepCBV emphasized vascular‑rich regions, and the integrated approach revealed stronger differentiation between stable and progressive MCI, indicating sensitivity to early vascular changes.
By Jordan Jomsky, Zongyu Li, Kay C. Igwe, Yiren Zhang, Max Lashley, Tal Nuriel, Andrew Laine, Scott A. Small, Jia Guo, for the Frontotemporal Lobar Degeneration Neuroimaging Initiative, for the Alzheimer's Disease Neuroimaging Initiative
The paper introduces SPADE, a labelled, multi‑modal dataset for detecting attacks on Signal Phase and Timing (SPaT) messages from the perspective of connected vehicles. Generated via Eclipse MOSAIC, SPADE includes 1.89 million timestep records across six attack classes and one benign class, combining SPaT fields, camera confidence scores, and V2V peer data over 40 features. The dataset, along with generation code and scenario configurations, is publicly released on GitHub to enable reproducible deep‑learning intrusion detection research in C‑V2X security.
By James Di Novo, Hany Ragab, Sylvain P. Leblanc
YesTrack introduces a two‑stage referring multi‑object tracking approach that treats the task as a discriminative Yes/No verification problem, directly using multimodal large language models (MLLMs) without generating captions. It adds lightweight temporal consistency constraints—Temporal Confidence Prior (TCP) and Temporal Reference Propagation (TRP)—to improve reliability and efficiency. The method also extends to generic multi‑object tracking as YesTrack‑MOT, achieving state‑of‑the‑art performance on Refer‑KITTI datasets while remaining efficient even with the smallest Qwen3‑VL variant.
By Quansheng Hu, Qin Sun, Qiansen Dai, Jin Ding, Wan Zhang, Xue Zhou, Jianxiao Zou
Bernini proposes a unified framework that separates semantic planning and pixel rendering for video generation and editing. An MLLM-based planner predicts target semantics in ViT embedding space, while a DiT-based renderer synthesizes pixels conditioned on this plan, text features, and source VAE features for editing. The approach introduces Segment-Aware 3D Rotary Positional Embedding and chain-of-thought reasoning, achieving state‑of‑the‑art performance on diverse video benchmarks.
By Bernini Team, Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan
RILA is an execution‑driven agent that integrates browser rendering into the generation loop for interactive web development. It uses an Action Interaction Verification module to replay reference interactions on generated pages, collecting execution‑aware observations, and an Execution‑aware Rendering Score to jointly assess interaction correctness and visual fidelity during iterative optimization. A data synthesis pipeline further augments training data, enabling RILA to significantly improve interaction and visual quality across foundation models, even outperforming larger one‑shot generators.
By Yilong Guo, Hanqi Chen, Zixiao Ye, Guanzhong Wang, Chen Yu, Zeyu Chen
arXiv:2609. 01649v1 Announce Type: cross Abstract: With the rapid growth of live-streaming e-commerce and digital marketing, abnormal marketing behaviors have become increasingly concealed and coordinated across heterogeneous modalities, challenging platform governance and early risk identification.
By Jingwen Luo, Pinrui Zhu, Yiyan Wang, Zilin Xiao, Jingqi Li, Xuebei Kong, Yan Zhan
The paper introduces a causal and temporal evaluation framework for vision‑language models (VLMs) that tracks how visual input, question text, and generated prefixes influence autoregressive decoding. It defines three step‑indexed causal‑drive metrics—Visual Causal Drive (VCD), Question Causal Drive (QCD), and Prefix Causal Drive (PCD)—using a Structural Causal Model and interventions. Experiments on Qwen3‑VL‑8B‑Instruct and other datasets show a shift from early question and visual guidance to increased reliance on generated prefixes, and demonstrate that QCD and PCD reduce recovery error and improve bias detection.
By Shuyao Xiao, Shengling Wang, Haoyu Niu, Ke Chao, Changwei Xu, Xinran Duan, Chaoyong Jiang
The paper introduces a multimodal framework that learns subcellularly resolved cell embeddings by integrating RNA expression profiles, protein sequence representations, and protein structural information using a cross‑attention architecture. This approach models interactions within distinct subcellular compartments, producing fine‑grained embeddings that capture both molecular expression patterns and functional protein properties. It is presented as the first method to jointly incorporate transcriptomic data, sequence, and structural knowledge for subcellularly resolved cell representation.
By Zhen Zhou, Jiachen Li, Yuan Liu, Xiaoyong Pan, Hong-Bin Shen
Multimodal Language Models as Text-to-Image Model Evaluators presents MT2IE, a framework where a multimodal large language model generates evaluation prompts and scores images, achieving higher correlation with human judgment than prior metrics. MT2IE recovers official T2I model rankings using only 20 prompts—far fewer than traditional benchmarks—and adapts prompts to each model’s performance, maintaining informative scoring ranges. The approach demonstrates that dynamic, interactive evaluation can replace static benchmarks as T2I models improve.
By Jiahui Chen, Candace Ross, Reyhane Askari-Hemmat, Koustuv Sinha, Melissa Hall, Amy Zhang, Michal Drozdzal, Adriana Romero-Soriano
UTP-Bench is a new benchmark for uncertainty-aware travel planning that evaluates large language models on their ability to generate robust itineraries under real-world stochastic conditions. The dataset covers 504 Indian cities, incorporating attractions, restaurants, accommodations, and multi-modal transportation networks, and includes empirical delay distributions and crowd-density patterns to simulate realistic disruptions. Three new metrics—Buffer Adequacy Score, Crowd-Aware Timing Score, and Transport Delay Absorption Score—measure how well generated plans maintain robustness against transit delays and crowd variability, revealing significant gaps between state-of-the-art LLMs and human-authored itineraries.
By Etcharla Revanth Rao, Priyanshu Karmakar, Shubhojit Mallick, Manish Gupta, Shreya Ghosh, Abhik Jana
AVERT is a method for spoken dialogue state tracking that improves upon a per-turn text editor by incorporating an audio-conditioned verifier to score candidate slot values. It addresses three types of recoverable errors—inconsistent values across turns, omitted slots, and values unsupported by audio—using three specialized operators: vote, add, and swap, each limited to relevant slots. On the SpokenWOZ dataset, AVERT achieves a joint goal accuracy of 40.13, surpassing both a base speech-LLM (33.04) and a text editor (38.34) without retraining, and matching the performance of a larger end‑to‑end system that processes the full spoken history.
By Chunggi Lee, Hanspeter Pfister
ChartAttack is a framework that evaluates how multimodal large language models (MLLMs) can be misled by design misleaders to produce charts that cause incorrect interpretations. The authors also present AttackViz, a chart question‑answering dataset that labels effective misleaders and their induced wrong answers. Experiments show that ChartAttack can reduce MLLM QA accuracy by 17.2 points in‑domain and 11.9 points cross‑domain, and that fine‑tuning on AttackViz improves robustness to misleading charts.
By Jesus-German Ortiz-Barajas, Jonathan Tonglet, Vivek Gupta, Iryna Gurevych
Vision‑Language Models (VLMs) are increasingly replacing traditional OCR for document understanding, but this study shows they often rewrite imperfect text into more plausible forms, a flaw that clean‑text OCR benchmarks miss. The authors created FaithC4, a multilingual perturbation benchmark of 1,455 single‑page documents with scramble, random substitution, and visually similar substitution attacks, and evaluated 15 systems across general‑purpose VLMs, OCR‑specialized VLMs, and traditional OCR pipelines. Results reveal that general‑purpose VLMs suffer up to 6.9 WER points under perturbation, OCR‑specialized VLMs 0.1–3.4 points, and traditional OCR less than 0.8 points on English; probing Qwen3‑VL‑4B shows rewriting occurs only when a perturbed word’s final‑layer representation remains close to the original, with short words (4–6 characters) rewritten up to 10% of the time.
whyItMatters":"The findings highlight a critical limitation of VLMs in document transcription, underscoring the need for robust evaluation benchmarks that capture rewriting behavior beyond clean‑text accuracy."
By Gwang Gook Lee, Kenan Emir Ak, Jay Mohta, Yan Xu, Dimitrios Dimitriadis
OptimusKG is a multimodal biomedical labeled property graph that integrates structured and semi‑structured resources to preserve detailed, type‑specific metadata across molecular, anatomical, clinical, and environmental domains. The graph contains nearly 191,000 nodes, over 21.8 million edges, and more than 67 million property instances derived from 18 ontologies, with a top‑level schema that enforces node and edge constraints while retaining granular provenance. Validation using the PaperQA3 agent found that 70.0% of sampled edges are supported by literature evidence, and the graph offers a standardized resource for machine learning, knowledge‑grounded retrieval, and hypothesis generation in biomedical research.
By Lucas Vittor, Ayush Noori, I\~naki Arango, Joaqu\'in Polonuer, Sam Rodriques, Andrew White, David A. Clifton, Marinka Zitnik
SonicCaps is a large-scale audio captioning dataset featuring approximately 15 million captions paired with 700,000 audio clips, created using the Qwen3-Omni multimodal language model. The dataset emphasizes diversity by generating around 24 captions per clip through structured prompt engineering and few-shot generation, covering main descriptions, rephrased variants, and semantic tags. Human evaluations rate SonicCaps higher than existing datasets, and training CLAP models on it improves audio retrieval and zero-shot classification across public and commercial benchmarks.
By Zineb Lahrichi, Marc Ferras, Ga\"el Richard, Geoffroy Peeters
The paper introduces OpenRef, a benchmark for Referring Expression Comprehension (REC) designed for open‑world scenarios. OpenRef expands beyond simple settings by including diverse visual domains, variable target counts (multi‑target and none‑target), and a rich vocabulary with proper nouns, polysemous words, and ordinal terms. It also proposes new evaluation metrics—F1 for grounding accuracy and N3R for negative expression rejection—and presents a training‑free Multi‑task Consistency Checker (MCC) that improves model performance with a single click.
By Zongjian Wu, Lei Zhang
DiDrive introduces a risk‑aware hierarchical diffusion framework for offline reinforcement learning in autonomous driving. It combines a low‑level risk‑gated encoder with a high‑level contextual modulator to filter redundant state information, and a 3DICE policy optimization that reduces out‑of‑distribution overestimation and stabilizes gradients. On the CARLA benchmark, DiDrive outperforms baselines such as IQL, CQL, and Diffusion‑QL, achieving an 85% success rate and a 4295.68 average reward in dense traffic with 60 vehicles.
By Qisong Guo, Jingtang Chen, Zhilin Chen, Pei Xu, Mingjian Fu, Wenxi Liu, Yuanlong Yu