TAME is a CLIP‑based framework for Text‑Video Retrieval that incorporates temporal modeling through three key innovations: sparse Mixture‑of‑Experts layers with frame‑consistent routing, Frame‑Temporal tokens that aggregate cross‑frame information, and a Cross‑Temporal Interaction and Aggregation module for refining frame‑wise similarities. These components enable the model to capture both local visual patterns and long‑range temporal dependencies, leading to consistent performance gains over CLIP‑based baselines on multiple TVR benchmarks, including a 4.0 R@1 improvement on MSR‑VTT. The code is publicly available on GitHub.
By Uicheol Jung, Juyoung Hong, Hojung Kwon, Yukyung Choi
LookStep is a new end‑to‑end framework for Vision‑Language Navigation that integrates Language‑Centric Future State Modeling with an Event‑Driven Rolling Memory. It uses language labels to predict coarse navigation progress and future states for candidate actions, and autonomously decides which observations to store in a bounded memory with semantic roles. Empirical results show that LookStep outperforms existing methods on VLN‑CE tasks, achieving a 49.7% success rate on R2R‑CE Val‑Unseen while improving memory efficiency and reducing data requirements.
By Kun-Yang Yu, Yingzhe Li, Hongyu Xu, Shi-Yu Tian, Zhi Zhou, Yang Chen, Ming Yang, Sheng Wang, Qing Yu, Lan-Zhe Guo, Yu-Feng Li
Beauty assessments from Multimodal Large Language Models (MLLMs) are increasingly popular, prompting a study comparing 2,513 human ratings to four commercial AI models—Claude, Gemini, GPT, and Grok. The study found that MLLMs consistently rate faces more favorably and with a narrower range than humans, yet they maintain strong correlations with human judgments and accurately track the rank‑ordering of faces. While all models agree strongly with each other, Grok showed the lowest agreement with human ratings, and only face age emerged as a common predictor of attractiveness across humans and MLLMs.
By Santiago Grandas, Juan Sebastian Cely-Acosta, Mohit Mendiratta, Shafee Hassan, Macken Murphy
RILA is an execution‑driven agent that integrates browser rendering into the generation loop for interactive web development. It uses an Action Interaction Verification module to replay reference interactions on generated pages, collecting execution‑aware observations, and an Execution‑aware Rendering Score to jointly assess interaction correctness and visual fidelity during iterative optimization. A data synthesis pipeline further augments training data, enabling RILA to significantly improve interaction and visual quality across foundation models, even outperforming larger one‑shot generators.
By Yilong Guo, Hanqi Chen, Zixiao Ye, Guanzhong Wang, Chen Yu, Zeyu Chen
The paper introduces TIC‑Bench, a new benchmark for evaluating multimodal large language models on deeply interleaved text‑image contexts. It covers logical, temporal, and spatial association tasks, totaling 2,280 questions across eight specific types. The authors benchmarked ten state‑of‑the‑art MLLMs, finding a significant performance gap versus human experts and highlighting persistent challenges in integrating evidence across interleaved visual and textual inputs.
By Zihao Wang, Xi Xiang, Yuwen Sun, Yingyu Li, Yabo Zhang, Yihan Zeng, Fan Li, Wangmeng Zuo
The paper evaluates federated learning with Low‑Rank Adaptation (LoRA) for fine‑tuning the BiomedCLIP vision‑language model on chest X‑ray classification across four international cohorts. Federated LoRA improves shared‑class AUC from 0.687 to 0.802, outperforming isolated single‑cohort training and approaching a centralized reference. The study shows that SVD‑based product‑space aggregation (FlexLoRA) is crucial for performance, while FedProx offers no advantage over FedAvg in this setting.
By Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal, Debesh Jha, Sunil Kumar Gaire
The paper investigates why post‑training boosts reasoning more than perception in vision‑language models. Using a diagnostic framework with synthetic tasks, it finds a perception‑reasoning asymmetry: supervised fine‑tuning suffers from token imbalance, while reinforcement learning suffers from reward coupling. The authors propose reweighting losses and perception‑aware rewards, achieving up to 18.2‑point and 6.0‑point gains respectively, and show that these methods also improve real‑world visual reasoning by up to 3.3 points.
By Xueqing Wu, Yu-Chi Lin, Kai-Wei Chang, Nanyun Peng
The paper introduces BLUEPRINT, a safety‑evaluation framework that separates a factorized social‑influence strategy space from WORLDVIEWSIM, a cross‑turn situational context module. Using Monte Carlo Tree Search, it optimizes turn‑level combinations of 18 theory‑grounded influence factors across a four‑turn trajectory, achieving near‑ceiling ASR on six frontier models with an average of only 2.46 queries. The study reveals that model‑specific vulnerabilities arise from distinct influence factors and strategy transitions, yet all models share a recovery pathway that shifts toward concrete, executable task framing to escape hard‑refusal states, highlighting the importance of monitoring how dialogue state makes unsafe requests appear actionable.
By Siyu Chen, Haoran Wang, Xiaojian Li, Yao Huang, Yinpeng Dong, Wei Xu
DiDrive introduces a risk‑aware hierarchical diffusion framework for offline reinforcement learning in autonomous driving. It combines a low‑level risk‑gated encoder with a high‑level contextual modulator to filter redundant state information, and a 3DICE policy optimization that reduces out‑of‑distribution overestimation and stabilizes gradients. On the CARLA benchmark, DiDrive outperforms baselines such as IQL, CQL, and Diffusion‑QL, achieving an 85% success rate and a 4295.68 average reward in dense traffic with 60 vehicles.
By Qisong Guo, Jingtang Chen, Zhilin Chen, Pei Xu, Mingjian Fu, Wenxi Liu, Yuanlong Yu
MARS introduces a multi‑layer, multi‑slot embedding framework for text‑video retrieval that constructs adaptive representation slots by combining hidden states from different decoder layers. By comparing corresponding text and video slots and aggregating their similarities, MARS captures fine‑grained cues that single‑token embeddings miss. A hard‑negative‑aware slot specialization objective further encourages slots to focus on discriminative matching cues, leading to state‑of‑the‑art results on four benchmarks.
By Uicheol Jung, Juyoung Hong, Geuntaek Lim, Yukyung Choi
The paper introduces DynaTokens, a transformer-based token generator that produces fine‑tuning tokens on demand for continual VideoQA with multimodal large language models. It uses shared generation weights and meta‑learning‑inspired regularisers to reduce task interference and forgetting, connecting the objective to sharpness‑aware optimisation for flatter cross‑task minima. Experiments on standard continual VideoQA benchmarks show that DynaTokens achieves higher average accuracy, lower forgetting, better zero‑shot generalisation, and robust cross‑modal transfer in a new ImageQA→VideoQA protocol.
By Toan Nguyen, Yang Liu, Celso De Melo, Flora D. Salim
The paper introduces Cue2Narrate, a two‑stage pipeline that jointly predicts what visual events to narrate and when to insert the narration in long, untrimmed movie clips. It uses a dual‑head audio‑visual localizer to identify visual cue and narration windows, followed by a LoRA‑adapted vision‑language model that generates concise audio descriptions, trained with a Description Ranking Loss. The authors also present the LongLSMDC benchmark, comprising up to 8‑minute clips, and show that Cue2Narrate outperforms video‑only and audio‑only baselines by 5–12 points in average mAP and improves AD generation over fine‑tuned base VLMs.
By Akshita Gupta, Aditya Arora, Federico Tombari, Marcus Rohrbach, Anna Rohrbach
Video2Reaction is a multimodal dataset that links short movie segments to the emotional reactions of viewers, gathered from social media comments. The dataset models reactions as distributions over categorical emotions, capturing the subjective and ambiguous nature of emotional perception. Experiments show that vision‑language models fine‑tuned with LoRA learn effectively from Video2Reaction and outperform specialized baselines, and that models pre‑fine‑tuned on this dataset transfer well to other emotion prediction tasks.
By Sidong Zhang, Trang Nguyen, Shiv Shankar, Gauri Jagatap, Deepak Chandran, Andrea Fanelli, Madalina Fiterau
The paper introduces SEAL, a training-time, parameter‑efficient defense that attaches a plug‑and‑play adapter to the shared expert component of Mixture‑of‑Experts models, and SEAL++, which adds an orthogonal constraint to preserve existing safety subspaces. By leveraging the always‑activated shared expert, SEAL mitigates the structural vulnerability of sparse routing to adversarial manipulation, reducing attack success rates by up to 60% with minimal impact on model capability. The approach is evaluated across six attack scenarios involving harmful prompting, jailbreaks, malicious fine‑tuning, and neuron pruning.
By Qingyu Meng, Yiwei Zha, Jiahuan Pei, Koen Hindriks, Herbert Bos, Min Chen
The paper introduces a black-box membership inference attack framework tailored for fine-tuned text-to-speech models, addressing challenges in query generation and representation engineering. It evaluates five query types, finding recitation queries most effective, and uses multi-level speech embeddings with temporal alignment for fine-grained comparison. Experiments on CosyVoice2, F5-TTS, and XTTS-v2 trained on VCTK and British Dialect datasets show high privacy leakage, with speaker-level AUC above 0.80 and record-level AUC between 0.80 and 0.90.
By Kunlin Cai, Kaiyuan Zhang, Zihang Xiang, Jinghuai Zhang, Abeer Alwan, Fnu Suya, Yuan Tian
Bernini proposes a unified framework that separates semantic planning and pixel rendering for video generation and editing. An MLLM-based planner predicts target semantics in ViT embedding space, while a DiT-based renderer synthesizes pixels conditioned on this plan, text features, and source VAE features for editing. The approach introduces Segment-Aware 3D Rotary Positional Embedding and chain-of-thought reasoning, achieving state‑of‑the‑art performance on diverse video benchmarks.
By Bernini Team, Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan
OmniEdit-Bench introduces a comprehensive benchmark for instruction-based video editing (IVE), addressing limitations of existing datasets by covering spatial, temporal, audio, and reference-based editing tasks and distinguishing explicit from implicit instructions. The evaluation framework assesses editing quality across accuracy, preservation, realism, and consistency, using human judgments and vision-language models, and incorporates an accuracy-aware penalty to ensure instruction fidelity. Experiments reveal that current IVE models perform poorly, highlighting the need for improved methods.
By Chenxuan Miao, Yutong Feng, Yi Lu, Yunfeng Yan, Donglian Qi, Shiwei Zhang, Yu Liu, Xi Chen, Hengshuang Zhao
The paper presents CycleGRPO, a reinforcement learning framework that unifies region understanding and localization for multimodal large language models (MLLMs). By treating the MLLM as both actor and critic, the method generates region captions and immediately grounds them back into spatial coordinates, using a token‑level cycle‑consistency reward that obviates the need for textual ground truths. Experiments on SAMTok demonstrate that CycleGRPO can bootstrap region captioning, VQA, grounded dialogue, and referring segmentation simultaneously, achieving consistent performance gains without task‑specific fine‑tuning.
By Xin Zhang, Haochen Wang, Yikang Zhou, Zhuochen Wang, Xiangtai Li, Robby T. Tan
The paper investigates how vision‑language models (VLMs) express visual content in discourse‑appropriate forms, focusing on information structure (IS) in Hungarian where Topic and Focus occupy distinct syntactic positions. By comparing six VLMs to human participants in visually grounded question answering, the authors find that VLMs produce IS‑relevant constructions but over‑regularise, collapsing onto narrow response templates, whereas humans use variable strategies influenced by discourse status, grammatical role, and definiteness. The study argues that VLM evaluation should extend beyond content accuracy to consider how content is packaged for discourse.
By Marcell Fekete, Johannes Bjerva, Tam\'as K\'aldi
LeakageBench is a new benchmark comprising 500 document images with 11,954 GDPR‑aligned PII annotations, designed to evaluate document‑level redaction risk. It measures how well OCR pipelines, OCR‑dependent detectors, and OCR‑free vision‑language models can localize and remove sensitive information, using entity‑level F1, group‑wise leakage, and document‑level leakage metrics. The study shows that while advanced models improve localization, most pages still exhibit critical leakage, highlighting the need for higher‑recall, spatially grounded redaction methods.
By Vishnu Prasad Vijaya Kumar, Santhosh Venkatesh, Ivan P. Yamshchikov