The paper introduces Self‑Evolutionary CLIP (SE‑CLIP), a semi‑supervised framework that adapts vision‑language models like CLIP to satellite imagery. SE‑CLIP uses a two‑phase pipeline: an initial warm‑up on a small set of annotated seeds followed by a recursive discovery phase that iteratively selects high‑confidence samples from unlabeled data. A class‑balanced selection strategy is applied to keep the evolving support set balanced, and experiments on the UCM and NWPU benchmarks show that SE‑CLIP outperforms existing semi‑supervised methods.
By Mohamed L. Mekhalfi, Mohamad M. Al Rahhal, Yakoub Bazi, Salah E. Khenfer, Mingdeng Shi, Hua Zou, Mansour Zuair
PANDA (Prototype‑Anchored Data Alignment) is a two‑stage framework that enables a primary‑modality model to benefit from auxiliary modalities even when those modalities are only partially paired or absent at inference. In Stage 1, a shared embedding is learned from the paired subset and class prototypes are estimated from the auxiliary data; in Stage 2, the primary encoder is trained on all subjects using cross‑entropy and alignment to the frozen prototypes. PANDA was evaluated on Alzheimer’s MRI and TCGA‑Lung pathology, achieving significant AUC gains and improved survival prediction while requiring no auxiliary inputs during deployment.
By Sheethal Bhat, Mahfuzur Rahman Chowdhury, Paula Andrea Perez-Toro, Stephan Wunderlich, Rose Dawn Bharat, Siming Bayer, Andreas Maier
The paper introduces Segmentation-Based Attention Entropy (SAE), a method that uses semantic segmentation to measure visual attention uncertainty in large vision‑language models (LVLMs). SAE provides a reliability score for detecting hallucinated objects and an attention‑adjustment technique that reduces hallucinations during inference. Experiments on public benchmarks and real quadruped robot scenarios demonstrate that SAE improves LVLM reliability without requiring additional training.
By Jiale Song, Jiaxin Luo, Xue-song Tang, Kuangrong Hao, Mingbo Zhao
The paper introduces Interaction‑Aligned Pruning (IAprune), a training‑free method for visual token pruning in embodied manipulation tasks. IAprune jointly decides per‑frame budget and token selection, using semantic‑motion spatial agreement to choose between conservative and aggressive coverage, and applies geometric residual correction to focus on under‑represented boundaries. Experiments on four policies, three simulation benchmarks, and a real‑robot platform show that IAprune matches unpruned performance on LIBERO while achieving up to 1.54× speed‑up and 1.48× acceleration on a real robot.
By Jintao Cheng, Weibin Li, Haozhe Wang, Gang Wang, Yipu Zhang, Xiaoyu Tang, Jin Wu, Xieyuanli Chen, Yunhui Liu, Wei Zhang
The paper introduces a hierarchical multimodal mixture-of-experts (MoE) model for interstitial lung disease (ILD) classification. It combines a frozen, pre‑trained imaging expert with structured electronic health records (EHR) through a two‑stage gating system: a modality‑level gate weights imaging and EHR predictions, while a sub‑gating module further decomposes the EHR branch into clinically defined feature groups with learned, group‑specific contributions. The approach preserves stable imaging representations, allows input‑dependent clinical weighting, and enhances interpretability across anatomical regions, imaging–EHR utilization, and EHR feature groups, achieving the highest mean AUC (0.8750 ± 0.0443) under strict patient‑level cross‑validation.
By Alec K. Peltekian, Gorkem Durak, Halil Ertugrul Aktas, Carrie Lynn Richardson, Mary Carns, Kathleen Aren, GR Scott Budinger, Anthony J. Esposito, Alexander Misharin, Alok Nidhi Choudhary, Ankit Agrawal, Ulas Bagci
The paper introduces the Hybrid Hierarchical Multi-Agent Framework (H$^{2}$MAF), which fuses decision-level outputs from EfficientNet-B3 and ConvNeXt-Tiny with semantic arbitration by multimodal large language models Gemma 4 E4B and Qwen3.5 4B to produce explainable plant disease diagnoses. Evaluated on 14,364 images from PlantDoc and two Cornell robotic field datasets, the framework achieves up to 99.3% accuracy, with Gemma improving PlantDoc accuracy from 63.9% to 68.5% and demonstrating low critical‑risk error. The results highlight the potential of MLLM arbitration for reliable, explainable agricultural AI under real‑world field conditions.
By Ranjan Sapkota, Konstantinos I. Roumeliotis, Pengyao Xie, Nikolaos D. Tselikas, Lirong Xiang, Manoj Karkee
The study fine‑tunes the Whisper Small model for Automatic Speech Recognition (ASR) in Baniwa, an indigenous Arawakan language. Using a 0.54‑hour corpus of 1,373 manually transcribed recordings, the fine‑tuned model achieved a Word Error Rate of 37.5% and a Character Error Rate of 7.45%. These results provide an initial baseline for Baniwa ASR and suggest that multilingual foundation models can be adapted to extremely low‑resource languages.
By Leonardo Duart, Tiago Fonseca, Thiago Chac\'on
The paper introduces a three‑layer security framework designed to protect retrieval‑augmented generation (RAG) chatbots from both direct and indirect prompt injection attacks. Layer 1 filters user input with rule‑based patterns and a semantic anomaly classifier; Layer 2 enforces a provenance‑based instruction hierarchy during context assembly; Layer 3 audits model output with a policy rule engine and semantic drift detector. Evaluations on GPT‑4o, Llama 3, and Mistral 7B demonstrate a reduction in attack success rate from 71.4 % to 11.3 %, outperforming existing single‑layer defenses while keeping false positives low and latency acceptable.
By Gulshan Saleem, Nisar Ahmed, Muhammad Imran Zaman, Ali Hassan, Umar Mujahid
The paper exposes a hidden vulnerability in Vision Mixture-of-Experts (MoE) models that use capacity-bounded token dispatch, which varies with batch size. It presents a three-phase backdoor attack: injecting a backdoor into an early MoE layer, training a neutralizer in a deeper layer to suppress it under normal capacity, and then adjusting the batch-adaptive capacity factor so that the neutralizer is disabled when large batches are used at deployment. Experiments on V-MoE and Swin-MoE show high attack success rates (76‑87%) for large batches while keeping the attack dormant and undetected during small-batch audits, evading several state‑of‑the‑art defenses.
By Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu, Ruyi Ding
DEFUSE is a backdoor detection framework for self‑supervised encoders that uses a conditional diffusion generative model to estimate representation‑conditioned image likelihoods. By fine‑tuning a pretrained diffusion model, DEFUSE performs semantic reconstruction in a reference encoder’s representation space, enabling it to detect backdoors without needing uninfected data or precomputed pseudo‑labels. Experiments show that DEFUSE outperforms existing detectors on both visual SSL and vision‑language encoders, reducing reliance on prior knowledge of the victim model or attack strategy.
By Tuo Chen, Jie Gui, Minjing Dong, Lanting Fang, Ju Jia, Benlei Cui, Jian Liu
VISA (Visual Instruction Synthesis Agent) is an agentic framework that transforms multimodal instruction synthesis into a self‑evolving loop. Each cycle analyzes images to filter constraints, samples new constraint sets, generates candidate instructions, and verifies them using executable tools and large language model judges. Failed samples trigger diagnostic recovery, while accepted samples are evaluated against the target model to estimate difficulty, with all feedback written back to memory to adapt future rounds and provide reward signals for reinforcement learning.
By Min Zeng, Guanxin Tan, Libin Cen, Yawei Wen, Rui Hu, Liuyang Bian, Xiaolong Chen, Xiaoxin Chen
arXiv:2607.18227v2 Announce Type: replace
Abstract: In line with the prevailing direction of vision research, we explore the integration of both generation and editing capabilities for video and imag...
By Dingyun Zhang, Lixue Gong, Wei Liu
The paper introduces FRAME, a two‑step framework for auditing fairness claims in medical imaging. First, it derives a fair‑model reference distribution that captures the portion of subgroup performance differences attributable to sampling variation. Second, it tests the remaining difference using operators in representation space to assess whether demographic information or disease entanglement drives the bias. Across a large dataset of 702,206 images and 36 encoders, the reference explains a substantial median share of race and age differences, while interventions such as injecting demographic decodability or entangling disease direction have limited impact on the residual bias.
By Mahshad Lotfinia, Daniel Truhn, Andreas Maier, Soroosh Tayebi Arasteh
Latent-CoT-Drive (LCDrive) is a vision‑language‑action model for autonomous driving that replaces natural‑language chain‑of‑thought reasoning with a latent language capturing possible outcomes of driving actions. The model interleaves action‑proposal tokens, aligned with the model’s output actions, and world‑model tokens grounded in a learned latent world model to reason about future outcomes. After a supervised cold‑start using ground‑truth future rollouts, LCDrive is further refined with closed‑loop reinforcement learning, achieving faster inference, higher‑quality trajectories, and greater gains from interactive RL than both non‑reasoning and text‑reasoning baselines on a large‑scale end‑to‑end driving benchmark.
By Shuhan Tan, Kashyap Chitta, Yuxiao Chen, Ran Tian, Yurong You, Yan Wang, Wenjie Luo, Yulong Cao, Philipp Krahenbuhl, Marco Pavone, Boris Ivanovic
TacForcing is a streaming action‑generation framework that incorporates execution‑time tactile feedback into contact‑rich manipulation. It replaces a separate reactive controller with a streaming action expert that conditions actions on evolving tactile observations, and introduces Execution‑Aware Tactile Attention (EATA) to focus tactile conditioning on actions near execution. The method achieves 65% success in six simulated UniVTAC tasks and 69% in three real‑world contact‑rich manipulation tasks, outperforming strong baselines.
By Jianbo Zhou, Boyuan Zhao, Yuzheng Zhang, Yiyang Chen, Wenxin Chen, Qiuyue Li, Xiangyang Gu, Yuhan Cao, Xiao Xia, Yanzhe Hu, Zhijie Deng
The paper introduces CAT‑GS, a training controller that stabilizes multimodal neural networks by addressing three failure modes: modality imbalance, unstable gating, and fusion interference. CAT‑GS calibrates teacher-derived reliability, applies a margin‑thresholded gating policy, caps gradient budgets, and uses fusion‑only PCGrad, all without altering model architectures or losses. Experiments on audio‑visual, tri‑modal, synthetic, and cross‑domain benchmarks show that CAT‑GS matches or surpasses strong imbalance‑aware baselines while producing smoother gating and fewer conflicting fusion gradients.
By Mahir Shahriar Tamim, Sharjil Khan, Md. Samiul Alim, Tanvir Ahmed Khan, Shafin Rahman, Nabeel Mohammed
The paper introduces PART, a multimodal predictive framework for tennis that combines physiological, training, sleep, questionnaire, jump, and video data from nine collegiate players to assess overall wellness, injury risk, physical capability, and playing style. Using machine learning and deep learning, PART provides holistic athlete assessments and forecasts specific injury risks to body areas such as elbows and knees. Evaluation shows strong performance in predicting wellness and injury risk, with potential benefits for recreational players who often injure themselves due to poor technique.
By Francisco Erramuspe Alvarez, Shobharani Polasa, Weihao Qu, Jay Wang, Ling Zheng
TAU-Agent is a retrieval‑augmented framework designed for traffic anomaly understanding in transportation videos. It uses a central retrieval agent to coordinate a Video Captioning Tool and an Open‑Vocabulary Tracking Tool, gathering captions, temporal intervals, and object trajectories relevant to a query. The collected evidence, along with sampled frames and the query, is fed into a fine‑tuned vision‑language model that reasons and generates an answer. TAU-Agent was evaluated on the AI City Challenge 2026 benchmarks, achieving notable scores across multiple tracks and ranking second, twelfth, and fifth respectively.
By Yuqiang Lin, Yan Shi, Sam Lockyer, Harish Tayyar Madabushi, Adrian Evans, Wenbin Li, Yinhai Wang, Nic Zhang
arXiv:2608.03176v2 Announce Type: replace
Abstract: Imagined handwriting offers a temporally rich paradigm for non-invasive neural decoding, yet reliable recognition across unseen participants remain...
By Xiao Fan, Hongbin Guo, Yubo Han, Yi Zhang
Video-IFBench is a new benchmark designed to evaluate how well multimodal large language models (MLLMs) follow user-specified instructions in video understanding tasks. It introduces an instruction taxonomy with four templates—single-task, multi-task, selection, and nested—covering 32 task types and 39 constraint categories that span semantic and format requirements. The benchmark was built using a semi-automatic pipeline that combines MLLMs, programmatic processing, and human verification, producing 1.5K samples, and a large-scale evaluation of over 20 recent MLLMs shows that instruction following remains difficult, especially for complex constraints and conditional structures.
By Hongbo Liu, Peixian Chen, Sihan Liu, Peiyuan Zhang, Kai Zou, Dian Zheng, Xiaoxing Hu, Yuhao Dong, Mengdan Zhang, Yunhang Shen, Haoyu Cao, Wei Liu, Weibo Gu, Xing Sun, Shengjie Zhao