The paper investigates how the choice of Transformer layers for fine‑tuning affects both task performance and the preservation of pre‑trained capabilities. By measuring layer sensitivity with a forward‑only input–output cosine similarity metric, the authors identify layers that are more suitable for adaptation. They introduce Layer‑Selective LoRA (LS‑LoRA), which trains adapters only in low‑similarity layers, and demonstrate that this approach improves target‑task results while better retaining commonsense reasoning compared to standard all‑layer LoRA.
By Zhiqiang Pang, Zihong Sun, Qi Xie, Jun Shu, Deyu Meng, Zongben Xu
The paper introduces Internalizer, a portable Context-to-Parameter Mapping hypernetwork that generates document‑specific LoRA adapters for the 284B‑parameter DeepSeek v4 Flash model. Unlike previous work limited to models up to 14B parameters, Internalizer’s architecture places most parameters in a model‑agnostic trunk, allowing cheap training on small models before porting to the large one. On unseen 4096‑token documents, the generated adapters achieve 84.9% top‑1 and 97.8% top‑5 teacher‑forced accuracy, far surpassing the base model’s 63.4% and 83.5% respectively, using only a three‑word instruction in the context window.
By Peter Devine, Nick Ryan, Benjamin Sirb, Alex Chiocchi
The paper reports that fine‑tuned small language models (SLMs) can outperform large frontier models in annotating grammar concepts from learner–tutor transcripts. By training a 0.8B Qwen3.5 model on teacher‑generated data and embedding the annotation rules into adapter weights, the authors deploy an efficient end‑to‑end grammar mastery tracker that matches or exceeds the performance of 4B reference models and GPT‑5.4/5.6 on two human‑curated benchmarks. The lightweight model also cuts serving costs by about 16× and boosts learner engagement and key business metrics in an online experiment.
By Marjan Celikik, Ana Peleteiro Ramallo, Javier Morales
VAMR is a video agent that handles multiple questions about a single long‑form video using a single shared tool‑use trajectory, rather than treating each question separately. It employs question‑conditioned visual perception, layered multi‑question memory, and a question‑horizon policy optimization to jointly acquire evidence and progressively build a shared understanding. In benchmark tests (LVBench, Video‑Holmes, LongVideoBench), VAMR outperforms prior methods, achieving higher accuracy and fewer reasoning rounds and processed frames.
By Runquan Gui, Hanzhu Chen, Zehao Wang, Hanxin Zhu, Xin Li, Zhibo Chen
VibeEdit is a new image‑editing tool that lets users draw spatial marks and add brief notes directly on an image to create a canvas instruction. These instructions guide the model to perform tasks such as adding, removing, replacing, or moving objects, as well as modifying attributes, without needing a separate text prompt. The system is trained on 1.55 million annotated edit pairs and uses a layer‑decoupled conditioning approach, achieving higher VLM rubric scores and PSNR on a benchmark that tests target selection among similar objects.
By Jinjing Zhao, Fangyun Wei, Yitong Wang, Xiuyu Wu, Yunuo Chen, Yang Yue, Sirui Zhang, Wenbo Wang, Hongyang Zhang, Dong Chen, Yan Lu, Chang Xu
The paper introduces ARC, a reasoning recipe that enhances zero‑shot performance of robot foundation models (RFMs) without additional data or large‑scale training. ARC combines a reasoning trace grounded in the robot’s next action, an automatic labeling pipeline to generate these traces from existing demonstrations, and a strategy to adapt pretrained RFMs to use the traces for control. Applying ARC yields significant gains, including up to 50 percentage points on RoboLab-Reasoning-50 and an 82.2‑point improvement in task success for the π_{0.5} model on real robots.
By Gokul Puthumanaillam, Tao Sun, Elie Aljalbout, Moritz Reuss, Zhaoshuo Li, Fabio Ramos, Ankit Goyal, Jenai Xuning Yang
The paper introduces the task of predicting alignment generalization, which measures how fine‑tuning a language model to follow a specific value affects its behavior across a broad set of other values. Through a large‑scale study of 66 values, the authors show that representations derived from model activations when applying values in context outperform textual description‑based methods, achieving a correlation of 0.45 versus 0.05. They further demonstrate that these representations can predict downstream robustness and help construct a taxonomy of LLM values based on empirical generalization dynamics.
By Andy Liu, Mehar Bhatia, Karolina Stanczak, Mona Diab, Vered Shwartz, Daniel Fried
The paper introduces DriveMind, a large-scale driving Visual Question Answering corpus that aligns chain-of-thought reasoning with trajectory planning. Experiments with Vision‑Language Model agents show a causal disconnect: removing navigation priors significantly harms planning, while removing the chain-of-thought has little effect. The authors propose the Reasoning‑Planning Decoupling Hypothesis and present a training‑free probe to assess an agent’s reliance on priors.
By Xurui Song, Shuo Huai, JingJing Jiang, Jiayi Kong, Jun Luo
The paper "ActionCodec: What Makes for Good Action Tokenizers" addresses the lack of design guidance for action tokenization in Vision‑Language‑Action (VLA) models. By deriving information‑theoretic principles—maximizing temporal token overlap, reducing vocabulary redundancy, boosting multimodal mutual information, and ensuring token independence—the authors propose ActionCodec, a tokenizer that improves training efficiency and VLA performance. Experiments show that a SmolVLM2‑2.2B fine‑tuned with ActionCodec reaches a 95.5 % success rate on LIBERO, rising to 97.4 % with architectural tweaks, setting a new state‑of‑the‑art for VLA models without robotics pre‑training.
By Zibin Dong, Yicheng Liu, Shiduo Zhang, Baijun Ye, Yifu Yuan, Fei Ni, Jingjing Gong, Xipeng Qiu, Hang Zhao, Yinchuan Li, Jianye Hao
The paper demonstrates that routing information logged by Mixture-of-Experts (MoE) language models can be used to infer whether a specific example was part of a model’s fine‑tuning data. By combining router telemetry with standard output signals and training a membership classifier on shadow models, the authors achieve higher true‑positive rates across multiple MoE architectures and data domains. The study shows that this privacy leakage persists under various fine‑tuning regimes and even with limited telemetry, indicating that router logs can become an additional privacy risk for deployed MoE models.
By Yixin Tan, Jiayang Liu, Lu Sun, Yuke Hu, Zheng Li, Rui Wen
The paper introduces a leakage‑controlled multimodal learning framework for diagnosing and predicting the progression of Alzheimer’s disease. It combines an adapted SFCN MRI encoder, four causal clinical Transformers, shared fusion, and task‑specific ODE‑GRU dynamics, with fine‑tuning and LoRA for MRI blocks. Across 2,649 subjects and 17,317 visits, the model achieves high AUROCs for diagnosis and progression, and demonstrates strong external validation on OASIS‑3 data.
By Akeem Temitope Otapo, Ghazaleh Khodabandelou, Zuheng Ming, Alice Othmani
SteerCast is a retrieval‑based latent steering technique that enhances decoder‑only time‑series forecasters during inference without updating model parameters. It builds a database from the training set, storing each history window’s representation and a steering vector that captures the difference between the model’s prediction and the ground‑truth continuation in latent space. During testing, SteerCast retrieves nearest neighbors, aggregates their steering vectors, and injects the resulting signal into the forecaster’s hidden states at each autoregressive step, guiding predictions toward trajectories seen in similar training cases. Experiments on diverse multivariate benchmarks and multiple horizons demonstrate consistent accuracy improvements over fine‑tuned backbones and other retrieval‑based baselines, all while requiring no additional training beyond the original fine‑tuning and using only the training set as a retrieval corpus.
By Van Dai Do, Huu Hiep Nguyen, Minh Hoang Nguyen, Hung Le
SpatialOPSD is a self‑distillation framework that teaches a multimodal large language model (MLLM) to perform spatial reasoning without relying on external tools. By feeding the MLLM with verified execution traces from a spatial coding agent, the model learns an internal spatial chain‑of‑thought. The method includes Repetition‑Aware Distillation to prevent privileged‑information leakage, and experiments show it outperforms SFT and GRPO on spatial and out‑of‑distribution benchmarks.
By Rongxue Li, Meng Yang, Yiru Mao, Yongliang Tao, Lulu Hu, Bin Yang, Zhao Xu, Weihua Luo, Bowen Xu
The paper investigates how synthetic tasks influence language‑model pretraining by distinguishing diagnostic, teachable, and transferable signals. In a controlled setting where 70% of the data is general Python and 30% is a mixture of curated synthetic tasks, literature‑derived probes, and OpenCodeInstruct, 14 of 27 tasks show teachability, with curated tasks outperforming literature probes. Curated synthetic data provides conditional value: it boosts downstream performance when mixed with OpenCodeInstruct but is less effective when used alone, and a loss‑based scheduler can misalign short‑term loss reduction with long‑term transfer.
By Ohad Rubin
The paper investigates how test‑time compute techniques—adaptation, aggregation, and context construction—can improve predictions of pretrained tabular foundation models (TFMs). It introduces DiagScale, a lightweight adaptation method that trains a minuscule fraction of parameters yet matches full fine‑tuning gains, and shows that selective aggregation of predictions from diverse preprocessing variants can reduce error, while broader context expansion offers inconsistent benefits. The study concludes that adaptation and selective aggregation consistently yield benchmark‑level improvements, but their combined use demands more computation, guiding strategy selection based on available resources.
By Kanghui Ning, Marin Bilo\v{s}, James T. Wilson, Yilang Zhang, Kashif Rasul, Dongjin Song, Anderson Schneider, Yuriy Nevmyvaka
The paper proposes two new 4‑bit quantization schemes for AdamW’s optimizer states, focusing on the rounding space of the second moment. Zero‑Inclusive Preconditioner‑space Stochastic Rounding (ZIP‑SR) keeps zero in the codebook and applies stochastic rounding in preconditioner space, while Zero‑Excluding EDEN (ZE‑EDEN) removes zero from the codebook and rescales the quantized block to reduce distortion. Experiments on GPT‑ and Llama‑style models from 130 M to 2.7 B parameters show that both methods close the validation‑loss gap to 32‑bit AdamW, achieving up to a 70 % reduction in the gap and outperforming TorchAO in fine‑tuning tasks.
By Hanyang Li, Shao Tang, Daniel Thomas Braithwaite, Gregory Dexter, Leonardo Neves, Aman Gupta, Hiroto Udagawa, Abhishek Shivanna, Daniel Silva, Rohan Ramanath
The paper investigates Observation‑Space Shift (OSS), a failure mode in long‑horizon robotic manipulation where downstream skills start from states altered by preceding skills. By diagnosing that displaced scene elements—not robot joints or target objects—cause most OSS, the authors develop a detect‑restore‑resume system that monitors task progress, restores the scene, and fine‑tunes the skill to resume. On the BOSS‑44 benchmark, this approach raises full‑chain success from 7.6 % to 26.5 %, outperforming several baselines and demonstrating the value of scene restoration before policy resumption.
By Pranav Wagh, Yu Fang, Yue Yang, Mingyu Ding
The paper presents a preference-driven unlearning framework for few-step distilled text-to-image diffusion models. It identifies that standard Direct Preference Optimization (DPO) fails on these models due to altered generation dynamics and proposes a modified formulation aligned with few-step properties. The method enables direct concept removal while preserving desirable capabilities, and is evaluated on identity, NSFW, and object-level unlearning tasks with strong forgetting and retention results.
By Gaurav Patel, Jun Fang, Greg Ver Steeg, Qiang Qiu, Sravan Sripada
The paper introduces a novel four‑stream complex‑valued architecture for Open‑Vocabulary Audio‑Visual Event Localization (OV‑AVEL). By embedding visual, audio, and textual modalities into a shared space and computing complex‑valued similarities—real parts from standard embeddings and imaginary parts from companion streams (iHSV for vision and CycleGAN‑translated phase for audio)—the method learns fusion via a complex‑valued neural network while keeping encoders frozen. This approach achieves new state‑of‑the‑art results on OV‑AVEBench and a modified AVE dataset, outperforming previous fine‑tuned baselines by significant margins.
"whyItMatters":"The proposed complex‑valued fusion strategy demonstrates that incorporating phase information and learning fusion in the complex domain can substantially improve open‑vocabulary event localization performance, setting new benchmarks for the task."
By Anirudh Praveen, Koteswar Rao Jerripothula, Pratik Joshi, Aveen Dayal, Neela Sawant
The paper corrects earlier claims about how system prompts affect hidden‑state geometry in open‑weight language models. After auditing the code, the authors withdraw regime‑specific and direction‑to‑magnitude claims, but find that centroid distance, maximum mean discrepancy, and a linear probe still distinguish prompt conditions across all models. Curvature statistics only exceed noise in a minority of comparisons, and in Gemma‑4‑E4B‑it the identity prompt yields a lower‑norm state than a token‑matched generic prompt, aligning with observed pause behavior in outputs.
By Jorge Castillo Sep\'ulveda, Marco Torres Y\'evenes, Juan Carlos Lanas