The study examines parameter‑efficient continual learning with LoRA, comparing gradient‑based attribution masks to random masks for selecting adapter parameters. It finds that when both methods are restricted to the same parameter pool, uniform random masks consistently outperform gradient attribution in reducing loss‑based backward transfer across multiple seeds on the Qwen2.5‑3B model, especially on the medical task. This suggests that random masking may better balance stability and plasticity in this setting.
By Yu Sun, Junwei Zhou, Zuodong Xiang, Yike Zhang, Pengcheng Xu, Xinyu Guan, Ruoyun Ma, Hailu Xu
The paper introduces fixed‑point flow matching (FFM), a data‑free method for training and fine‑tuning flow policies in reinforcement learning. FFM formulates the actor update as a fixed‑point iteration using a flow‑matching regression objective, where the critic’s action gradients act as targets. Experiments show that FFM can effectively fine‑tune pretrained flow policies in offline‑to‑online scenarios and train flow policies from scratch in online RL, establishing it as a principled approach for learning expressive flow‑based policies without training data.
By Denis Blessing, Onur Celik, Gerhard Neumann
arXiv:2610. 12865v1 Announce Type: new Abstract: Existing convergence guarantees for non-log-concave sampling with stochastic gradient Langevin Monte Carlo (SG-LMC) typically assume globally Lipschitz gradients and uniformly bounded noise variance.
By M. Berk Sahin, Ege C. Kaya, Arda Fazla, Behzad Sharif, Abolfazl Hashemi
Monarch-MoE introduces structured experts for multimodal continual instruction tuning by replacing dense LoRA factors with block‑diagonal ones linked by a fixed permutation, enabling blockwise gradient isolation. This design reduces adapter parameters by up to 75% and 50% on two benchmarks while keeping first‑task forgetting below 0.52 and 0.10 percentage points, respectively. The method is optimized for memory‑limited hardware, allowing fine‑tuning and inference on a single 24 GB RTX 4090 with lower memory usage and competitive runtime.
By Xi Zhang, Boyu Diao, Zesen Chen, Ruiqi Liu, Hangda Liu, Yongjun Xu
The paper introduces the Constrained Triple-Head Price Optimization (C3PO) network, a transformer-based model designed for real‑time bi‑level decision making in discrete‑choice settings. C3PO integrates price imitation learning, multi‑task revenue prediction, and in‑context elasticity learning, while a lightweight differentiable constraint layer enforces business pricing rules. Experiments on simulated, synthetic, and real‑world datasets show that C3PO outperforms calibrated MNL baselines by up to 7.9 percentage points in expected revenue and reduces constraint violations by over 50%, with significant improvements in healthcare tender and airline ancillary pricing applications.
By Shivaram Subramanian, Zhengliang Xue, Markus Ettl, Yingdong Lu, Jayant Kalagnanam
MoPrO‑SD is a stance‑specific prompt optimization framework that breaks prompts into semantic modules and refines them sequentially with LLM feedback. It evaluates three refinement strategies—instruction‑only, instruction with reasoning, and instruction with per‑example reasoning—using class‑conditioned errors to improve stance detection. The authors also release VaccineStance, a new human‑annotated benchmark for vaccine stance detection, and demonstrate that MoPrO‑SD outperforms human‑written base prompts across multiple datasets and open‑weight LLMs.
By Meysam Sabbaghan, Nikesh Gyawali, Arman Zareian Jahromi, Sebastian Loera-Lozano, Doina Caragea, Cornelia Caragea
The paper introduces user simulators that provide intrinsic feedback by incorporating latent evaluation states such as internal thoughts, reactions, and scalar ratings. These simulators, trained on real user–assistant interaction logs, outperform prompted baselines in realism and critiquing behaviors, and their turn‑level evaluations better align with human judgments. Additionally, the intrinsic feedback serves as an effective reward signal for multi‑turn reinforcement learning, reducing compute requirements by over 50% compared to setups using external judge LLMs.
By Tarek Naous, Craig Boutilier, Wei Xu, Ofer Meshi
The paper studies how large language models (LLMs) handle the discourse particles "just" and "only" in natural conversation. Behavioral experiments show that both humans and LLMs prefer these particles and are more sensitive to "just" than "only", while instruction tuning enhances this sensitivity. Mechanistic analysis identifies specific units in the models that respond to each particle, revealing an asymmetric overlap in their internal representations.
By William Sheffield, Jiahui Chen, Amy Zhang, Alex Warstadt, Junyi Jessy Li, Kanishka Misra
The paper reports fine‑tuning the Segment Anything Model (SAM) 2.1 on the HBD dataset of 2,679 images from 42 buildings to improve multi‑class segmentation of 2D building images. The fine‑tuned model achieves a higher mean Intersection‑over‑Union (mIoU) of 69.6% compared to 49.9% for the pretrained version, and shows better performance on larger object classes. These enhanced 2D masks are then used in Group Anything with Radiance Fields (GARField) to group building components more accurately in 3D neural radiance fields, facilitating more precise 3D auto‑reconstruction of building structures.
By Ehsan Agha Ebrahimi, Yong-Cheol Lee, Keith G. Mills
MotionRFT introduces a reinforcement fine‑tuning framework for text‑to‑motion generation that unifies heterogeneous motion representations into a shared semantic space using the MotionReward model. The framework includes a fine‑grained, memory‑efficient fine‑tuning method called EasyTune, which optimizes step‑wise denoising steps to reduce computational overhead. Experiments show significant improvements in FID, R‑Precision, and memory usage across several motion models.
By Xiaofeng Tan, Wanjiang Weng, Hongsong Wang, Fang Zhao, Xin Geng, Liang Wang
The paper introduces CIG‑SFT, a supervised fine‑tuning method that uses conditional information gain to evaluate how much each demonstrated token reduces uncertainty about the task. By comparing token likelihoods from a frozen reference model with and without task input, the method derives a KL‑regularized projection of the current policy, yielding a scalar‑weighted SFT objective whose gradients match the projected target. Experiments on mathematical reasoning and code‑generation benchmarks show that CIG‑SFT outperforms standard SFT and adaptive fine‑tuning baselines, especially in generalizing to challenging evaluation distributions, without requiring extra data or online rollouts.
By Yuxuan Gu, Kequan Niu, Xiaocheng Feng, Yun Li, Yiqiao Yang, Bing Qin
The paper introduces Synthetic Self-Distillation (SynthSelf), a method that enables continual post‑training for small language models by outsourcing in‑distribution learning to a slightly larger aligner. SynthSelf combines off‑policy supervision and rejection sampling to provide high‑quality supervision while preserving the model’s original distribution. Experiments on four acquisition datasets and fourteen retention benchmarks show SynthSelf consistently outperforms six baseline methods across three small‑model scales, demonstrating its effectiveness for low‑latency, on‑device continual learning.
By Woosung Koh, Hyunsoo Lee, Kyungjae Lee, Haeju Park, Dahyun Lee, Dasol Hwang, Moontae Lee
The study examines how 105 language models choose a single word from a large set of valid options, finding that 46% of the time they all pick the same word, "serendipity." Using 96 prompts that ask for one-word answers across various categories, the researchers measure convergence and each model’s share of the consensus. They observe that post‑training and persona‑tuned models diverge more, while heavily post‑trained assistants are more conformist, and that the field’s answer choices are more concentrated than human norms in most categories.
By Tapan Parikh
The paper introduces Personalized Federated Direct Preference Optimization (pFedDPO), which extends FedDPO by allowing each client to keep a small private LoRA adapter on top of a shared global base model. In pFedDPO, the base model is updated globally via FedAvg while the LoRA adapters remain local, capturing client‑specific preference patterns. The authors prove that this approach reduces the effective gradient diversity compared to vanilla FedDPO and demonstrate experimentally that pFedDPO achieves lower gradient norms while maintaining competitive training loss and per‑client reward accuracy.
By Zhanhong Jiang, Soumik Sarkar
arXiv:2610. 13205v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target.
By Rong Wu, Daocheng Fu, Yu Yang, Xuemeng Yang, Siqi Li, Shu Zou, Yao Zhu, Botian Shi, Bo Zhang, Cong Zhang, Pinlong Cai, Jianbiao Mei
The paper introduces Outlier-Aware Attention Linearization (OAL), a method that improves linearized softmax attention by explicitly accounting for a few outlier dimensions that disproportionately affect query-key dot products. OAL builds on a second-order Taylor approximation, uses a Hadamard-product formulation to expose dimension-specific contributions, and groups dimensions with similar scales to share coefficients while treating outliers separately. Experiments on Qwen2.5-0.5B show an 8.9% reduction in frozen-attention output MSE, and fine-tuning with LoRA lowers WikiText-103 perplexity from 31.71 to 27.50 while boosting performance on PIQA, HellaSwag, and ARC-E.
By Weiqi Sun, Qianyue Wu, Wenjia Wang, Tianyang Hu
The paper introduces FastCDTW, an efficient approximation for Continuous Dynamic Time Warping (CDTW). It offers two computational strategies: an exact closed‑form solution for piecewise‑linear warping functions and an unbiased Monte Carlo estimator for smooth warping functions such as those implemented by implicit neural networks. Both approaches provide gradients with respect to the warping function and the input time series, making the algorithm GPU‑friendly and suitable for integration into complex learning tasks like time‑series averaging, classification, and deep domain adaptation.
By Pierre Houedry, Romain Tavenard, Nicolas Courty, Romaric Gaudel
Dream to Remember (D2R) is a continual learning framework that steers a frozen generalist robot policy toward new tasks while preserving previously acquired skills. It uses a pretrained world model to rehearse past tasks through imagined rollouts, eliminating the need for replay buffers or new data. Experiments on CALVIN, LIBERO, and a real Franka robot show that D2R can learn multiple tasks sequentially—often entirely in imagination—without catastrophic forgetting, outperforming traditional fine‑tuning methods.
By Bahey Tharwat, Iman Nematollahi, Marvin Alles, Xingyuan Zhang, Akshay L Chandra, Abhinav Valada
The study demonstrates that large language models can be compressed by clustering weight matrices into only 16–64 shared values without retraining, preserving strong accuracy. Fine‑tuning the cluster centroids recovers most of the remaining accuracy loss. Analysis reveals that maintaining the relative rank of weights and their overall variance is crucial for performance, while early layers and spatial geometry of cluster assignments also play significant roles.
By Borja Aizpurua, Sukhbinder Singh, Rom\'an Or\'us
The paper presents GRASP, a method for unsupervised detection and removal of spurious correlations that arise during fine‑tuning of pretrained language models. By identifying latent factors from LoRA fine‑tune weights and projecting gradients to prevent new reliance on them, GRASP preserves genuine task signals while mitigating bias. Experiments on three tasks—misaligned code generation, medical advice, and political bias—show that GRASP reduces misalignment and drift more effectively than existing baselines while maintaining task performance.
By Ciar\'an M. Gilligan-Lee, Joseph Egan, Yuchen Zhu, Michael O'Riordan