The paper investigates how the choice of Transformer layers for fine‑tuning affects both task performance and the preservation of pre‑trained capabilities. By measuring layer sensitivity with a forward‑only input–output cosine similarity metric, the authors identify layers that are more suitable for adaptation. They introduce Layer‑Selective LoRA (LS‑LoRA), which trains adapters only in low‑similarity layers, and demonstrate that this approach improves target‑task results while better retaining commonsense reasoning compared to standard all‑layer LoRA.
By Zhiqiang Pang, Zihong Sun, Qi Xie, Jun Shu, Deyu Meng, Zongben Xu
arXiv:2606. 24841v1 Announce Type: new Abstract: Prompt-based learning has emerged as a dominant paradigm in natural language processing.
By Ahmad Pouramini, Hesham Faili
arXiv:2609.35868v1 Announce Type: new
Abstract: Is human readability necessary for effective fine-tuning of large language models? We investigate whether model-conditioned training representations ca...
By Jinhao Zhang, Zeyu Liu, Zicheng Yan, Yunquan Zhang, Daning Cheng, Song Tang
arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.
By Heming Zou, Yixiu Mao, Yun Qu, Qi Wang, Xiangyang Ji
arXiv:2607. 04733v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) is the standard approach for adapting pretrained language models to downstream domains, yet it often improves target-domain behavior at the cost of degrading pre-existing capabilities.
By Yueyang Wang, Baolong Bi, Shuo Lu, Jingyuan Zhang
arXiv:2606.22272v2 Announce Type: replace
Abstract: Applying pre-trained language models to new domains through full fine-tuning is computationally expensive and prone to catastrophic forgetting. To...
By Mohammed Rawhani, Dervi\c{s} Karabo\u{g}a, \"Ozkan Ufuk Nalbanto\u{g}lu, Alper Ba\c{s}t\"urk, Bahriye Akay
arXiv:2607. 03453v1 Announce Type: cross Abstract: Inference-time alignment methods, such as Best-of-$N$, offer a flexible alternative to training-based alignment by using reward models to select high-quality responses generated by a reference LLM.
By Eric Lei, Hsiang Hsu, Chun-Fu Chen
arXiv:2607. 11505v1 Announce Type: cross Abstract: Post-training is essential for refining the domain-specific capabilities of large language models (LLMs), yet existing reward optimization and distribution matching methods tightly couple policy exploration with distribution alignment.
By Daocheng Fu, Rong Wu, Yu Yang, Xuemeng Yang, Jianbiao Mei, Licheng Wen, Pinlong Cai, Yong Liu, Botian Shi, Yu Qiao
The paper introduces HiDeR, a High Information Density Replay framework for Lifelong Person Re-Identification that replaces discrete sample selection with information compression. It uses a complexity‑aware memory allocation based on intra‑class variance and a metric‑guided condensation objective to preserve essential identity topologies, while a cross‑modality adaptation strategy bridges synthetic and real styles to improve training. Experiments show HiDeR outperforms state‑of‑the‑art methods in knowledge retention and generalization, and reduces cumulative replay cost.
By Mingyu Wang, Wei Jiang, Haojie Liu, Zhiyong Li, Weijie Mao
arXiv:2610.00873v1 Announce Type: cross
Abstract: In many industrial applications, 1) tabular data is scarce and imbalanced and thus requires synthetic expansion; 2) input distributions drift between...
By Hongyu Cao, Xinyuan Wang, Arun Vignesh Malarkkan, Kunpeng Liu, Haifeng Chen, Yanjie Fu
arXiv:2610.07008v1 Announce Type: new
Abstract: Few-shot class-incremental learning (FSCIL) aims to learn novel classes from limited annotations while preserving prior knowledge. Existing methods typ...
By Junhui Yin, Yuchen Yang, Yilin Yin, Shuai Na, Haoran Xi, Jianhua Yang, Muyi Sun, Man Zhang, Shengfeng He
arXiv:2608. 09507v1 Announce Type: cross Abstract: Natural language user preferences provide an interpretable interface for LLM personalization.
By Yuting Liu, Wei Wu, Jianzhe Zhao, Guibing Guo