Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,201 stories · RSS feed

arXiv Machine Learning

Oct 15

Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents

arXiv:2610.19439v1 Announce Type: new Abstract: While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substanti...

By Yuanpu Cao, Saket Sathe, Hanyu Wang, Ziyi Yin, Fenglong Ma, Jinghui Chen

llms agents fine-tuning efficiency benchmarks

arXiv Machine Learning

Oct 15

ReGRA: Residual-Gradient Adaptation for Complementary Low-Rank Fine-Tuning

arXiv:2610.19700v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) enables parameter-efficient fine-tuning by learning low-rank weight updates. Increasing rank expands adaptation capacity, bu...

By Fanyao Li, Yuxi Liu, Xiaohao Liu, He Chang, Zhaoxin Wang, Zhulin Tao

fine-tuning benchmarks

arXiv Machine Learning

Oct 15

MAEM: Mitigating Recency Bias via a Replay-Free, Capacity-Controlled Dynamic Mixture of Experts for Class-Incremental Graph Learning

arXiv:2610.20238v1 Announce Type: new Abstract: Graph Neural Networks (GNNs) excel on graph-structured data but suffer catastrophic forgetting when classes arrive sequentially. In class-incremental g...

By Junlin Chen, Wenhao Wang, Junchao Xu, Qingsong Yao, Fei Xue, Li Sun, Xiao Wang, Ruijie Wang

fine-tuning benchmarks safety

arXiv Machine Learning

Oct 15

Training Dynamics of Transformers on Unbounded Inputs: Attention Learning Can Lag Output Layer Learning

arXiv:2610.20337v1 Announce Type: new Abstract: The success of transformers is attributed to several factors, including their capacity to express formal languages over variable-length inputs, and abi...

By Yaomengxi Han, Debarghya Ghoshdastidar

llms fine-tuning

arXiv Machine Learning

Oct 15

Learning by Examples: Power Grid Steady State Analysis with a Tabular Foundation Model

arXiv:2610.20462v1 Announce Type: new Abstract: Foundation models have shown strong generalization across scientific domains, but their use in power systems remains largely task-specific. We investig...

By Olayiwola Arowolo, Oluwadara Adedeji, Peter Palensky, Jochen L. Cremer

fine-tuning

arXiv Machine Learning

Oct 15

DAGFormer: Global Topology Prediction for Transformer Cross-Layer Connections

arXiv:2610.19625v1 Announce Type: cross Abstract: We propose DAGFormer, a Transformer language model whose cross-layer connections are planned by a jointly trained topology predictor. Beyond the resi...

By Xiaocong Yang, Yuren Hao, ChengXiang Zhai

llms fine-tuning efficiency

arXiv Machine Learning

Oct 15

Following the Flow the Law Requires: Training Voice Agents to Transfer to Unseen Regulated Procedures in Debt Collection

arXiv:2610.18881v1 Announce Type: cross Abstract: Voice agents built on large language models (LLMs) now handle customer calls, but in regulated applications completing the task is not enough: the ag...

By Petmongkon Pornpichitsuwan, Krittin Pachtrachai, Touchapon Kraisingkorn

llms agents fine-tuning

arXiv Machine Learning

Oct 15

Continual pretraining Prostate MRI ViT Encoder for task alignment

arXiv:2610.20217v1 Announce Type: cross Abstract: Building foundation models for MRI has emerged as a promising direction for advancing medical image analysis and improving accessibility of task spec...

By Bilal A. Sidiqi, Yipei Wang, Shaheer U. Saeed, Shiqi Huang, Daniel C. Alexander, Yipeng Hu

llms computer-vision fine-tuning safety

arXiv Machine Learning

Oct 15

Forget About It: Mitigating Forgetting Without Replay or Auxiliary Objectives

arXiv:2501.05559v2 Announce Type: replace Abstract: Have we made mitigating catastrophic forgetting more complicated than it needs to be? Continual adaptation often degrades performance on earlier ta...

By Anat Kleiman, Gintare Karolina Dziugaite, Jonathan Frankle, Sham Kakade, Mansheej Paul

fine-tuning

arXiv Machine Learning

Oct 15

Long-Range Distillation: Distilling 10,000 Years of Simulated Climate into Long Timestep AI Weather Models

arXiv:2512.22814v2 Announce Type: replace Abstract: Accurate long-range weather forecasting remains a major challenge for AI models, both because errors accumulate over autoregressive rollouts and be...

By Scott A. Martin, Noah Brenowitz, Dale Durran, Michael Pritchard

diffusion fine-tuning efficiency

arXiv Machine Learning

Oct 15

Activation Steering Induces Emergent Misalignment: Steering Vector Construction and Comprehensive Evaluation

arXiv:2606.08682v2 Announce Type: replace Abstract: Activation steering (AS) has emerged as a popular inference-time technique for modulating the behavior of large language models (LLMs). AS modulate...

By Qi Cao, Jian Lou, Meiting Liu, Wenjie Feng, Dan Li, Anh Tuan Luu

llms fine-tuning safety