← Back to all news

Fine-tuning and adaptation

LoRA, PEFT, instruction tuning and domain adaptation — adapting a pretrained model without paying to train one.

8,201 stories · RSS feed

arXiv AI
Oct 1

Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving

Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving proposes EMPlan, a hybrid trajectory planning method that combines sparse anchors with an offset refinement module for low-latency, high-accuracy predictions. The approach uses a two-stage training paradigm—pretraining followed by reward-guided fine-tuning—to improve safety without extra inference cost, leveraging rule-based reward signals and unpaired preference supervision. EMPlan is evaluated on the non-reactive NAVSIM benchmark, achieving a favorable balance between planning accuracy and efficiency under real-time constraints.

By Chenglin Chen, Lujia Wang, Xinhu Zheng, Jun Ma, Haoang Li
agentsroboticsfine-tuningmultimodalbenchmarkssafety
More like this →
arXiv AI
Oct 1

SCIC: Scope- and Codebook-Aware Instruction Conditioning for Speaker-Adapted Expressive TTS

The paper introduces SCIC, a method that enhances speaker‑adapted expressive TTS by conditioning prosody instructions on both scope and codebook usage. SCIC uses a Temporal Instruction Router to activate frame‑level tags and Tag‑Specific Codebook Weighting to target residual codebooks for Pitch, Energy, and Speed, while Pause uses absolute durations. The approach improves clause‑level prosodic control and yields a clearer paragraph‑level expressive hierarchy compared to standard instruction fine‑tuning, with post‑training optimization preserving quality and speaker similarity.

By Longyu Lu, Zongwei Du, Mengtao Xing, Zhuoqun Liu, Zifan Guan, Meiguang Jin, Junfeng Ma
fine-tuning
More like this →
arXiv AI
Oct 1

EviRover: Reinforcing Agentic Perception Beyond a Glance

EviRover is a perception agent that goes beyond a single glance by actively gathering information to resolve perceptual queries. The authors created two data generation pipelines, producing EviRover-SFT-5K and EviRover-RL-12K, and a human‑verified benchmark called EviLens with 688 instances across five perception categories. Trained with supervised fine‑tuning and agentic reinforcement learning, the 4B EviRover outperforms its backbone by an average of 30 points on EviLens and shows strong transfer to other benchmarks such as WebEyes and BrowseComp‑VL.

By Kaixuan Fan, Kaituo Feng, Tianshuo Peng, Yilei Jiang, Manyuan Zhang, Junke Wang, Xiangyu Yue
agentsreinforcement-learningfine-tuningmultimodalbenchmarks
More like this →
arXiv Computation and Language
Oct 1

GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis

arXiv:2609.38923v1 Announce Type: new Abstract: Working agents need to read diverse files, coordinate tools, and produce deliverables. Training such agents requires tasks built on many real files wit...

By Qisheng Su, Hanchen Wang, Guanru Zhu, Huicheng Jiang, Qiuyinzhe Zhang, Kou Shi, Zhen Fang, Ziao Zhang, Qingnan Ren, Zehui Chen, Tao Gui, Feng Zhao
llmsagentsfine-tuningbenchmarks
More like this →
arXiv AI
Oct 1

ComputerSD: Online Self-Distillation from Real-Time Feedback for Computer-Use Agents

ComputerSD is an online self‑distillation method for computer‑use agents that leverages real‑time feedback from executed GUI transitions. It uses a fine‑tuned GUI analyzer to generate guidance and a step‑level value score after each action, combining token‑level OPSD with trajectory‑level GRPO in an asynchronous training framework. On the OSWorld‑Verified benchmark, ComputerSD improves performance over outcome‑only GRPO by 1.9 and 4.1 percentage points on Qwen3‑VL‑8B‑Thinking and EvoCUA‑8B backbones, and shows strong generalizability in out‑of‑distribution tests.

By Yong Du, Tongbo Chen, Zhengxi Lu, Yizhou Liu, Bofan Chen, Tao Jiang, Wenhao Xu, Yongliang Shen
agentsfine-tuningefficiency
More like this →
arXiv AI
Oct 1

Understanding Off- vs On-Policy Distillation: A Tale of Distinct Training Objectives

The paper investigates on‑policy distillation (OPD) versus supervised fine‑tuning (SFT), focusing on how students learn from multiple teachers by minimizing divergence. It shows that using forward KL divergence leads to a weighted arithmetic mixture, while reverse KL produces a normalized weighted geometric aggregate. The authors develop algorithms for both off‑policy and on‑policy settings, prove logarithmic regret bounds in tabular cases, extend the analysis to function approximation, and analyze how these aggregation targets explain OPD’s benefits and fragility.

By Qiwei Di, Xuheng Li, Kaixuan Ji, Chenggong Zhang, Heyang Zhao, Quanquan Gu
fine-tuningefficiency
More like this →
arXiv AI
Oct 1

Evaluating Persistent Calibration under Evolving Model Knowledge

The paper introduces the concept of persistent calibration, which requires a confidence estimator to accurately reflect a model’s evolving knowledge without additional supervision. It evaluates this by comparing confidence estimators trained on earlier checkpoints to their performance on later checkpoints using knowledge contrast sets—questions that shift from correct to incorrect answers across checkpoints. The study finds that standard inference-time and fine-tuning methods underperform compared to oracle methods, and suggests that multi-checkpoint training can improve calibration by identifying robust confidence features.

By Victor Wang, Thomas Hofweber, Mohit Bansal, Elias Stengel-Eskin
agentsfine-tuning
More like this →
arXiv AI
Oct 1

Unmerge: Efficient Machine Unlearning via Task Arithmetic

The paper introduces Unmerge, an efficient machine unlearning algorithm that treats unlearning as the inverse of task arithmetic. By representing the forget component as a low‑rank basis at each layer, Unmerge optimizes three goals—matching the merged vector, suppressing leakage, and bounding correction size—to limit forget leakage and retain damage. Experiments on ResNet‑50, ViT‑S/16, and Llama‑3.2‑3B show significant performance gains over existing methods while maintaining privacy and feature‑distribution fidelity.

By Haoran Tang, Andrew Tan, Rajiv Khanna
llmsfine-tuning
More like this →
arXiv AI
Oct 1

Signal-Routed Temperature Scaling: Low-Capacity Risk-Conditioned Calibration for Small Validation Budgets

Signal‑Routed Temperature Scaling (SRTS‑BCE) is a 10‑parameter, argmax‑preserving calibration method that separates calibration objectives from adaptive capacity. It cross‑fits a correctness‑risk score over six logit statistics and assigns a top‑label BCE temperature to each of three risk groups, generalizing TvA‑TS when K=1. Experiments on fine‑tuned CIFAR‑100 and ViT‑B/16 show that SRTS‑BCE reduces ECE from 1.65 to 0.96 with a small calibration budget, outperforming higher‑capacity SMART+BCE when only 250 examples are available, and revealing a budget‑dependent ranking reversal on Swin‑T.

By Wenhao Liang, Liangwei Nathan Zheng, Lin Yue, Wei Emma Zhang, Mingyu Guo, Olaf Maennel, Weitong Chen
fine-tuning
More like this →