arXiv Machine Learning

Pairwise AUC Optimization Needs Corrective Power: A Unified View

The paper investigates why direct pairwise AUC surrogate training, though aligned with the ranking metric, can fail to provide corrective updates during stochastic training. It introduces the concept of Corrective Power, showing that in certain regions severe ranking errors persist without generating strong gradient signals, a phenomenon linked to degeneracies in the U‑statistics of pairwise gradients. The authors explain empirical observations—such as the effectiveness of squared‑hinge loss, the benefit of cross‑entropy warm‑up, and the advantage of parameter‑efficient fine‑tuning—through this theoretical lens, and validate their claims with experiments on eight computer‑vision and eight NLP tasks using various models.

arXiv Machine Learning

Oct 9

Keeping Score: Adaptive, Tuning-Free Loss Weighting for Score-Augmented Neural Ratio Estimation

The paper introduces an adaptive, tuning‑free method for weighting the score‑matching term in the loss function of neural ratio estimators, eliminating the need for manual hyper‑parameter tuning. By adjusting the score loss weights during training based on loss gradients, the approach adds minimal overhead to standard classifier training. Experiments on network dynamics and spatial process models show that this technique improves surrogate quality and can match the inference performance of a 10× larger training set while keeping training time under 1.1× higher.

By Alexander Shen, Mikael Kuusela

arXiv Machine Learning

Oct 3

Every Batch Is Its Own Validation Set: Leave-One-Out Gradient Matching for Online Data Selection in LLM Fine-Tuning

arXiv:2610. 00436v1 Announce Type: new Abstract: Online batch selection fine-tunes a language model on the most useful part of each candidate batch.

By Hongyu Chen, Xinyi Luo, Ming Zhao, Lin Tang, Zihan Xu, Jing Li, Yuxuan Wang, Haoran Deng, Wei Zhang

llms fine-tuning

arXiv Machine Learning

Jun 10

AdaGC: Enhancing LLM Pretraining Stability via Adaptive Gradient Clipping

arXiv:2502. 11034v3 Announce Type: replace Abstract: Loss spikes remain a persistent obstacle in large-scale language model pretraining.

By Guoxia Wang, Shuai Li, Congliang Chen, Jinle Zeng, Jiabin Yang, Dianhai Yu, Yanjun Ma, Li Shen

llms

arXiv AI

4d ago

The Shape of the DPO Hyperparameter Landscape: What to Measure, How It Moves with Scale, and When Not to Transfer It

arXiv:2610.27895v1 Announce Type: cross Abstract: Direct preference optimisation (DPO) is widely used for language model post-training, but its performance depends strongly on the choice of its hyper...

By Hannan Mahadik, David Salinas, Jacek Golebiowski, Aaron Klein

llms reinforcement-learning benchmarks

arXiv AI

Jul 21

First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers

arXiv:2607. 16821v1 Announce Type: cross Abstract: Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space.

By Irina Piontkovskaia, Sergey Nikolenko

llms fine-tuning

arXiv Machine Learning

Jun 9

Lost in the Non-convex Loss Landscape: How to Fine-tune the Large Time Series Model?

arXiv:2606. 08578v1 Announce Type: new Abstract: Recently, large time series models (LTSMs) have gained increasing attention due to their similarities to large language models, including flexible context length, scalability, and task generality, outperforming advanced task-specific models.

By Xu Zhang, Peang Wang, Wei Wang

llms fine-tuning benchmarks

arXiv Machine Learning

Jul 31

Towards Stability of Parameter-Free Optimization

arXiv:2405. 04376v4 Announce Type: replace Abstract: Hyperparameter tuning, particularly the selection of an appropriate learning rate in adaptive gradient training methods, remains a challenge.

By Yijiang Pang, Shuyang Yu, Bao Hoang, Jiayu Zhou

arXiv AI

Jul 21

Normalized Rewards for Preference Optimization

arXiv:2607. 16240v1 Announce Type: cross Abstract: Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs with human preferences.

By Shawn Im, Federico Danieli, Skyler Seto, Barry-John Theobald, Katherine Metcalf

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Sep 1

PRIME: Mitigating Subgroup Optimization Competition in Shared CTR Top Networks with Plug-in Residual Input-Conditioned Mixture of Expert

PRIME is a plug‑in residual input‑conditioned mixture of experts that preserves the original dense prediction path while adding low‑rank, input‑dependent logit corrections. By initializing residuals to zero, PRIME matches the baseline dense model at training start and stabilizes conditional estimation with multi‑bag aggregation and EMA load biases. Experiments on Avazu and Criteo across 13 CTR architectures show modest AUC and LogLoss gains, with PRIME outperforming APG on FiBiNET and DCNv2 while using fewer parameters and lower latency.

By Heng Yao, Siyun Hou, Tianying Liu, Yulou Shu, Yong He, Chuan Yuan, Kaibin Qiu, Guowei Chen, Jiayu Zhao, Chao Yu, Ke Ding

efficiency

arXiv Machine Learning

Oct 16

Noise-Space Reward Alignment for Few-Step Generative Models

The paper introduces Noise Policy Optimization (NPO), a method for aligning few-step generative models with black-box reward signals. NPO learns an amortized flow policy that transforms initial Gaussian noise into a reward-tilted distribution, allowing the pretrained generator to produce higher-reward outputs without fine-tuning. The approach achieves superior reward-quality trade-offs across image, language, and protein generation tasks while drastically reducing inference cost compared to test-time search methods.

By Woocheol Shin, Taeyoung Yun, Jiwon Sung, Hyeongyu Kang, Jeongmin Park, Jaewoo Lee, Kyuil Sim, Dohyun Kim, Yutong He, Jinkyoo Park

diffusion reinforcement-learning fine-tuning safety

arXiv Machine Learning

Aug 31

DARTS: Decoder-Aware Representation Tuning via Surgery for Model Merging

The paper introduces DARTS, a method for tuning decoder representations during model merging. It addresses representation bias in autoregressive decoders by using an entropy‑weighted L1 loss and a per‑position additive bias to correct errors that accumulate across token positions. Experiments on code generation, mathematical reasoning, and instruction following with Llama‑2‑7B show that DARTS improves performance over standard surgery while adding only 0.1% extra parameters.

By Aaryan Ajay Sharma, Sai Nishanth Padala, Seganrasan Subramanian

llms fine-tuning benchmarks safety

arXiv Machine Learning

3d ago

ExGrad: Beyond Suppression - Explicit Clean-Gradient Subtraction in Adversarial Training

arXiv:2610.30018v1 Announce Type: new Abstract: Adversarial training derives updates from surrogate losses, but source-loss descent does not directly characterize an update's effect on adversarial ma...

By Zhiyuan Ye, Xiangyu Zhou, Ji Qi, Hao Zhang, Yi Zhou

safety

Related stories