arXiv AI
Jul 21
Normalized Rewards for Preference Optimization
arXiv:2607. 16240v1 Announce Type: cross Abstract: Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs with human preferences.
Unimodality-Promoting Regularized Learning for Ordinal Regression
arXiv:2608. 08359v1 Announce Type: new Abstract: Ordinal regression, also called ordinal classification, is classification of ordinal data, in which the underlying target variable is categorical and considered to have a natural ordinal relation.
Related stories
arXiv Statistics ML
Oct 28
LLM knowledge for improved Out-of-Distribution Tabular Prediction - A Cross-Validated Regularisation Approach
The study investigates whether coefficient priors derived from large language models (LLMs) can enhance out‑of‑distribution (OOD) tabular prediction by serving as a shrinkage target in regularised regression. Across five datasets, LLM‑elicited priors improved OOD mean squared error on four datasets by up to 7.9% and yielded better probabilistic accuracy in classification tasks, though the gains varied and were sometimes comparable to in‑domain improvements. Cross‑validation was used to select priors, but the variability among candidates meant that the method mainly avoided the worst priors rather than consistently finding the best. "whyItMatters":"The findings suggest that LLM‑derived priors can provide a scalable alternative to expert elicitation for improving model robustness to distribution shifts, though their effectiveness depends on dataset size and prior variability."
arXiv Machine Learning
Sep 22
On Mitigation of Subliminal Learning in Large Language Models
arXiv:2609.22215v1 Announce Type: cross Abstract: Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unr...
arXiv Machine Learning
Sep 30
Reference-Guided Machine Unlearning
Reference-Guided Machine Unlearning (ReGUn) is a vision unlearning framework that prioritizes distributional indistinguishability over degradation-based heuristics. It uses disjoint held-out data to create a class-conditioned reference distribution for distillation, guiding forget samples toward non-member behavior without explicitly degrading predictions. Experiments across various architectures and datasets show that ReGUn achieves a competitive forgetting–utility trade-off and closely matches retrain-like membership inference behavior.
arXiv Machine Learning
Aug 28
Scaling Model-Generated Distillation Data Can Make Latent Teacher Traits More Recoverable
The paper investigates how scaling the amount of model-generated, off‑task distillation data affects the recoverability of teacher‑induced traits in student models. In a controlled subliminal‑learning setup, teachers are prompted to express a target trait, producing restricted data such as number‑only completions. Students trained on larger independent datasets show a clearer manifestation of the teacher’s trait in a separate evaluation domain, with the effect being strongest when the trait is already favored or when alternative traits are present. The authors find this trend holds across model families, trait types, multi‑trait settings, and cross‑model transfer, and suggest that scaling should be coupled with trait‑aware curation and evaluation.
arXiv Machine Learning
Oct 23
Tracing Subliminal Learning Through Pretraining
The paper investigates subliminal learning, where large language models inherit unintended traits from data generated by other models. It compares hard‑label distillation (students learn from biased teacher outputs) and on‑policy soft‑label distillation (students generate data while teacher logits guide learning), finding that transfer persists across various training conditions and that teacher‑student roles can be asymmetrically effective. Synthetic experiments show that preference signals can be transmitted even without prompt dependence, and that removing certain tokens reduces but does not eliminate transfer, with soft‑label distillation yielding more consistent results with less training.
arXiv Machine Learning
Sep 2
Patterning in Practice: Debiasing Reward Models with Susceptibilities
The paper introduces patterning, a reweighting technique that adjusts preference pairs based on their susceptibility to bias, to debias a Gemma 2 9B Instruct reward model trained on Skywork-Reward-Preference v0.2. Using this method, the authors achieve a +14.2 ± 1.2 percentage point improvement on the RM‑Bench Hard split while maintaining overall accuracy, matching the best reported Hard‑split gain from a comparable model. The study also demonstrates that the learned weights are interpretable, transferable across Gemma variants, and partially effective on Llama 3.1 8B.
arXiv AI
Sep 1
Learning Dynamics of Logits Debiasing for Long-Tailed Semi-Supervised Learning
arXiv:2608.30699v1 Announce Type: cross Abstract: Long-tailed distributions are prevalent in real-world semi-supervised learning (SSL), where pseudo-labels tend to favor majority classes, leading to...
arXiv Machine Learning
6d ago
Continual Classification Beyond the Kernel Regime: Harnessing the Sparsity Bias of L2 Regularization in Diagonal Linear Networks
The paper investigates logistic regression for continual learning with linear predictors in high-dimensional settings, deriving the optimal L2 regularization strength and its generalization performance. It then explores deeper models via a two‑layer diagonal linear network, showing that standard L2 regularization loses its sparsity bias in continual learning. To restore this benefit, the authors propose a weighted L2 variant that accelerates convergence for sparse teachers, outperforming conventional L2 or L1 regularization over moderate task horizons.
arXiv Machine Learning
Jun 2
Optimal Regularization for Performative Learning
arXiv:2510. 12249v2 Announce Type: replace Abstract: In performative learning, the data distribution reacts to the deployed model - for example, because strategic users adapt their features to game it - which creates a more complex dynamic than in classical supervised learning.
arXiv Machine Learning
Sep 11
Adaptive Margin Ordinal Loss: Penalizing Center-Class Hedging in Ordinal Classification
The paper introduces the Adaptive Margin Ordinal Loss (AMOL), a new loss function designed to reduce the tendency of neural networks to predict center classes in ordinal classification tasks—a problem called center‑class hedging. AMOL applies a multiplicative weight to per‑class loss terms that is large only when a candidate class is near the center while the true label is far from it, thereby discouraging hedging. The authors also propose the Center‑Hedging Rate (CHR) metric to quantify this failure mode and demonstrate that AMOL achieves state‑of‑the‑art Quadratic Weighted Kappa scores on four benchmarks, with an asymmetric variant eliminating hedging on the Abalone dataset.
arXiv AI
Oct 21
Help Me Help You: The Aggregate Value of Source and Target Data in Transfer Learning
arXiv:2408.16189v4 Announce Type: replace-cross Abstract: Transfer learning, in regression or classification, concerns problems where one aims to use imperfect data from a source distribution P to im...