arXiv Machine Learning July 9, 2026 By Aoxiong Zeng, Yuxin Yang, Xiangquan Yang

Online Data Selection Is Implicit Alignment

Read the original on arXiv Machine Learning →

arXiv:2607. 07023v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) is often treated as a capability-adaptation step, while alignment is attributed to later preference optimization or reinforcement learning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Jul 8

Online Data Selection Is Implicit Alignment

Supervised fine-tuning (SFT) is often treated as a capability-adaptation step, while alignment is attributed to later preference optimization or reinforcement learning. This separation is incomplete: when examples are scored and kept online during fine-tuning, the choice of which data to train on already changes the model's behavioral preferences.

reinforcement-learning fine-tuning safety

arXiv AI

Jul 28

How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift

arXiv:2607. 22676v1 Announce Type: new Abstract: Post-training is a key mechanism for adapting large language models to downstream tasks.

By James Elcock, William F. Shen, Xinchi Qiu, Nicholas D. Lane

llms reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 16

Value Drifts: Tracing Value Alignment During LLM Post-Training

arXiv:2510. 26707v2 Announce Type: replace-cross Abstract: As LLMs occupy an increasingly important role in society, they are more and more confronted with questions that require them not only to draw on their general knowledge but also to align with certain human value systems.

By Mehar Bhatia, Shravan Nayak, Gaurav Kamath, Marius Mosbach, Karolina Sta\'nczak, Vered Shwartz, Siva Reddy

llms robotics fine-tuning safety

arXiv AI

4d ago

Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning

arXiv:2609.36862v1 Announce Type: cross Abstract: Fine-tuning-as-a-service lets users adapt a safety-aligned language model to their own data, but it also creates a harmful fine-tuning attack surface...

By Muhammad Zeeshan Akram, Mufid Kamel Marican, Anvesh Reddy Yenugu, Ali Zain Kaimkhani, Minghong Fang

llms rag fine-tuning safety

arXiv AI

Jul 21

Normalized Rewards for Preference Optimization

arXiv:2607. 16240v1 Announce Type: cross Abstract: Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs with human preferences.

By Shawn Im, Federico Danieli, Skyler Seto, Barry-John Theobald, Katherine Metcalf

llms reinforcement-learning benchmarks safety

arXiv AI

Sep 15

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

arXiv:2602.13576v2 Announce Type: replace-cross Abstract: Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-langu...

By Ruomeng Ding, Yifei Pang, He Sun, Yizhong Wang, Zhiwei Steven Wu, Zhun Deng

llms benchmarks safety

One email a morning, machine-written

Related stories

Online Data Selection Is Implicit Alignment

How LLM Task-Adaptation Reshapes Alignment: A Multi-dimensional Study of Behavioral and Representational Drift

Value Drifts: Tracing Value Alignment During LLM Post-Training

Safer Content or Firmer Refusals? A Hybrid Perturbation Defense for Alignment under Harmful Fine-tuning

Normalized Rewards for Preference Optimization

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges