arXiv Computation and Language August 28, 2026 By Seohyeong Lee, Hwaran Lee, Buru Chang

Instruction Quality Matters: Refining Instructions for Effective Preference Learning

Read the original on arXiv Computation and Language →

The paper investigates how the quality of instructions used to generate response pairs affects preference learning for language models. It shows that low‑quality or ambiguous instructions limit the range of response quality, weakening preference signals, and introduces an instruction‑refinement pipeline that improves data quality without discarding examples. Experiments across models and benchmarks demonstrate that refining instructions leads to better alignment and complements other data‑improvement methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Sep 1

Cross-Relational Preference Learning for Better LLM Instruction Following

arXiv:2608.29352v1 Announce Type: new Abstract: Large Language Models (LLMs) still exhibit limited capability in following complex instructions. While existing approaches often rely on preference lea...

By Runsheng Li, Kai Sun, Bin Shi, Bo Dong

llms reinforcement-learning benchmarks

arXiv AI

Jun 24

Towards Spec Learning: Inference-Time Alignment from Preference Pairs

arXiv:2606. 24004v1 Announce Type: cross Abstract: Steering a large language model (LLM) toward a desired behavior typically relies on an iterative process of hand-crafting a prompt based on a careful inspection of the model's responses.

By Dhriti Krishnan, Tejas Goyal, Jaromir Savelka

llms reinforcement-learning fine-tuning safety

arXiv AI

Jun 19

Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings

arXiv:2606. 19744v1 Announce Type: cross Abstract: Aligning language models with human preferences often requires optimising multiple behavioural objectives.

By Pranav Bhandari, Nicolas Fay, Amitava Datta, Usman Naseem, Mehwish Nasim

llms reinforcement-learning fine-tuning safety

arXiv AI

Jun 11

Toward Preference-aligned Large Language Models via Residual-based Model Steering

arXiv:2509. 23982v2 Announce Type: replace-cross Abstract: Preference alignment is a critical step in making Large Language Models (LLMs) useful and aligned with (human) preferences.

By Lucio La Cava, Andrea Tagarelli

llms reinforcement-learning benchmarks safety

arXiv AI

Aug 18

Leveraging Machine Unlearning for Cost-Efficient Preference Alignment

arXiv:2504. 06659v2 Announce Type: replace-cross Abstract: Despite advances in Preference Alignment (PA) for Large Language Models (LLMs), mainstream methods like reinforcement learning with human feedback face notable challenges.

By Xiaohua Feng, Yuyuan Li, Huwei Ji, Jiaming Zhang, Li Zhang, Tianyu Du, Chaochao Chen

llms reinforcement-learning safety

arXiv Machine Learning

Jun 19

A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)

arXiv:2602. 14696v2 Announce Type: replace Abstract: Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.

By Nihal V. Nayak, Paula Rodriguez-Diaz, Neha Hulkund, Sara Beery, David Alvarez-Melis

llms fine-tuning

One email a morning, machine-written

Related stories

Cross-Relational Preference Learning for Better LLM Instruction Following

Towards Spec Learning: Inference-Time Alignment from Preference Pairs

Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings

Toward Preference-aligned Large Language Models via Residual-based Model Steering

Leveraging Machine Unlearning for Cost-Efficient Preference Alignment

A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)