arXiv:2607. 03248v1 Announce Type: cross Abstract: The alignment of large language models with human preferences is commonly achieved through Reinforcement Learning from Human Feedback or Direct Preference Optimization.
arXiv:2509. 23982v2 Announce Type: replace-cross Abstract: Preference alignment is a critical step in making Large Language Models (LLMs) useful and aligned with (human) preferences.
Aligning large language models to human preferences is crucial for real-world deployment but frequently incurs an alignment tax, leading to the catastrophic forgetting of pre-trained general capabilit...
arXiv:2607. 24845v1 Announce Type: cross Abstract: Large language models (LLMs) have been applied to sequential recommendation by formulating it as a natural language task.
Re2A is a new framework for situated conversational recommendation that models user interactions within shared physical environments. It introduces rubric-based preference reasoning to explicitly capture user preferences from dialogue history and scene context, and a preference-conditioned optimization to align generated responses with both user satisfaction and situational consistency. Experiments on two SCR datasets show that Re2A outperforms existing methods, providing more precise and context-aware recommendations.
arXiv:2609.01246v1 Announce Type: new Abstract: Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet p...
The paper introduces BALIGN, a balanced data selection strategy designed to reduce catastrophic forgetting—referred to as the alignment tax—in large language models during preference-based alignment. By analyzing preference optimization gradients, the authors identify three data-centric features that influence parameter drift: the reference model's log-probability margin, token length differences between chosen and rejected responses, and TF‑IDF similarity to general capability corpora. BALIGN aggregates these features into a composite risk score to filter out high-risk preference samples, thereby preserving foundational capabilities while maintaining alignment gains with minimal computational overhead.
arXiv:2509. 26169v2 Announce Type: replace Abstract: Alignment of large language models remains a central challenge in natural language processing.
arXiv:2609.05899v1 Announce Type: cross Abstract: Aligning large language models with human preferences remains a challenge, primarily due to the critical role of preference data quality in effective...
arXiv:2602. 02898v3 Announce Type: replace Abstract: Language model benchmarks are pervasive and computationally-efficient proxies for real-world performance.
The paper introduces GAP-DPO, a method for personalizing large language models by selecting preference pairs based on gradient alignment with user utility. It formalizes personalized preference learning as a geometry‑aligned optimization problem, showing that off‑policy sampling can shift DPO updates from error correction to reinforcement when preference margins align with utility gradients. Experiments demonstrate that GAP‑DPO improves stylistic fidelity, preference alignment, and overall generation quality over standard DPO variants.
arXiv:2504. 06659v2 Announce Type: replace-cross Abstract: Despite advances in Preference Alignment (PA) for Large Language Models (LLMs), mainstream methods like reinforcement learning with human feedback face notable challenges.