arXiv AI October 1, 2026 By Christina Hahn, Shangbin Feng, Dean Light, Swastik Roy, Hila Gonen, Yulia Tsvetkov

Multi-LLM Collaborative Alignment via Stackelberg Games

The paper introduces Stackelberg Alignment, a leader‑follower framework that lets a pool of language models collaborate and improve by learning from each other’s responses. An EXP3 bandit leader adaptively selects instructions based on difficulty and discriminability, while the models act as followers, evaluating peers and learning via DPO or GRPO with Elo‑style reputation weighting and opponent matching. Experiments on diverse benchmarks show that this adaptive curriculum outperforms static baselines by up to 12‑25% and improves multi‑LLM evolution.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 23

Rewarding Better Thinking for LLM Preference Alignment

arXiv:2607. 19824v1 Announce Type: new Abstract: LLM preference alignment aims to optimize models toward human preferences across diverse user instructions.

By Xubo Liu, Wenya Guo, Ruxue Yan, Xinying Qian, Ying Zhang

llms reinforcement-learning benchmarks safety

arXiv Computation and Language

Sep 1

Small Reward Models via Backward Inference

arXiv:2602.13551v3 Announce Type: replace Abstract: Reward models (RMs) play a central role throughout the language model (LM) pipeline, particularly in non-verifiable domains. However, the dominant...

By Yike Wang, Faeze Brahman, Shangbin Feng, Teng Xiao, Hannaneh Hajishirzi, Yulia Tsvetkov

llms reinforcement-learning

arXiv AI

Jun 19

AAPA: Adversarially Anchored Preference Alignment for Post-Training of Large Language Models

arXiv:2509. 25148v2 Announce Type: replace Abstract: Post-training alignment of large language models often combines supervised fine-tuning (SFT) on expert demonstrations with reinforcement learning (RL) from preference or verifiable feedback.

By Faqiang Qian, Kang An, Weikun Zhang, Ziliang Wang, Xuhui Zheng, Liangjian Wen, Yong Dai, Mengya Gao, Yichao Wu

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Aug 12

Multiplayer Nash Preference Optimization

arXiv:2509. 23102v4 Announce Type: replace Abstract: Reinforcement learning from human feedback (RLHF) has emerged as the standard paradigm for aligning large language models with human preferences.

By Fang Wu, Xu Huang, Weihao Xuan, Zhiwei Zhang, Yijia Xiao, Guancheng Wan, Xiaomin Li, Bing Hu, Peng Xia, Jure Leskovec, Yejin Choi

llms reinforcement-learning benchmarks safety

arXiv AI

Jul 3

Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling

arXiv:2607. 01612v1 Announce Type: new Abstract: Training large language models (LLMs) with reinforcement learning (RL) has significantly advanced their performance on reasoning and question-answering tasks.

By Xuqing Yang, Yi Yuan, Shanzhe Lei, Xuhong Wang

llms reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Sep 23

PACT: From Credit Assignment to Critic Alignment

arXiv:2609.26355v1 Announce Type: new Abstract: Reinforcement learning has become a central component of large language model (LLM) post-training, yet token-level credit lacks a generally accepted ma...

By Jiayan Fu, Hang Xu, Yong Zhang, Zhaokai Luo, Yao Hu, Dongyan Zhao, Mu Chuan

llms agents reinforcement-learning efficiency benchmarks safety

One email a morning, machine-written

Related stories

Rewarding Better Thinking for LLM Preference Alignment

Small Reward Models via Backward Inference

AAPA: Adversarially Anchored Preference Alignment for Post-Training of Large Language Models

Multiplayer Nash Preference Optimization

Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling

PACT: From Credit Assignment to Critic Alignment