arXiv:2608. 06179v1 Announce Type: new Abstract: Preference optimisation has proven effective for improving large language models but typically relies on costly human preference annotations.
By Hoda Fakharzadehjahromy, Emil Wiman, Andreas Bueff, Hafsteinn Einarsson, Fredrik Heintz
arXiv:2608.23149v1 Announce Type: cross
Abstract: The alignment of Large Language Models heavily relies on English-centric high-quality preference data, which often leads to suboptimal performance in...
By Seungyoon Lee, Minhyuk Kim, Jungseob Lee, Heuiseok Lim
The paper proposes a method for culturally aligning large language models (LLMs) using soft prompt tuning optimized via Differential Evolution (DE). Unlike traditional fine‑tuning or reinforcement learning, this approach keeps model weights frozen and requires no preference data, instead leveraging aggregated survey scores from Hofstede's Value Survey Module (VSM13). Experiments on four countries and four instruction‑tuned models show that DE‑optimized prompts reduce cultural discrepancy, improve agreement with the World Values Survey, and are preferred in blinded pairwise evaluations by LLM judges.
By Reem I. Masoud, Martin Ferianc, Philip Treleaven, Miguel Rodrigues
The paper introduces Cross‑lingual Ranking Preference Optimization (CRPO), a framework that uses high‑quality English preference data to improve alignment of large language models in other languages. CRPO builds a hierarchical structure over parallel preference pairs, jointly optimizing intra‑ and inter‑lingual preferences and providing a relative ranking signal beyond binary comparisons. Experiments on five languages show consistent gains in instruction‑following and knowledge utilization, with robust performance across different weighting schemes and improved reward margins and log‑probabilities of desirable responses.
By Seungyoon Lee, Minhyuk Kim, Jungseob Lee, Heuiseok Lim
arXiv:2605. 11632v2 Announce Type: replace-cross Abstract: Self-generated counterfactual explanations (SCEs) are minimally modified inputs (minimality) generated by large language models (LLMs) that flip their own predictions (validity), offering a causally grounded approach to unraveling black-box LLM behavior.
By Yilong Wang, Qianli Wang, Bohao Chu, Yihong Liu, Jing Yang, Simon Ostermann
PADM'E is a method for synthesizing preference‑aligned data to meta‑evaluate language‑model (LM) evaluators of agentic behaviors. It reframes meta‑evaluation as a preference judgment problem, generating criterion‑based data with small LMs and no human involvement. In a prototype, PADM'E produced 1,000 samples across four domains and three criteria, and human validation showed agreement with human judgment rising from 73% to 85% compared to a naive baseline.
By Cheng Chang, Yining Mao, Peng Qi