arXiv:2610.28229v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) minimizes empirical risk over observed preference comparisons, implicitly assuming that the training preference...
arXiv:2605. 00155v3 Announce Type: replace Abstract: Reinforcement learning from human feedback (RLHF) is a central post-training tool for aligning large language models, but its training reward is only a learned proxy for true human utility.
arXiv:2505. 10892v2 Announce Type: replace Abstract: Post-training LLMs with RLHF and preference optimization methods (e.
arXiv:2504. 10796v4 Announce Type: replace-cross Abstract: Distributionally robust optimization (DRO) is widely used for decision-making under uncertainty, but its adversarial focus on worst-case loss can lead to overly conservative policies.
The paper introduces Diagnostic-Transport DRO (DT‑DRO), a method that calibrates the geometry of the ambiguity set in distributionally robust optimization using held‑out data. By applying the conditional probability integral transform to detect systematic probability misallocation, DT‑DRO adjusts both the ambiguity‑set center and ground cost through an outcome‑level transport, yielding a computationally tractable dual reformulation. The authors provide theoretical guarantees on ambiguity radii and decision‑risk bounds that improve as estimation errors shrink, and demonstrate that DT‑DRO can remove the robustness floor caused by model misspecification in synthetic and power‑outage experiments.
arXiv:2608. 13133v1 Announce Type: cross Abstract: Distributional shifts arise when the target deployment environment differs from the source environment that generated the training data.
The paper introduces a doubly robust framework for distributionally robust offline contextual pricing, addressing challenges of distributional shifts and continuous actions. It proposes a localized DR estimator that is computationally efficient and semiparametrically optimal, and a learning algorithm that exploits demand noise smoothness to achieve finite‑sample regret bounds of τ^{-σ/(2σ+1)} for smoothness orders s=1,2. Experiments across varying distribution shifts demonstrate the effectiveness of the proposed methods.
arXiv:2605.28165v2 Announce Type: replace Abstract: Robust learning methods such as distributionally robust optimization, vicinal risk minimization, Mixup, and label smoothing address different sourc...
Robust Nash Alignment introduces a game-theoretic framework that seeks a policy with a high worst-case win rate against both an adversarial competitor and any preference kernel within an ambiguity set around a nominal preference. The authors propose a four-player primal-dual proxy game and an optimistic mirror descent-ascent algorithm to efficiently optimize this robust objective, proving convergence guarantees and demonstrating improved performance in controlled tabular games and LLM alignment experiments.
arXiv:2509. 09371v2 Announce Type: replace-cross Abstract: Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions.
arXiv:2607. 09820v1 Announce Type: new Abstract: Predict-then-optimize systems usually compress uncertainty into a point forecast and then solve a downstream optimization problem as if the forecast were reliable.
The paper introduces Geometric Anchor Preference Optimization (GAPO), a method that replaces the static reference policy in Direct Preference Optimization with a dynamic, geometry-aware anchor—a small adversarial perturbation of the current policy. GAPO uses this anchor to adaptively reweight preference pairs based on local sensitivity, and defines an Anchor Gap that approximates worst‑case local margin degradation. Experiments show that GAPO improves robustness to noisy supervision while matching or surpassing existing LLM alignment and reasoning benchmarks.