Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2610.28474v1 Announce Type: new Abstract: People adapt how they talk to the person in front of them. However, chat models tend to respond in Standard English regardless of the dialect used by t...
arXiv:2605.20682v3 Announce Type: replace Abstract: Small defects in unfamiliar product categories can be difficult to distinguish from benign appearance variations in a global view. Additional reaso...
Navalia is a framework that advances cowork agents by combining verifiable task synthesis with long‑horizon post‑training. It selects supporting evidence from corporate financial reports, builds executable computation chains, and uses independent LLM review and programmatic checks to ensure evidence consistency and answer correctness. The verified tasks are then used for trajectory collection, supervised fine‑tuning, and reinforcement learning, yielding consistent performance gains over the base model across five financial and office‑task benchmarks.
The paper investigates how large language models develop brand preferences, noting that models often name market‑leading brands like Nike when asked for a running shoe brand but recommend niche brands such as Brooks. Across 102 models and 42 brand categories, the brand chosen for recommendation differs from the named brand in 26% of cases, and preference optimization (DPO) shifts these choices by 11–24 points. The study suggests that pretraining already links recommended brands to advice, supervised fine‑tuning preserves these preferences, and DPO can shift them even when the preferred brand is absent from the preference data, indicating that the judge models used in training play a key role.
The paper introduces a new class of Bayesian neural networks that place Dirichlet Process priors on the data-generating distribution instead of on network weights, aiming to better capture epistemic uncertainty. Using this framework, the authors develop DP-DQN, a reinforcement learning algorithm that maintains posterior distributions over value functions and performs efficient Thompson sampling. DP-DQN combines data-driven posterior reweighting via the Bayesian Bootstrap with generative prior hypotheses from the DP base measure, enabling deep exploration with a single neural network without requiring multi-network ensembles.
AdaDex is a goal‑conditioned reinforcement learning framework that transforms a short hand‑object trajectory—captured via mocap or in‑the‑wild RGB(D) video—into an adaptable dexterous skill. It trains a shared policy on two parallel environment populations: motion‑tracking environments that embed human priors and generalization environments that enforce robustness to noisy states and goals. The resulting policies achieve diverse, functional dexterous behaviors such as grasping, re‑posing, in‑hand reorientation, and dynamic throw‑and‑catch, outperforming baselines in generalization and demonstrating transfer from simulation to the real world.
The paper introduces a reinforcement learning framework that uses large language models to generate RTL circuits while optimizing for Pareto-optimality across power, performance, and area (PPA). It proposes a generation-based training pipeline and a Hypervolume Contribution reward to improve both the quality and diversity of the entire set of generated circuits. Experiments on the RTLLM v2 benchmark show significant gains in hypervolume and diversity compared to baseline and single-objective methods.
The paper investigates how optimizing the harness of language model agents—without changing model weights—affects performance. It distinguishes between gains that improve success on tasks already within reach (elicitation) and gains that broaden the range of solvable tasks (expansion). By comparing optimized harnesses to their pre‑optimization versions across static and dynamic tasks, the study finds that benefits can reverse at higher attempt budgets and that aggregate gains may mask shortcomings on harder tasks, highlighting the need for detailed diagnostics.
arXiv:2610.27895v1 Announce Type: cross Abstract: Direct preference optimisation (DPO) is widely used for language model post-training, but its performance depends strongly on the choice of its hyper...
arXiv:2610.27214v1 Announce Type: new Abstract: Estimating extreme risk in reinforcement learning becomes increasingly difficult as the confidence level increases, since only a small number of observ...
arXiv:2610.28010v1 Announce Type: new Abstract: The integration of Large Language Models and Reinforcement Learning offers a promising direction for complex sequential decision-making tasks, such as...
arXiv:2610.28136v1 Announce Type: cross Abstract: Gaussian process upper confidence bound (GP-UCB) has been widely studied, though its known regret upper bound is suboptimal. To obtain near-optimal r...
arXiv:2510.03811v2 Announce Type: replace Abstract: We develop an approach for the de novo synthesis of messenger RNA (mRNA) sequences encoding target proteins. This requires optimizing multiple biol...
arXiv:2607.08925v2 Announce Type: replace Abstract: Training agents directly on the task they will be deployed on often leads to the best policy performance, but training on physical robots can cause...
arXiv:2609.33848v2 Announce Type: replace Abstract: Large language model (LLM) agents increasingly undertake extreme-long (xlong) horizon tasks, where a single execution can span hours, hundreds of m...
arXiv:2610.05220v2 Announce Type: replace Abstract: Carousel interfaces allow a recommender system to directly observe how far a user has browsed. This signal distinguishes displayed but unclicked it...
arXiv:2509.14203v4 Announce Type: replace-cross Abstract: Learning and optimal control under robust Markov decision processes (MDPs) have received increasing attention, yet most existing theory, algo...
arXiv:2610.27237v1 Announce Type: new Abstract: We present a study on controlled automatic simplification of Russian texts using preference-based optimization algorithms with a focus on adapting text...
arXiv:2504.19024v2 Announce Type: replace Abstract: We propose a novel k-step return estimation method (called KETCHUP) for Reinforcement Learning(RL)-based knowledge distillation (KD) in text genera...
arXiv:2601.16400v2 Announce Type: replace Abstract: Real-world visual question answering (VQA) may be context-dependent: an image-question pair could be under-specified, such that the correct answer...