Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.
arXiv:2610.07578v1 Announce Type: new Abstract: In cooperative Multi-Agent Reinforcement Learning (MARL), agents are often trained under concurrent participation, while in many tasks some agents act...
The paper introduces Personal-Agent Mediated Recommendation, a new paradigm where a personal LLM agent uses cross‑platform user history to adjust a platform’s recommendation ranking. It presents MediateRec, a benchmark for evaluating this mediation, and proposes Personal Attribution Mediation Optimization (PAMO) to balance beneficial rescues against harmful overrides. Experiments show that PAMO improves over outcome‑only reinforcement learning, achieving a better rescue‑harm trade‑off on both synthetic and real cross‑platform tests.
The paper introduces LSC-DPO, a variant of Direct Preference Optimization that dynamically controls the learning signal to maintain sensitivity during training. By analyzing the logistic DPO loss geometrically, the authors identify the sigmoid factor as a key learning signal and propose a log‑space framework for stable target‑regime tracking. Experiments on AlpacaEval 2, MT‑Bench, and Anthropic‑HH demonstrate that LSC‑DPO outperforms standard DPO and other preference‑optimization baselines, and a signal‑budget compensation rule further reduces variability across different coefficient initializations.
The paper proposes a method that mimics human cognition to train reinforcement learning agents to generate explainable strategies and policies as executable procedures derived from sequences of gameplay actions. It demonstrates automatic learning of such programs for chess and a grid‑based environment, showing that the resulting strategies produce effective actions and can be learned from gameplay data.
The paper presents a symbolic sub‑policy model for chess that incorporates domain knowledge to improve interpretability. By adapting patterns learned through inductive logic programming (PAL), the authors evaluate the model using a divergence metric against a random baseline and show that it can suggest moves comparable to a human beginner. They also propose a computational evaluation scheme that augments an existing engine with their model.
The paper introduces DHCG, a framework that dynamically constructs hierarchical collaboration graphs for large language model–based multi‑agent systems. DHCG coordinates Planner, Worker, and Generator modules to adaptively determine the composition and scale of agents during execution, guided by feedback and action‑aware preference optimization. Experiments on code generation, mathematical reasoning, and domain‑specific tasks show DHCG surpasses static and dynamic baselines, improving performance by 2.77–8.02 points and achieving a 13.06‑point gain over single‑agent baselines.
arXiv:2610.08683v1 Announce Type: new Abstract: Full-duplex speech models are trained to converse with a person, but they are increasingly made to converse with each other, in self-play data generati...
arXiv:2610.08778v1 Announce Type: new Abstract: Large language models (LLMs) have become increasingly capable problem solvers, but being able to solve a problem is not the same as being able to teach...
arXiv:2610.06993v1 Announce Type: cross Abstract: Evolution Strategies (ES) enable memory efficient full parameter fine-tuning of large language models (LLMs) using only forward computation. However,...
arXiv:2610.07177v1 Announce Type: cross Abstract: An open contrastive decision model is near chance as a judge on the hard public benchmarks: Contrastive-LM/CLM-v0.1-8B scores between 0.351 (best- of...
arXiv:2610.07491v1 Announce Type: cross Abstract: When multiple agents share a cost budget, a common Lagrange multiplier can enforce the aggregate constraint but does not determine how its penalty sh...
arXiv:2610.07681v1 Announce Type: cross Abstract: Dexterous manipulation poses a challenging high-dimensional optimization problem, as useful behaviors require coordinated motion across many hand joi...
arXiv:2610.07730v1 Announce Type: cross Abstract: Typed decision models answer a declared question without generating text: a decision head returns a probability for each of the declared options in a...
arXiv:2610.07731v1 Announce Type: cross Abstract: Dense retrieval models are typically trained with contrastive objectives that learn effective representations but do not directly optimize retrieval...
NeMo-DCR is a delta‑compressed refit method that sends only the changes in a trillion‑parameter policy model, achieving bit‑exact synchronization across AWS regions. By using fixed affine mappings, XOR masks, and a relay‑tree object storage, it avoids full checkpoint transfers and cross‑cluster collectives, reducing refit times from 87.5 min to 150 s for a 1 T model at 3% change rate. The approach scales to 30 B–1 T models, delivering 12–40× speedups over traditional full‑checkpoint refits.
arXiv:2610.08659v1 Announce Type: cross Abstract: Model merging provides a training-free way to transfer reasoning capabilities from language models to vision-language models (VLMs), but endpoint-bas...
arXiv:2610.08743v1 Announce Type: cross Abstract: Sequential recommenders typically use a fixed slate size even though the number of useful alternatives changes within a session. We propose Reinforce...
arXiv:2508.16821v2 Announce Type: replace Abstract: We introduce PuzzleJAX, a GPU-accelerated puzzle game engine and description language designed to support rapid benchmarking of tree search, reinfo...
arXiv:2605.10791v2 Announce Type: replace Abstract: Knowledge Graph Question Answering (KGQA) aims to answer user questions by reasoning over Knowledge Graphs (KGs). Recent methods use supervision de...
arXiv:2610.05370v2 Announce Type: replace Abstract: Generative reward models (GRMs) are important for LLM optimization. Unlike scalar reward models, GRMs generate natural-language critiques alongside...