Reinforcement learning

CODESKILL: Learning Self-Evolving Skills for Coding Agents

CODESKILL is an LLM-based framework that learns to extract, evolve, and maintain procedural skills from coding-agent trajectories. It treats skill extraction and skill-bank management as a learnable policy trained with reinforcement learning, using a hybrid reward combining rubric-based skill quality and verifiable execution feedback. Experiments on EnvBench, SWE-Bench Verified, and Terminal-Bench 2 demonstrate that CODESKILL raises average pass rates by 11.03 over a no-skill baseline and by 5.10 over the strongest prompt-based or memory baseline while keeping a compact skill bank.

Topology-Driven Anti-Entanglement Control for Soft Robots

The paper introduces a topology-driven Multi-Agent Reinforcement Learning (TD-MARL) framework designed to coordinate soft robots in precision manufacturing tasks, specifically to prevent entanglement during unwinding operations in highly constrained environments. By employing centralized learning with a shared topological state, the approach improves observability and training stability, while distributed execution reduces communication demands and enhances system reliability. Simulation results demonstrate that TD-MARL outperforms current deep reinforcement learning methods in convergence speed and anti-winding effectiveness.

StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction

StraTA introduces Strategic Trajectory Abstraction, a framework that samples a compact strategy from the initial task state and conditions subsequent actions on that strategy, training strategy generation and action execution jointly with a hierarchical GRPO-style rollout design. The method enhances exploration and credit assignment over long horizons by incorporating diverse strategy rollouts and critical self-judgment. Experiments on ALFWorld, WebShop, and SciWorld demonstrate that StraTA consistently improves sample efficiency and final performance, achieving success rates of 93.1% on ALFWorld, 84.2% on WebShop, and a 63.5% overall score on SciWorld, surpassing frontier closed‑source models.

Genetic Algorithms with Optimization Guided Operators

The paper introduces a new framework for genetic algorithms where mutation and recombination operators are guided by machine‑learning optimization rather than random changes. It shows that such operators can improve objective values but at higher computational cost, and demonstrates three key phenomena: the necessity of solution‑pool diversity for parity learning, the simultaneous need for generation, mutation, and recombination to achieve near‑optimal solutions, and a phase transition in Gaussian settings where positive drift yields exponential speedup.

A Progressive Design Study of Visual Encoders and Value Estimation for Replay-Free Parallelized Q-Learning

The paper presents a three‑phase study of visual encoders and value‑estimation methods for replay‑free parallelized Q‑learning within the Parallelized Q‑Network framework. It compares eight convolutional encoder topologies, adds Hadamax‑style multiplicative interactions and pooling, and evaluates categorical‑dueling, ensemble‑dueling, and categorical ensemble‑dueling value‑estimation configurations. The resulting architecture, Aftab, outperforms a baseline PQN on Atari‑57 and shows improved performance on Procgen Hard, demonstrating that visual topology, multiplicative representation, and downstream value‑estimation design significantly influence replay‑free Q‑learning when considered alongside computational complexity.

SUN: Agentic Robot Policy Learning with Persistent Task Programs

The paper introduces SUN (Semantically UNified) Programs, typed executables that translate grounded relations into optimal control objectives, satisfaction predicates, and learning rewards. Using the Kuafu harness, a foundation model orchestrates scene preparation, verification, residual reinforcement learning, and data generation, repairing candidate programs and calibrating reward weights. Across nine multi‑stage manipulation tasks, Kuafu achieves an 82.03% success rate, outperforms learned baselines, generates demonstrations 10.57× faster than human teleoperation, and transfers zero‑shot to physical Franka and Kinova robots.

Adaptive Random Matrices in Gaussian Bandits: Spectral Universality and Selection-Induced Outliers

arXiv:2609.30321v1 Announce Type: new Abstract: Adaptive arm selection changes the distribution of the observations collected by a bandit algorithm, but it need not change their limiting empirical sp...

Foundations of Reinforcement Learning and Interactive Decision Making

arXiv:2312.16730v2 Announce Type: replace-cross Abstract: Interactive decision making is the problem of learning to act well in an unknown environment, using the data that one's own actions generate...

Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs

Audio large language models (Audio LLMs) often fail to transcribe English‑Mandarin code‑switching speech, exhibiting language omission, translation‑instead‑of‑transcription, and hallucination. By applying Direct Preference Optimization (DPO) with 100 K preference pairs, the models learn to preserve mixed‑language content rather than translate, leading to significant reductions in mixed‑error rates (up to 89.6% in‑distribution). The study demonstrates that DPO can effectively align multilingual Audio LLMs for accurate code‑switching transcription.

MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimization

MVAgent is a multi‑agent pipeline for multi‑shot video generation that ensures consistent character appearance, stable spatial layout, and continuous character state across shots. The system uses typed conditioning inputs: a Spatial Grounding agent samples camera views, an Observer records shot endings into a continuity memory, a Transition agent builds action and spatial references for subsequent shots, and an Orchestrator composes these inputs into generator requests. Trained with agentic reinforcement learning (Trunk‑GDPO) while keeping the generator and judges frozen, MVAgent achieves the highest cross‑shot consistency and narrative‑planning quality on ViMax‑Bench and is preferred over the strongest agentic baseline in human evaluation.

Welcome RL Environments to the hub

2026 in LLMs (so far)

Simon Willison delivered a closing keynote at the WeAreDevelopers World Congress North America, outlining the key developments in large language models for 2026. He highlighted the release of Claude Opus 4.5 and GPT‑5.1 in November 2025, noting that their coding agents became reliable enough for daily use. Willison also discussed early benchmarks, such as generating an SVG of a pelican riding a bicycle, and mentioned a new GitHub repository called "Warelay" that emerged during the year.

Canopy: Exploiting Piecewise Smooth Tree Priors for Multi-Fidelity Bandits

CANOPY is a multi‑fidelity tree bandit algorithm that learns where a piecewise‑smooth prior holds instead of assuming global smoothness. It uses cheap random‑path probes to certify local aggregation bias and then focuses expensive leaf evaluations on cells where smoothness is violated. The method achieves provable fixed‑budget and regret guarantees that scale with the number of discontinuities, matching smooth‑tree rates when no violations exist and approaching structure‑blind search when violations are dense.

TabSieve: Explicit In-Table Evidence Selection for Tabular Prediction

TabSieve is a select‑then‑predict framework that explicitly chooses a small set of informative rows from a table as evidence before predicting a missing target. The authors build a large synthetic dataset, TabSieve‑SFT‑40K, and introduce a reinforcement learning method, TAB‑GRPO, to jointly optimize evidence selection and prediction. Experiments on 75 classification and 52 regression tables show consistent performance gains, with TabSieve improving classification by 2.92% and regression by 4.45% over the best baseline while enhancing robustness to noisy context.

CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models

CONSISTRE is a consistency‑aware framework for document‑level relation extraction that tackles contradictions in large language model predictions. It offers two tracks: an inference‑time track that refines black‑box LLM outputs through constraint‑aware prompting, verification, and self‑reflection, and a training‑time track that distills consistency knowledge into smaller open‑source models via supervised fine‑tuning and reinforcement learning. Experiments on DocRED show both tracks outperform baselines, with the inference‑time track matching competitive F1 scores and the training‑time track narrowing the performance gap to proprietary LLMs while reducing inference cost.

MQSS-Selector: RL-Guided Pass Selection for an MLIR Compilation Pipeline

The paper introduces MQSS-Selector, a reinforcement‑learning guided pass selection system for an MLIR compilation pipeline aimed at unified High Performance Computing‑Quantum Computing (HPCQC) infrastructures. It addresses the challenges of Noisy Intermediate‑Scale Quantum (NISQ) devices by integrating device selection, compiler‑pass optimization, and job queue scheduling into a single learning‑based framework. The selector can simultaneously optimize multiple objectives—fidelity, compilation time, and scheduling latency—while adapting to circuit characteristics and device conditions.

Rufus-Air: An Open LLM Post-Training Recipe

Rufus-Air is an open, reproducible post‑training recipe for the GLM‑4.5‑Air‑Base model, structured as a serial pipeline of eight stages: SFT, Reasoning RL, Coding RL, Instruction‑Following RL, General Agent, Coding Agent, Search Agent, and RLHF. The paper documents the data, reward design, infrastructure, stage order, and stagewise results required to reproduce the recipe, emphasizing that stages progress from basic to advanced capabilities and from hard, verifiable rewards to softer judge‑based signals. Key findings highlight the importance of diverse, high‑quality SFT, difficulty filtering, reward reliability for stage ordering, and the role of infrastructure and engineering choices.

Exact Bayes Regret and Asymptotic Optimality in High-Dimensional Gaussian Bandits

The paper analyzes Bayesian linear bandits with isotropic Gaussian parameters, independent Gaussian arms, and Gaussian reward noise when the time horizon scales with the dimension. It derives explicit limits for the normalized posterior uncertainty and parameter overlaps, yielding exact regret curves for several policies—including Thompson sampling, posterior‑mean greedy selection, and scaled‑covariance variants. The results show that posterior‑mean greedy selection achieves the optimal Bayes regret, while Thompson sampling incurs a strictly larger leading regret whose ratio to greedy lies between one and two, approaching two for long horizons.

RLVR landscapes for iterated multiplications can be benign: Insights from spin-glass theory

The paper investigates the optimization landscape of reinforcement learning with verifiable rewards (RLVR) on algorithmic tasks such as iterated group and quasigroup multiplication. By mapping entropy‑regularized RLVR with myopic tabular policies to a spin‑glass energy model, the authors prove that for a broad class of models with uncorrelated inputs the landscape is benign—there are no local minima that can trap training. They identify practical obstacles like diffusive barriers and gradient‑estimation errors, show these can be mitigated by choosing an appropriate entropy regulator, and demonstrate that a transformer trained from scratch with only last‑token rewards can learn an algorithmic chain of thought for iterated non‑Abelian group multiplications.

AdaPilot: Towards Scene-Adaptive Policy Learning for Cross-Generator Text-to-Image Quality Optimization

AdaPilot introduces a scene-adaptive, cross-generator policy for optimizing text-to-image generation quality. By framing multi-turn image generation as a Markov Decision Process and using reinforcement learning, it decouples the policy from specific generator internals, incorporates scene-aware and process-level rewards, and achieves superior quality and generalization compared to baselines. Experiments demonstrate that a single AdaPilot policy can transfer zero‑shot to unseen generators while consistently improving performance across all evaluated models.