arXiv AI
Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Oct 9
Agentic Critical Training
Agentic Critical Training (ACT) is a reinforcement learning approach that trains language‑model agents to judge actions directly by pairing expert actions with alternatives and rewarding correct selection. Unlike imitation learning or self‑reflection, ACT does not require reference rationales and can be reused across model sizes. Experiments on Qwen3‑8B and Olmo‑3‑7B‑Instruct show that ACT improves performance on ALFWorld, WebShop, ScienceWorld, and outperforms supervised reflection baselines, with significant gains over IL and IL→RL baselines.
arXiv AI
Sep 30
AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
arXiv:2609.38142v1 Announce Type: new Abstract: A small trainable advisor can steer a frozen language-model executor using natural-language advice. In addition to learning from task rewards, the advi...
arXiv AI
Oct 1
Trust the Critic More
The paper introduces Actor‑Critic with Action Chunking (AC2), a method that assigns credit to short action chunks instead of entire trajectories, enabling policy updates without waiting for terminal rewards. AC2 employs local readiness, reference solutions, and 10k‑token chunks to make critic‑based credit assignment reliable. Experiments on Qwen3‑4B with FineProofs‑RL show AC2 surpasses GRPO’s peak validation score while using 2.5× fewer decoding FLOPs and fewer training steps.
arXiv AI
Aug 25
Let Credit Follow Computation: Architecture-Aware Credit Transport for Large Language Model Reinforcement Learning
arXiv:2608.21501v1 Announce Type: new Abstract: Credit assignment in large-language-model reinforcement learning (LLM RL) can be separated into three objects: evidence about success, a transport oper...
arXiv AI
Aug 26
PROOF-Gen: From Optimized Data to Better Distillation
PROOF-Gen is a method that improves distillation of tool‑calling models by recovering successful trajectories from teacher failures. It uses per‑scenario prompt optimization to generate corrective guidance that steers the teacher to a passing trajectory, then removes this guidance before training so the student learns from clean demonstrations. On τ2‑bench, PROOF-Gen recovers 93% of failed scenarios, boosting Qwen3‑4B‑Instruct‑2507’s Pass^1 from 0.132 to 0.529 and improving Gemma 4 E4B‑it by 7.2pp on BFCL v4 multi‑turn, while also raising deployed on‑device model performance by up to 5.0pp across response‑quality metrics.
arXiv AI
Jul 7
Weak-to-Strong Generalization via Direct On-Policy Distillation
arXiv:2607. 05394v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is a powerful recipe for improving language-model reasoning, but it is expensive to repeat on every new strong model because the target model must generate many rollouts during training.