arXiv:2606. 28281v1 Announce Type: cross Abstract: PAC-Bayesian bounds provide finite-sample guarantees for data-dependent randomized predictors, but applying them to learning-based control is difficult because the natural objective is a quadratic trajectory cost.
The paper introduces a framework for intrinsic‑extrinsic coupling in learning dynamics, defining it via a continuation‑conditioned value of a constrained learning‑state intervention and observation‑relative fibers. It presents an executable finite‑frame classifier‑head that protects current logits while repairing historical margins, and distinguishes local admissibility, intervention value, and complete‑policy performance. Experiments on CLINC‑derived class‑incremental tasks, output distillation with RoBERTa, and SGDW dynamics demonstrate that coupling can produce both positive and negative interactions, and that coordinated content controls can match or exceed development gains while guided allocation reduces cross‑entropy loss compared to standard replay.
The paper introduces a framework for intrinsic‑extrinsic coupling in learning dynamics, where a learner’s current observations do not solely dictate its future training responses. It formalizes this coupling through a continuation‑conditioned value of a constrained learning‑state intervention and employs an executable finite‑frame classifier‑head to protect current logits while adjusting historical margins. Experiments across CLINC‑derived class‑incremental settings, output distillation with RoBERTa, and SGDW dynamics reveal that coupling can produce both positive and negative interactions, and that coordinated interventions can match or exceed development gains while reducing cross‑entropy loss compared to standard replay.
arXiv:2606. 16923v1 Announce Type: new Abstract: Simulation-based inference (SBI) of latent parameters is often hindered by simulator misspecification, the mismatch between simulated and real-world observations caused by inherent modeling simplifications.
The paper introduces Trust Region Q Adjoint Matching (TRQAM), an off‑policy fine‑tuning algorithm that stabilizes reinforcement learning of pretrained flow policies by adaptively controlling the path‑space KL divergence between the fine‑tuned and pretrained policies. TRQAM achieves this through projected dual descent, which adjusts a trust‑region parameter λ during sampling and ensures the KL penalty is exactly weighted in the stochastic optimal control objective. Experiments on 50 OGBench tasks show TRQAM consistently outperforms prior methods, achieving a 68% success rate in offline RL versus 46% for the strongest baseline.
The paper introduces COPC, a Coupled Off-Policy Correction method for asynchronous reinforcement learning of large language models. COPC coordinates policy-side and advantage-side corrections by combining token-level ratio masking with two-sided clipped-ratio weighting of TD residuals, addressing both policy mismatch and advantage staleness. Experiments show COPC outperforms existing asynchronous baselines on tool-integrated mathematical reasoning and search tasks, while maintaining training stability and minimal overhead.
The paper introduces Nous, a framework that separates learning, calibration, and revision certification for belief‑based agent memory. It shows that learning and certifying useful memory decisions can require quadratically fewer records than source calibration, and provides finite‑sample certificates for policy improvement without needing to recover source reliability. Experiments on MiniGrid environments demonstrate that the new certificate reliably accepts improvements over incumbents, outperforming earlier methods.
BCPPO is a new variant of Proximal Policy Optimization that uses Bachelier-inspired cost‑prediction networks to generate a smooth penalty based on disagreement among critics. The method keeps temporal‑difference learning unchanged, applies a saturation‑aware controller to manage cost penalties, and deploys only the policy network. Across extensive experiments, BCPPO outperforms comparators in achieving higher mean returns while maintaining lower or comparable CVaR in all tested tasks.
arXiv:2610.02828v1 Announce Type: new Abstract: Adaptive LLM reinforcement-learning post-training changes multiple training actuators online, including rollout temperature, group size, clipping, KL r...
arXiv:2606. 08779v1 Announce Type: new Abstract: Reinforcement Learning (RL) has emerged as a pivotal post-training paradigm, yet it frequently suffers from unpredictable sub-optimum performance or even training collapses.
arXiv:2610.20532v1 Announce Type: cross Abstract: A multimodal agent must decide whether visual evidence is worth acquiring, which candidate or resolution to query, and when further interaction can n...
arXiv:2512. 23075v5 Announce Type: replace-cross Abstract: Policy gradient methods for Large Language Models optimize a policy $\pi_\theta$ via a surrogate objective computed from samples of a rollout policy $\pi_{\text{roll}}$.