The paper introduces Diffusion LAIR, a listwise preference optimization technique that leverages continuous reward scores instead of binary pairwise comparisons to align text‑to‑image diffusion models. LAIR transforms reward scores into centered advantage weights and optimizes an advantage‑weighted regression objective on an implicit reward defined by denoising‑loss improvement over a reference model, with a quadratic penalty to regulate reward magnitude. Experiments demonstrate that Diffusion LAIR surpasses strong baseline methods on SD1.5 and SDXL across generation, compositional, and editing tasks.
By Austin Wang, Jiaqi Han, Stefano Ermon, Yisong Yue
The paper introduces Patterns of Past Rewards (PPR), a lightweight, algorithm‑agnostic online change‑point detector for cooperative multi‑agent reinforcement learning. PPR smooths agents’ return streams, highlights recent changes, and applies a statistical drift detector to flag significant shifts. Experiments in a custom Speaker‑Listener environment show that PPR balances detection speed and alarm stability, outperforming both a smoothed‑return baseline and a raw‑return detector.
By Fatemeh Saberi Khomami, Julita Vassileva
The paper introduces a reinforcement learning method to improve the quality of feedback generated by large language models for creative writers. By training with group relative policy optimization and a multi‑component reward that emphasizes tailored, actionable, and critical‑issue‑focused feedback, the authors demonstrate that their approach outperforms existing LLMs and baselines on three story corpora. The study shows that actionable suggestions are the key factor driving constructive feedback.
By Maja Stahl, Timon Ziegenbein, Henning Wachsmuth
PerfReasoning is a new benchmark that tests large language models (LLMs) on their ability to reason about hardware performance and generate analytical performance‑model code. The benchmark presents workloads, architectures, and mapping specifications, asking models to compare mappings and predict off‑chip traffic and buffer requirements. While the best closed‑source models achieve over 90% accuracy on reasoning‑based Q&A and the top open‑weight model scores 82.4%, constructing full performance models remains difficult, with most models scoring below 15% and significant variability across runs. Task‑specific reinforcement learning can improve a 4B model’s mapping‑reasoning accuracy by 15.7 points, but feedback‑free self‑revision prompting is not reliably effective.
By Dan Zhao, Karthikeyan Sankaralingam, Christos Kozyrakis, Qijing Huang
The paper reports that in on‑policy distillation for large language models, reasoning performance can be improved by supervising only a tiny fraction of generated tokens—sometimes just one or two tokens per reasoning trajectory, about 0.05% of all tokens. This sparse supervision consistently matches or exceeds full‑token training across nine teacher‑student setups on mathematical reasoning, and is also validated on coding reasoning, Llama models, and PPO‑based reinforcement learning with verifiable reward. The findings suggest that effective post‑training does not require token‑intensive supervision and may align more closely with natural learning processes that focus on critical reasoning steps.
By Zhishuai Liu, Xingzi Xu, Mehmet Saygin Seyfioglu, Pan Xu, Karim Bouyarmane
SQL-Zero is a self‑playing framework that trains a Text‑to‑SQL agent without any human‑annotated natural‑language/SQL pairs. It uses a challenger and a solver, both based on the same large language model, to generate SQL queries that are only validated by execution against the database. On the BIRD dataset, self‑play improves performance over a zero‑shot baseline by 6.6 to 7.3 points and shows promising transfer to unseen Spider databases, especially at 3B parameter scale.
By Daniel Machado Pedrozo, Julia Soares Dollis, Bryan Lincoln Marques de Oliveira, Vinicius Alboneti Aguiar, S\'avio Salvarino Teles de Oliveira, Telma Woerle de Lima Soares
Artificial Intelligence now underpins investment workflows from data and prediction to execution and tool use, yet its technical prowess does not automatically translate into profitability. A comprehensive review of public research up to 31 August 2026 across equities, ETFs, crypto spot, perpetual futures, and on‑chain markets shows real progress in prediction, text processing, portfolio design, and workflow integration, but evidence for durable net performance remains thin. The study highlights that factors such as temporal contamination, survivorship bias, weak benchmarks, implementation costs, and venue mechanics can erode alpha, and no single AI architecture has proven to deliver persistent, cross‑regime, capacity‑aware net alpha.
"whyItMatters":"The findings underscore that while AI advances are evident, investors must rigorously test and govern AI systems to avoid overestimating their profitability potential."
By Linsen Zhu, Mengqing Cai
The paper presents a Bayesian framework that unifies several large‑language‑model training and evaluation paradigms—supervised fine‑tuning (SFT), few‑shot in‑context learning (ICL), and KL‑regularized reinforcement learning (RLHF/RLVR). It shows that each method can be viewed as a two‑step process: first constructing a Bayes or Gibbs posterior over outputs or actions using a prior and a utility signal, then approximating this posterior via a forward‑KL projection onto a parametric family. The authors formalize ICL and SFT as amortized weight projections, and demonstrate that reward‑weighted SFT, reward‑weighted ICL, and advantage‑weighted SFT are all special cases of forward‑KL projection onto reward‑induced posteriors, while also outlining where these equivalences hold and where they break down.
By Junxin Fan
The paper introduces GUT, a method that uses directed acyclic graphs to represent all possible reasoning branches of Large Language Models (LLMs). It comprises two modules: GUT-Q, which quantifies reasoning uncertainty by approximating graph complexity, and GUT-O, which reduces uncertainty through reinforcement learning that rewards lower uncertainty. Experiments on four LLMs across five datasets demonstrate GUT’s effectiveness in measuring and mitigating reasoning uncertainty.
By Shuang Liang, Xin-Yu Hu, Xiang-Jun Ou, Shao-Qun Zhang
REFINE is a framework that refines medical concept representations by creating patient‑specific temporal graphs from a global text‑attributed knowledge graph. It uses a reinforcement learning policy to allocate a personalized graph expansion budget for each observed code, then processes the resulting graph with a heterogeneous GNN and a frozen LLM that refines representations via graph‑aware soft prompts. Experiments on MIMIC‑III and MIMIC‑IV demonstrate that REFINE consistently improves various EHR prediction backbones, surpasses strong baselines, and shows robust gains across ablation studies, KG selection, and data insufficiency scenarios.
By Mohsen Nayebi Kerdabadi, Arya Hadizadeh Moghaddam, Dongjie Wang, Zijun Yao
Persistent Teacher Anchoring (PTA) is a method that extends on‑policy knowledge distillation by ensuring that a teacher verifies entire turns before any tool calls are executed. PTA builds on chunk‑level verification with an added turn‑level commitment, treating verified chunks as atomic units and introducing persistent lookahead to keep rollout capacity full. Experiments on Search‑R1 and DeepEyes show that PTA improves macro best@4 by 2.5–2.8 points over standard OPKD and boosts throughput by 24%.
By Hyun Bin Park (Sogang University), Kyungho Song (University of Michigan, Ann Arbor), Sangmin Lee (Sogang University), Du-Seong Chang (Sogang University)
RL-VLA$^3$ is a fully asynchronous distributed reinforcement learning framework designed for Vision‑Language‑Action (VLA) model training. It allows fine‑grained asynchronous interaction between simulation, inference, and training via dynamic batching schedulers and flexible environment sharding, addressing the variable, resource‑intensive latencies of physical simulators. Experiments across multiple simulation backends, VLA architectures, and RL algorithms show throughput gains of up to 85.2% over synchronous baselines while preserving sample efficiency, and the system scales from 8 to 256 GPUs.
By Haoran Sun, Yongjian Guo, Zhong Guan, Shuai Di, Xiaodong Bai, Jing Long, Tianyun Zhao, Mingxi Luo, Hongke Zhao, Likang Wu, Xiaotie Deng, Xu Chu, Xi Xiao, Sheng Wen, Yicheng Gong, Junwu Xiong
The paper explores how large language models (LLMs) can be trained for small-molecule drug design by using synthetic tasks that are cheaper to evaluate. By employing a curriculum that gradually increases task difficulty, the authors demonstrate that LLMs can learn design strategies that outperform larger models on structure-based lead optimization. This approach shows that scaling post‑training with synthetic tasks can effectively adapt LLMs to high‑cost experimental scenarios that are otherwise infeasible to train on directly.
By Frank Hu, Shriram Chennakesavalu, Zichen Wang, Patricia Suriana, Bodhi Vani, Kirill Shmilovich, Kangway Chuang, Colin Grambow
The paper introduces ROBORMBENCH, a benchmark comprising 2,390 real‑robot trajectories, 21,673 verified paraphrases, and ground‑truth progress labels, to evaluate paraphrase robustness in vision‑language reward models (VLMs). It demonstrates that current VLMs often give different rewards for semantically equivalent goal descriptions, sometimes flipping a robot’s outcome from failure to success. The study finds that this instability is widespread, worsens with more divergent rewrites, and is not mitigated by model scale or explicit reasoning, though dedicated reward models trained with trajectory‑grounded supervision show greater stability.
By Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Yoonjun Cho, Dongjae Jeon, Bumjun Kim, Jean Oh, Youngjae Yu, Albert No
The paper introduces a reinforcement learning post‑training scheme that trains robot world models on their own autoregressive rollouts, using a contrastive RL objective adapted from diffusion models. It also proposes a training protocol that compares multiple variable‑length futures, a multi‑view visual fidelity reward, and demonstrates state‑of‑the‑art rollout fidelity on the DROID dataset, outperforming baselines on LPIPS, SSIM, and human preference tests.
By Jai Bardhan, Patrik Drozdik, Josef Sivic, Vladimir Petrik
The paper introduces Objective-aware Trajectory Credit Assignment (OTCA), a framework that refines reinforcement learning for diffusion-based visual generation. OTCA decomposes credit across denoising steps and allocates multiple reward signals adaptively, addressing the coarse, uniform reward assignment of existing GRPO pipelines. Experiments demonstrate that OTCA consistently enhances image and video generation quality across various metrics.
By Rui Li, Ke Hao, Yuanzhi Liang, Haibin Huang, Chi Zhang, Yun Gu, Xuelong Li
The paper introduces Iris-mini and Iris-pro, two search agents trained at 35B and 397B parameter scales. They use a novel data pipeline that constructs reverse‑engineered multi‑hop queries from web hyperlinks, filters trajectories, and alternates supervised fine‑tuning with reinforcement learning in a process called SFT‑RL climbing. Evaluations on several benchmarks show that, with inference‑time context management, the agents achieve the best open‑source results in their parameter ranges.
By Ziyuan Liu, Hengqi Liu, Zichuan Wang, Yang Qin, Jiachen Liang, Xu Chu, Shaowei Chen, Yuantao Gu, Mu Chuan
The paper investigates the discrepancy between strong agents and their underlying weak world models in Atari Pong by reproducing five visual world-model agents and evaluating their frozen models. Closed‑loop rollouts reveal visual and dynamical failures such as ball disappearance and incorrect motion, while zero‑shot model‑based RL policies trained entirely within the frozen models perform poorly compared to the original agents. To address these issues, the authors introduce Concept‑Guided Spatial Regularization (CGSReg), an auxiliary loss focused on task‑critical ball regions, which improves both pixel‑space zero‑shot MBRL performance and closed‑loop rollouts for several agents.
By Yukuan Lu, Zaishuo Xia, Weyl Lu, Yubei Chen
The paper investigates how the diversity of solutions produced by large language models (LLMs) for a single problem correlates with their problem‑solving performance. It finds that higher solution divergence is linked to better outcomes across various models and proposes using this metric to enhance supervised fine‑tuning and reinforcement learning. Experiments on three problem domains show that incorporating solution divergence consistently raises success rates, indicating its potential as a simple yet effective tool for LLM training and evaluation.
By Hang Li, Kaiqi Yang, Yucheng Chu, Hui Liu, Jiliang Tang
The paper introduces Reflection-Aware GRPO (RA‑GRPO), a reinforcement‑learning framework that aligns diffusion generative models with human preferences. It uses Diffusion Reflection to correct intermediate sampling paths by reversing the diffusion process, and Counterfactual Path Synthesis to embed these corrected trajectories into the policy, avoiding extra inference cost. Experiments on text‑to‑image and text‑to‑video models show RA‑GRPO outperforms existing methods, reducing reward hacking and improving generalization while remaining architecture‑agnostic.
By Junlong Wu, Jiuzhou Lin, Jia Sun, Boheng Zhang, Huaiqing Wang, Dewen Fan, Houde Liu, Qianqian Gan, Fan Yang, Tingting Gao