GRADS is a graph-guided framework for generating synthetic data that explicitly controls difficulty by maintaining a graph of evidence and inference dependencies. It iteratively selects seeds, expands evidence, constructs question–answer pairs, and verifies them, ensuring that each generated task contains only indispensable inference steps. Experiments across security, software engineering, and scientific domains show that GRADS produces verified tasks with distinct difficulty levels, and training models on this data yields significant performance gains, including high solution-termination rates and reduced tool usage.
By Gaocheng Yang, Ruijie Tao, Kai Ye, Senhua Zhang, Zhengqin Zhu, Ziyang Li
Optimal Hydra is a transformer‑based heterogeneous federated learning framework designed for fine‑tuning large language models. It creates client‑specific sub‑networks, called Hydra‑nets, by selecting global layers and tailoring attention heads to each client’s local data distribution. Experiments on three datasets demonstrate significant performance gains over baseline methods while reducing local resource demands.
By Gwen Legate, Rahaf Aljundi, Eugene Belilovsky
OS-Helix is an end‑to‑end multimodal agent that unifies graphical user interface (GUI) actions and code execution into a single action space. It is trained on high‑quality hybrid data generated through application‑artifact composition and a decoupled instruction/evaluator pipeline, and further refined with online reinforcement learning using a step‑balanced GSPO algorithm. The resulting models achieve significant performance gains over hybrid baselines on the OSWorld‑Verified benchmark and demonstrate cross‑platform transfer to WindowsAgentArena.
By Bowen Yang, Kaiming Jin, Zichen Ding, Zhenyu Wu, Zehao Li, Zhoumianze Liu, Kanzhi Cheng, Fangzhi Xu, Yanan Sun, Qiushi Sun, Yu Qiao
EmbodiedMidtrain bridges the gap between Vision‑Language Models (VLMs) and Vision‑Language‑Action Models (VLAs) by first analyzing the distribution mismatch between their data. It introduces a lightweight proximity estimator that selects VLA‑aligned samples from a large VLM pool, creating the EmbodiedMidtrain‑1M dataset used to mid‑train VLMs before VLA fine‑tuning. Experiments on simulated and real‑world robot tasks show consistent performance gains across different VLM backbones and VLA action heads, with early training steps already benefiting from the improved initialization.
By Yiyang Du, Zhanqiu Guo, Haochen Zhang, Annabella Macaluso, Xin Ye, Liu Ren, Yonatan Bisk, Chenyan Xiong
The paper introduces SparK-SAR, a sparse convolutional masked pretraining framework that leverages ConvNeXt for high‑resolution Sentinel‑1 GRD SAR data. It presents three key contributions: a domain‑adapted ConvNeXt‑RS encoder with a reduced stem stride, a lightweight ConvNeXt‑512 decoder that cuts VRAM usage while matching UNet accuracy, and an analysis of FP16 instability and layer‑wise learning‑rate decay benefits. Experiments on four Copernicus‑Bench S1GRD tasks show that the 27.8 M‑parameter ConvNeXt‑RS encoder outperforms a larger ViT baseline on most tasks and demonstrates strong data efficiency, with classification saturating at 10–25 % of the SSL4EO‑S12 corpus.
By Yuki Natsume, Sang-Ho Yun, Chai Kiat Yeo
The paper introduces the Multi‑Tiered AI Accessibility Model (MAIAM), which categorizes AI access into three tiers based on purpose, safety, and capability rather than release format. Tier 1 (AI for Learning) is broadly accessible but capability‑bounded; Tier 2 (AI for Exploration) is available under registration and tighter controls as capability increases; Tier 3 (Full‑Scale Frontier AI) requires structured, recorded, and accountable access. The authors argue that accessibility should be proportional to capability and risk, and that safety limits alone are insufficient without a regulated tiered access regime.
By Chai K. Toh, Vasu Raj Jain
All-in-One Memory (AiO-Mem) is a reinforcement learning framework that unifies memory retrieval, writing, and stopping decisions into a single reward space. It compiles diverse histories into traceable multi‑view events, uses a unified policy actor to generate interaction trajectories, and employs a trajectory‑conditioned process reward model trained with supervised fine‑tuning and DPO. Experiments on long‑context memory benchmarks show AiO‑Mem consistently improves answer accuracy compared to static memory systems and other RL‑trained agents.
By Bingbing Wang, Jingjie Lin, Weiming Qiao, Jing Li, Ruifeng Xu
ARGUS is a neuro‑symbolic, agenda‑driven framework designed to improve the reliability of user simulators in interactive environments. It separates state tracking from natural language generation by using a dynamic agenda stack and symbolic operations to enforce goal progression and constraint‑aligned interactions. The authors distill the model into compact versions via evaluator‑guided SFT and online RL, and evaluate its behavioral fidelity with a new Composite Behavioral Score metric on $ au^3$‑Bench and MultiWOZ, showing significant performance gains.
By Hang Zheng, Nan Jiang, Zichen Zhu, Zixu Yang, Zhiyang Tang, Shuai Fan, Lu Chen, Xiaobao Wu, Kai Yu
QREF is a plug‑and‑play system that improves DeepSearch agents by refining search queries to match the current reasoning intent and filtering evidence to keep only what is relevant. The Query Refiner rewrites queries for complementary coverage across each search batch, while the Evidence Filter removes distracting retrieval results. Experiments show that QREF boosts DeepSearch performance on four benchmarks, achieving significant Pass@1 gains and dramatically reducing ineffective query batches without changing the agents’ architectures or retraining their reasoning policies.
By Yujia Chen, Yuhui Wang, Junzhe Wang, Shihan Dou, Di Liang, Yujiong Shen, Yuhao Zhou, Zhiheng Xi, Ming Zhang, Senjie Jin, Jiayi Chen, Dingwei Zhu, Yajie Yang, Yuan Hua, Chenyao Lu, Tao Gui, Qi Zhang, Xuanjing Huang
DeciCAD introduces a domain‑native harness for CAD agents that defines semantic CAD actions, decision‑oriented observations, environment feedback, and persistent state transitions. Using this harness, the authors develop a learning pipeline that extracts causal decision supervision from interaction trajectories and refines the policy through supervised fine‑tuning and geometry‑grounded online reinforcement learning. Across four CAD benchmarks, DeciCAD outperforms a general‑purpose coding harness, achieving higher F1 and IoU scores while reducing model‑output tokens by 45–88% and improving geometry availability from 66.5% to 73.5%.
By Yufeng Peng, Yixin Zhang, Ziyi Liu, Zhikun Xu, Jiarong Yang, Sijie Wang, Wenjie Feng, Qiyu Kang
Verdict-MQ is an open encoder that processes a single piece of state once and then answers multiple typed questions about that state, avoiding separate passes for each question. By using a structured attention mask, it ensures the state encoding is independent of question order and prevents questions from influencing one another. On the typed‑decisions benchmark, Verdict‑MQ outperforms a larger specialist model, achieving higher accuracy and better calibration, while on other benchmarks it shows mixed results compared to fine‑tuned baselines.
By Avdhesh Singh Chouhan, Jack Perry
The paper reviews how reinforcement learning (RL) can refine diffusion models (DMs) after pretraining, addressing limitations in reasoning, exploration, and alignment. It surveys RL-driven DMs across language, multimodal, robotic, and scientific domains, focusing on trajectory optimization and reward engineering. Experiments highlight the benefits of various RL post‑training strategies and identify key bottlenecks—reward fragility, exploration‑exploitation trade‑offs, and training instability—while proposing a roadmap for future research and a public repository for updates.
By Zhiyuan Ma, Jiabao Wei, Tianyu Wang, Hanqing Wang, Jiadong Pan, Haowen Xu, Liangliang Zhao, Ruixun Liu, Yuhan Wang, Yuwei Zhang, Liang Feng, Junhao Xiao, Kaiyan Zhang, Xuekai Zhu, Daizong Liu, Biqing Qi, Yihao Liu, Ganqu Cui, Ning Ding, Linfeng Zhang, Jianjun Li, Kun He, Xiang Bai, Bowen Zhou
The paper introduces HOW TO TRAIN YOUR LITTLE DRAGON, a method for teaching a small language model to generate and refine GPU optimization scripts. Using a compact, lossless representation of compiler traces, the authors train the model on 108,594 examples derived from TVM MetaSchedule, first with supervised fine‑tuning and then with reinforcement learning that rewards correctness and performance. After training, an 8‑billion‑parameter model achieves over 91% correctness in both construction and refinement tasks and outperforms several much larger LLMs on kernel performance.
By Annabelle Sujun Tang, Wei Tang, Songyao Jiang, Tianhui Li, Shengzhe Xu, Jiin Woo, Kaan Ozkara, Youngsuk Park, Zhen Jia, Hadi Esmaeilzadeh
The paper introduces a Bellman-consistent trajectory reinforcement learning approach for large language model (LLM) arithmetic reasoning, called Trajectory Bellman Residual Minimization (TBRM). By converting sparse terminal rewards into dense token-level signals and employing a Reinforce Leave-One-Out (RLOO) strategy to reduce gradient variance, the method improves learning stability. On the Countdown task, TBRM-RLOO achieves Pass@1 scores of 0.55–0.64 and a state-of-the-art Pass@16 accuracy of 76.0–84.0%, surpassing several baseline methods.
By Lianfa Li
The paper introduces an open midtraining recipe that primes base models for agentic post‑training, filling a gap between pretraining and full fine‑tuning. It trains on 256K context with hundreds of billions of tokens drawn from natural software‑engineering data and a small share of agentic traces. Applying this midtraining to Nemotron 3.5 Lightning and Super models yields measurable gains on SWE‑bench and DeepSWE benchmarks after identical full SFT, and improves robustness across agent harnesses.
By Soumye Singhal, Joseph Jennings, Ka Shun Shum, Alexander Bukharin, Khushi Bhardwaj, Zirui Liu, Venkat Srinivasan, Jonathan Li, Yash Jain, Kezhi Kong, Markus Kliegl, Olivier Delalleau, Sanjeev Satheesh, Mostofa Patwary
The paper introduces MaLoW, a streaming memory framework that internalizes long‑term video history into evolving lightweight parameter updates using a recurrent HyperNetwork and LoRA weights. MaLoW keeps current observations as input while carrying historical context forward without re‑visiting the full video stream. The authors also present MaLoW‑Instruct, a large streaming video dataset, and show that MaLoW outperforms prior memory methods on several benchmarks and improves action success in a vision‑language‑action setting.
By Zhuoming Liu, Jinhong Lin, Lin Zhang, Yiwu Zhong, Yin Li
The paper introduces Subspace Tuning (SST), a fine‑tuning technique that restricts weight updates to the bottom singular subspace of model parameters, thereby protecting the principal directions that are most prone to catastrophic forgetting. SST is optimizer‑agnostic, works with existing fine‑tuning recipes such as SFT and self‑distillation, and includes dense gradient estimation and a refresh mechanism to enhance learning. Experiments show that SST can preserve up to 99 % of the weight subspace, significantly reduce KL drift, and nearly eliminate forgetting in large‑scale language models while maintaining strong target‑task performance.
By Arnav Chavan, Kusha Sareen, Rishabh Tiwari, Abhranil Chandra, Johan Obando-Ceron, Kurt Keutzer, Scott Niekum, Inderjit S Dhillon, Devvrit Khatri
PEDBench is a new benchmark for phonetic error detection (PED) that includes natural recordings of children’s speech errors, scripted error productions by actors, and synthetic error productions generated via phone‑conditioned text‑to‑speech. The benchmark provides high‑quality phonetic annotations verified by trained annotators. Evaluation of 13 state‑of‑the‑art speech models shows that they struggle to reliably detect phonetic errors, even in controlled conditions, and often misclassify correct phonemes as errors. Fine‑tuned speech‑language models perform better on word‑level PED, but a sim‑to‑real domain gap persists.
By Zhonghao Shi, Zackary Rackauckas, Jasmine Yip, Savannah Halpern, Simon Campos, Lauren Brace, Emma Yang, Tiantian Feng, Xuan Shi, Shrikanth Narayanan, Teresa Satterfield, Ying Xu
SurgStreamer is a system that proactively monitors surgical video streams by separating procedure‑specific perception—mapping a 1‑FPS video to a structured state of tools, actions, and phases—from procedure‑agnostic reasoning that evaluates formal monitoring conditions over this state and an event memory. In a sealed test set of 12 laparoscopic cholecystectomies, SurgStreamer achieved a macro event‑F1 of 0.598 over full procedures and 0.677 on 75‑second windows, outperforming streaming vision‑language models. Re‑training only the perception component improved held‑out performance from 0.239 to 0.470, and a controller trained on cholecystectomy data performed comparably to an in‑domain controller on pituitary surgery, highlighting perception under distribution shift as the main limitation.
By Bowen Xiang, Jon S. Heiselman, Michael I. Miga
The paper investigates how fine‑tuned language agents can selectively withdraw actions when evidence changes or authority is revoked. Using a synthetic NAQD‑Env environment, the authors compare an original curriculum with an enriched one across multiple system‑prompt framings and evaluate four LoRA checkpoints of Qwen2.5‑3B‑Instruct against instruction‑tuned baselines. Results show that fine‑tuned checkpoints achieve higher decision agreement and withdrawal recall, reduce incorrect withdrawals, and narrow prohibited combinations, though they also omit valid event reports and exhibit mixed preservation of authorized actions.
By Mohamed Abouzahra