The paper introduces AnySearch, a reinforcement‑learning framework that trains a single policy to perform budget‑aware search for large language models under any budget constraint. The training proceeds in two phases: first, the agent learns with explicit budget state injection and structured reasoning prompts under linearly decaying budgets; second, the scaffold is removed and the agent adapts to randomly sampled budgets that match deployment conditions. The reward combines answer accuracy and budget efficiency, with adaptive weighting to emphasize efficiency for high‑accuracy queries and reduce it for low‑accuracy ones. Experiments on seven QA benchmarks demonstrate that AnySearch outperforms baselines across all budget scales, generalizes to unseen constraints, and improves tool productivity without excessive token overhead.
By Xiaowei Sun, Jin Li, Yili Hong, Yikun Fu, Yanghua Xiao
arXiv:2506.13502v4 Announce Type: replace
Abstract: Next-word prediction (NWP) trains language models against a single observed continuation, even though many contexts admit multiple plausible next w...
By Ming Shen, Zhikun Xu, Jacob Dineen, Xiao Ye, Ben Zhou
The paper introduces CRAFT, a data‑centric fine‑tuning approach that aligns small language models (SLMs) for pre‑hoc reasoning in AI‑native 6G radio access networks (RAN). By automatically generating verified (input, trace, label) triplets and fine‑tuning with low‑rank adaptation, CRAFT achieves high accuracy and F1 scores on TRACTOR and IC xApp datasets while avoiding parse failures that plague RL methods like GRPO. It also reduces energy consumption by 59% compared to GRPO baselines, offering a more sustainable path to auditable AI in 6G RAN.
By Pranshav Gajjar, Vijay K Shah
Group Adaptive Clipping Policy Optimization (GAPO) is a plug‑in modification to GRPO methods that adapts the importance‑sampling clipping boundary based on rollout advantage. By allowing rollouts with larger learning signals to receive proportionally greater update headroom, GAPO addresses the limitation of fixed clipping that suppresses rare but informative rollouts. Experiments on Qwen and Llama models show that GAPO consistently improves Pass@1 and Pass@k on math reasoning and coding benchmarks where base model pass rates are low.
By Sheng Jia, Xiao Wang, Shiva Prasad Kasiviswanathan, Rein Houthooft
Iterative GRPO is a batch‑online policy iteration framework that enables multi‑turn reinforcement learning for conversational agents without requiring an interactive user simulator. It alternates between learning a turn‑level Q‑function from logged returns (policy evaluation) and applying single‑turn GRPO against this Q‑function (policy improvement), thereby scoring candidate responses by their expected downstream return. The method is validated on six multi‑turn negotiation environments, demonstrating its practicality for real‑world deployment patterns.
By Daniel R. Jiang, Ankur Samanta, Yukai Yang, Jalaj Bhandari, R\'emi Munos, Tyler Lu
arXiv:2606.21811v2 Announce Type: replace-cross
Abstract: Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While...
By Shubham Gandhi, Yiqing Xie, Atharva Naik, Ruichen Zhu, Carolyn Rose
arXiv:2609.01188v1 Announce Type: new
Abstract: Large Language Models (LLMs) are revolutionizing digital communication by powering conversational agents deployed across domains such as customer servi...
By Rohan Kirti, Akash Ghosh, Aryan Vats, Niladri Ghosh, Shipra Shriparn, Roshni Ramnani, Anutosh Maitra, Sriparna Saha
arXiv:2609.01195v1 Announce Type: new
Abstract: Entity matching (EM) requires fine-grained contextual understanding and domain knowledge. Recent work shows that large language models (LLMs) can serve...
By Chaohui Guo, Michel Klein, Zhisheng Huang
arXiv:2509.25050v2 Announce Type: replace
Abstract: Reinforcement Learning (RL) has emerged as a central paradigm for advancing Large Language Models (LLMs), where both pre-training and RL post-train...
By Shuchen Xue, Chongjian Ge, Shilong Zhang, Yichen Li, Zhi-Ming Ma
The paper introduces Online Hyperparameter Optimization (OHPO), framing it as an infinitely many‑armed bandit problem over mixed and conditional search spaces. It proposes the IMABO framework, which couples any bandit policy with any oracle for proposing new configurations, and presents IMOSS—a restart‑free anytime policy with provable regret bounds. Experiments show that IMABO, combined with practical oracles such as TPE, an incumbent‑mutation oracle, and a pretrained tabular foundation model, outperforms random search across a range of settings from classical ML models to LLM‑based agents.
By Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine
CopyShield is a benchmark that compares three copyright‑defense methods—contrastive decoding, Direct Preference Optimization (DPO), and activation intervention—across two large language models (LLaMA‑3.1‑8B and Mistral‑7B‑v0.3). The study uses controlled memorization of five public‑domain books to measure literal leakage, calibrated non‑literal leakage, utility, and degeneracy, finding that each intervention level yields distinct compliance‑utility trade‑offs. Results show contrastive decoding limits degeneracy but hits a suppression floor, DPO nearly eliminates literal leakage yet causes paraphrase‑loop degeneracy, and activation intervention blocks most non‑literal queries before generation, with human evaluation highlighting coherence and perceived copyright risk differences.
By Maryam Alshehyari, Dushyant Singh Chauhan, Samuele Poppi, Martin Takac, Salem Lahlou, Nils Lukas
VerTox is a framework that turns corpus poisoning of neural ranking models into a verifiable reward‑guided reinforcement learning problem. By fine‑tuning compact large language models with reward shaping that couples ranking distortion and factual corruption, VerTox generates fluent, low‑perplexity adversarial documents that frequently outrank target items across multiple ranking architectures, including a commercial embedding model. Experiments show near‑perfect attack success and significant degradation of downstream retrieval‑augmented generation performance.
By Zhiqi Huang, Vivek Datla, Zhichao Xu, Puxuan Yu, Vivek Srikumar, Alfy Samuel
arXiv:2609.01354v1 Announce Type: cross
Abstract: Reinforcement learning with verifiable rewards (RLVR) and standard benchmark evaluation both rely on an automatic verifier that turns a free text ans...
By Esther Xin
The paper investigates whether large language models (LLMs) can leverage frozen relational‑transformer embeddings by injecting them as soft tokens. Using a learned MLP projection and LoRA adaptation, the authors fine‑tune Qwen3.5‑4B on chain‑of‑thought reasoning traces and group‑based reinforcement learning, then evaluate on ten binary classification tasks across six RelBench databases. The hybrid approach consistently underperforms the standalone relational transformer, showing sensitivity to serialization format, token budget, and RL stability, leading the authors to conclude that stronger alignment objectives and schema‑aware design are needed for reliable relational prediction.
By Francisco Galuppo Azevedo, Clarissa Lima Loures
The paper introduces a new online fair division framework where a learner must allocate indivisible items to agents in real time, balancing fairness and efficiency. Traditional methods rely on many copies of each item to estimate utilities, but this is unrealistic for platforms with many users and few interactions. By treating utility as an unknown function of item-agent features and framing the problem as a contextual bandit, the authors propose algorithms that achieve sublinear regret and demonstrate their effectiveness experimentally.
By Arun Verma, Indrajit Saha, Makoto Yokoo, Bryan Kian Hsiang Low
The study investigates whether post‑training methods—GRPO, SFT, and DPO—improve language models’ ability to follow prompt evidence that conflicts with memorized knowledge. By comparing nine training variants across different scales and families, the authors find that grounding gains are modest for GRPO, moderate for Conflict‑SFT, and near‑ceiling for DPO, but all largely rely on the same causal attention‑head set present in the starting checkpoint. Removing the starting‑model grounding direction suppresses these gains, while adding it back recovers a significant portion of DPO’s improvement, indicating that existing model machinery drives most of the observed gains.
By Prakhar Gupta, Vaibhav Gupta
arXiv:2511.08097v2 Announce Type: replace-cross
Abstract: We consider a general infinite horizon Heterogeneous Restless multi-armed Bandit (RMAB). Heterogeneity is a fundamental problem for many real...
By Dheeraj Narasimha, Nicolas Gast
NashDreamer is a new model-based reinforcement learning framework designed for two-player zero-sum imperfect-information games. It introduces a centralized Multi-Agent Recurrent State-Space Model that separates environment dynamics from player strategy effects, enabling the use of any policy gradient algorithm while preserving convergence guarantees to Nash equilibria. Experiments on four benchmark games show that NashDreamer achieves significantly better sample efficiency than model-free baselines early in training, and the authors analyze its optimization landscape, noting a potential vulnerability to posterior collapse in stochastic settings.
By Tom\'a\v{s} Hole\v{c}ek, Viliam Lis\'y
The paper introduces Fed‑LSVI, a federated online reinforcement learning algorithm that uses linear function approximation in episodic Markov decision processes. It achieves a regret bound of ≥O(√{Md^3H^4T}) while only exchanging compressed sufficient statistics, thereby meeting privacy constraints. The method reduces communication cost to logarithmic in the number of episodes, a marked improvement over previous approaches that required linear communication.
By Zihang Liang, Haochen Zhang, Lingzhou Xue
The paper introduces ReuseRL, a method that applies the Minimum Description Length principle to agentic reinforcement learning. By extracting a shared skill dictionary from successful trajectories and adding a segmentation cost to the RL objective, ReuseRL discourages idiosyncratic behaviors and promotes reusable abstract patterns. Experiments on ALFWorld, TextWorld-Cooking, and Countdown-Stepwise show that ReuseRL improves both in‑distribution and out‑of‑distribution success compared to vanilla GRPO and other baselines.
By Zhikun Xu, Yu Feng, Jacob Dineen, Taiwei Shi, Jieyu Zhao, Ben Zhou