The paper introduces EXPO, an algorithm that automatically optimizes the meta-prompt—specifically the task description and meta-instruction—for large language model agents in sequential decision-making tasks such as Bayesian optimization and multi-armed bandits. Building on adversarial bandit techniques to handle non-stationary rewards, the authors extend EXPO to EXPO-ES, which also optimizes exemplars (historical interactions) within the meta-prompt. Experiments demonstrate that these methods significantly improve the performance of LLM-based agents in sequential decision-making scenarios.
By Mingze Kong, Zhiyong Wang, Yao Shu, Zhongxiang Dai
Recent advances in Reinforcement Learning (RL) have substantially improved the capabilities of autonomous search agents, enabling sophisticated planning, and iterative retrieval over dynamic information sources. However, optimizing language models for specialized search behaviors often incurs an alignment tax, where gains in search performance come at the expense of general-purpose capabilities, limiting their effectiveness as universal assistants.
arXiv:2608. 16707v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as decision-making agents in settings that require sophisticated environmental exploration.
By David Eric Austin, Kaheer Suleman, Jackie Chi Kit Cheung
arXiv:2606. 24064v1 Announce Type: new Abstract: Distilling reasoning capabilities from strong to weak language models typically involves imitating specific solution trajectories, effectively transferring what to answer rather than how to reason.
By Tianyuan Shi, Canbin Huang, Bei Li, Xin Chen, Xiaojun Quan, Jingang Wang, Qifan Wang
arXiv:2607. 24779v1 Announce Type: new Abstract: Online advertising bidding systems typically deploy multiple offline-trained expert models (e.
By Ji Wu, Yunshan Peng, Wentao Bai, Yunke Bai, Wenzheng Shu, Jinan Pang, Yanxiang Zeng, Xialong Liu
Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models, but prompt groups with identical rollout rewards consume generation budget without effective learning signals. Pre-rollout prompt selection can reduce this waste by screening prompts before rollout generation.