arXiv AI

Canopy: Exploiting Piecewise Smooth Tree Priors for Multi-Fidelity Bandits

CANOPY is a multi‑fidelity tree bandit algorithm that learns where a piecewise‑smooth prior holds instead of assuming global smoothness. It uses cheap random‑path probes to certify local aggregation bias and then focuses expensive leaf evaluations on cells where smoothness is violated. The method achieves provable fixed‑budget and regret guarantees that scale with the number of discontinuities, matching smooth‑tree rates when no violations exist and approaching structure‑blind search when violations are dense.

arXiv AI

Sep 2

Bandits in Prod: Hyperparameter Optimization at Inference Time

The paper introduces Online Hyperparameter Optimization (OHPO), framing it as an infinitely many‑armed bandit problem over mixed and conditional search spaces. It proposes the IMABO framework, which couples any bandit policy with any oracle for proposing new configurations, and presents IMOSS—a restart‑free anytime policy with provable regret bounds. Experiments show that IMABO, combined with practical oracles such as TPE, an incumbent‑mutation oracle, and a pretrained tabular foundation model, outperforms random search across a range of settings from classical ML models to LLM‑based agents.

By Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine

llms agents reinforcement-learning

arXiv AI

Aug 14

Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing

arXiv:2608. 12831v1 Announce Type: cross Abstract: Online platforms increasingly compare many adaptive decision policies---ranking systems, recommendation algorithms, pricing rules, and language-model agents---while each reward-bearing interaction can be costly or risky.

By Yuxiao Wen

agents reinforcement-learning

arXiv AI

Jul 28

WISERouter: LLM Routing with Workload Budget Constraint

arXiv:2607. 23765v1 Announce Type: cross Abstract: Large language models (LLMs) achieve impressive performance across multiple domains, but using the most capable model for every query is prohibitive at scale.

By Yifei Li, Zihui Gao, Laks V. S. Lakshmanan

llms reinforcement-learning

arXiv Machine Learning

Jun 2

Tree-Guided Identify-Then-Exploit: A Unified Framework of Best Arm Identification and Regret Minimization for Dueling Bandits

arXiv:2606. 01799v1 Announce Type: new Abstract: We study $N$-armed stochastic dueling bandits under the Condorcet-winner assumption, where three widely adopted objectives are considered: best-arm identification (BAI), weak regret, and strong regret.

By Pu Wang, Yao-Xiang Ding

arXiv AI

Aug 3

RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems

arXiv:2607. 29241v1 Announce Type: cross Abstract: Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes.

By Haoran Ling, Yuecheng Li, Zeyu Song, Jing Yao, Shuwen Kang, Chi Lu, Wenjin Wu, Peng Jiang

llms agents reinforcement-learning

arXiv AI

Sep 15

Compute Allocation for Self-Evolving LLMs: From Depth-Breadth to Multi-Armed Bandits

arXiv:2605.29268v3 Announce Type: replace-cross Abstract: LLM-guided evolutionary search (Evolve systems) has reached state-of-the-art results on mathematical and combinatorial tasks, yet most existi...

By Sixue Xing, Haoyu He, Kerui Wu, Zhuo Yang, Haozheng Luo, Tianfan Fu, Aarthy Nagarajan

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 4

ALMAB-DC: Active Learning, Multi-Armed Bandits, and Distributed Computing for Sequential Experimental Design and Black-Box Optimization

arXiv:2603. 21180v4 Announce Type: replace Abstract: Sequential experimental design under expensive, gradient-free objectives is a central challenge in computational statistics: evaluation budgets are tightly constrained and information must be extracted efficiently from each observation.

By Foo Hui-Mean, Yuan-chin I Chang

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 30

Learning to Route and Schedule LLMs from User Retrials via Contextual Queueing Bandits

arXiv:2602. 02061v2 Announce Type: replace Abstract: Explosive demands for LLMs often cause user queries to accumulate in server queues, requiring efficient routing (query-LLM matching) and scheduling (query prioritization) mechanisms.

By Seoungbin Bae, Junyoung Son, Dabeen Lee

llms rag

arXiv AI

Jun 9

sGPO: Trading Inference FLOPs for Training Efficiency in RLVR

arXiv:2606. 08854v1 Announce Type: cross Abstract: Standard Reinforcement Learning with Verifiable Rewards (RLVR) training allocates a fixed rollout budget to every query, without regard for what each query's difficulty means for the current policy.

By Shivchander Sudalairaj, Kai Xu, Akash Srivastava, Giorgio Giannone

reinforcement-learning

arXiv Machine Learning

Jun 29

Taming the Monster Every Context: Complexity Measure and Unified Framework for Offline-Oracle Efficient Contextual Bandits

arXiv:2602. 09456v2 Announce Type: replace Abstract: We propose an algorithmic framework, Offline Estimation to Decisions (OE2D), that efficiently reduces contextual bandit learning with general reward function approximation to offline regression.

By Hao Qin, Chicheng Zhang

reinforcement-learning

arXiv Machine Learning

Jul 14

Diversified Multinomial Logit Contextual Bandits

arXiv:2607. 11684v1 Announce Type: cross Abstract: Existing contextual multinomial logit (MNL) bandits model relevance-driven choice but ignore the potential benefits of within-assortment diversity, while submodular/combinatorial bandits encode diversity in rewards but lack structured choice probabilities.

By Heesang Ann, Taehyun Hwang, Min-hwan Oh

reinforcement-learning

arXiv Machine Learning

Jul 20

Stochastic Reset Pathfinding: Path-Level Regret for Cascading Bandits over Graph Paths

arXiv:2607. 15440v1 Announce Type: new Abstract: We introduce Stochastic Reset Pathfinding (SRP), an episodic learning problem on a known directed graph with unknown stationary edge success probabilities.

By Guni Sharon, Wei Zhang

agents reinforcement-learning safety

Related stories