← Back to all news
arXiv Machine Learning By Ibne Farabi Shihab, Md Najmus Swaqeeb, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Joyanta Jyoti Mondal

When Is Language-Model Advice Worth Buying? A Matched-Budget Study of Predictive Gating in Reinforcement Learning

Read the original on arXiv Machine Learning →

The paper introduces a cost‑aware controller called certified predictive advice gating that decides when to request language‑model advice in reinforcement learning. By forecasting advisor responses, evaluating parsed actions, and reusing cached advice, the gate only requests advice when a calibrated lower bound on the value exceeds the cost. Experiments on BabyAI tasks show that the gate improves return while reducing advice calls by over 97 % compared to always asking, though matched‑budget comparisons do not show a clear advantage over an early‑query schedule.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 25

Certified Predictive Value-of-Advice Gating for Cost-Aware Language-Model Guidance in Reinforcement Learning

The paper proposes a method for selectively querying language‑model advice in reinforcement learning by predicting the value of potential responses and only querying when the expected benefit outweighs the cost. It introduces a certified, response‑contingent metareasoning framework that guarantees near‑optimal advice usage under certain assumptions, and demonstrates that a calibrated controller with Qwen2.5 advisors can improve task performance while drastically reducing the number of advice calls on the BabyAI benchmark.

By Ibne Farabi Shihab, Md Najmus Swaqeeb, Abu Sa-Adat Mohamed Moon-Im Al Ahsan
reinforcement-learning
More like this →
arXiv AI
2d ago

PORA: Learning When More Reasoning Pays Off for LLM Agents

The paper introduces PORA, a method that teaches large language model agents to decide when to allocate more reasoning steps versus fewer, based on the expected downstream benefit. PORA trains by running each state twice—once with short reasoning and once with long reasoning—and learns to predict the value of extended reasoning. During inference, a controller uses this prediction and remaining budget information to choose the appropriate reasoning length without running both options, improving performance across diverse tasks.

By Bo Yan, Qi Zhang, Zhen Tan, Tianlong Chen, Song Wang
llmsagents
More like this →
arXiv Machine Learning
Aug 13

When Offline Evaluation Misleads: A Diagnostic Protocol for Reward and Policy Selection in Delayed-Feedback Contextual Bandits

arXiv:2608. 11560v1 Announce Type: new Abstract: Personalizing marketing messages with contextual multi-armed bandits (CMABs) drives real business value, yet the objective that ultimately matters - a downstream conversion - is observed only weeks later, too late to drive online learning.

By Sang Su Lee, Vineeth Loganathan, Shishir Dash, Vijay Raghavan
reinforcement-learningbenchmarkssafety
More like this →
arXiv Machine Learning
Jul 30

Early Verdicts, Better Budgets: Sequential Adaptive Rollout Allocation for Compute-Efficient RLVR

arXiv:2607. 26253v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) is bottlenecked by rollout generation, yet many sampled prompts produce saturated groups (all responses correct or all incorrect) whose zero reward variance yields no policy-gradient signal.

By Pixel Nomand, Elena Voss, Marcus Hale, Sofia Reyes
reinforcement-learning
More like this →