arXiv AI

When is a better LLM worth paying for? Stochastic Optimization with Switching Mixed-Fidelity Oracles

The paper investigates when it is worthwhile to pay for higher‑fidelity large language models (LLMs) in stochastic optimization tasks. It introduces two cost‑aware algorithmic frameworks that adaptively switch between mixed‑fidelity first‑order oracles, one for known bias and one for unknown bias, and demonstrates that these methods can reduce cumulative cost while maintaining accuracy. Experiments with synthetic data and LLM completions confirm that the adaptive policies skip expensive oracles and achieve comparable performance to fixed high‑fidelity approaches.

arXiv Machine Learning

Oct 5

Online Verification of Language Model Responses Under Cost Constraints

The paper introduces OMVV, an online multi-verifier algorithm that maintains a pool of weak verifiers with varying costs and performance. It adaptively selects a verifier each round using an online score combiner and exponential-weights routing, providing distribution-free guarantees on false-accept and false-reject rates. Experiments on reasoning benchmarks show OMVV achieves higher accuracy at lower verification cost than any single fixed verifier across different budgets.

By Erfan Hajihashemi, Yanning Shen

llms benchmarks

arXiv AI

Jun 15

Rethinking the Trust Region in LLM Reinforcement Learning

arXiv:2602. 04879v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has become a cornerstone for fine-tuning Large Language Models (LLMs), with Proximal Policy Optimization (PPO) serving as the de facto standard algorithm.

By Penghui Qi, Xiangxin Zhou, Zichen Liu, Tianyu Pang, Chao Du, Min Lin, Wee Sun Lee

llms reinforcement-learning fine-tuning efficiency

arXiv AI

Jun 9

Agentic Search for Counterfactual Recourse under Fixed LLM Budgets

arXiv:2606. 08696v1 Announce Type: cross Abstract: Counterfactual recourse aims to provide actionable feature changes that would alter an unfavorable decision made by a predictive model.

By Yasuo Tabei

llms agents efficiency

arXiv Machine Learning

Jul 10

Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models

arXiv:2607. 08665v1 Announce Type: new Abstract: Routing among large language models (LLMs) trades response quality against serving cost, motivated by the reported gap between deployed routers and a per-instance oracle.

By Teng-Ruei Chen

llms benchmarks

arXiv Machine Learning

Jun 16

Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning

arXiv:2606. 14970v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) has become a central application of modern optimization, enabling pretrained models to adapt to diverse downstream tasks and domain-specific data.

By Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin, Aleksandr Beznosikov

llms fine-tuning

arXiv Machine Learning

Jun 11

Steering the Noise: Turning Random Perturbations into Effective Descent for Memory-Efficient LLM Fine-Tuning

arXiv:2601. 04710v2 Announce Type: replace-cross Abstract: Fine-tuning large language models (LLMs) achieves strong performance but is often limited by the memory overhead of backpropagation.

By Feihu Jin, Shipeng Cen, Ying Tan

llms fine-tuning benchmarks

arXiv Machine Learning

Jun 2

BLISS: A Lightweight Bilevel Influence Scoring Method for Data Selection in Language Model Pretraining

arXiv:2510. 06048v4 Announce Type: replace Abstract: Effective data selection is essential for pretraining large language models (LLMs), enhancing efficiency and improving generalization to downstream tasks.

By Jie Hao, Rui Yu, Wei Zhang, Huixia Wang, Jie Xu, Mingrui Liu

llms benchmarks

arXiv AI

2d ago

OSCC: Certified Observation-Safe Coupling Optimization for Gradient-Noise Control in Imperfect-Information Learning

arXiv:2609.33543v2 Announce Type: replace Abstract: Coupled rollouts can reduce the noise of counterfactual action comparisons, but two issues prevent standard common-random-number constructions from...

By Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Rui Chen, Daren Zha

safety

arXiv AI

Jul 14

Learning to Fine-tune Foundation Models under Resource Limitations

arXiv:2607. 10694v1 Announce Type: cross Abstract: We study the problem of optimal continual fine-tuning for a pre-trained Foundation Model deployed at a resource-limited device.

By Thomas Tsouparopoulos, Iordanis Koutsopoulos

nlp reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 5

Cross-Epoch Adaptive Rollout Optimization for RL Post-Training

arXiv:2606. 05606v1 Announce Type: new Abstract: LLM post-training often relies on reinforcement learning methods that sample multiple rollouts per prompt, yet most existing approaches use a fixed rollout budget for every prompt, despite large differences in the training signal different prompts provide.

By Yiming Zong, Yige Wang, Jiashuo Jiang

llms reinforcement-learning benchmarks

arXiv AI

Oct 1

Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation

arXiv:2605.07111v3 Announce Type: replace-cross Abstract: Recent literature on fine-tuning Large Language Models highlights a fundamental debate. While Full Fine-Tuning (FFT) provides greater represe...

By Haozhan Tang, Xiuqi Zhu, Xinyin Zhang, Boxun Li, Virginia Smith, Kevin Kuo

llms fine-tuning data-engineering

arXiv AI

Jun 10

Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents

arXiv:2606. 10616v1 Announce Type: new Abstract: Long-horizon language agents accumulate observations, reasoning traces, and retrieved facts that exceed their finite context windows, making memory retention a fundamental resource-allocation problem.

By Qingcan Kang, Liu Mingyang, Shixiong Kai, Kaichao Liang, Tao Zhong, Mingxuan Yuan

agents

Related stories