arXiv AI
Jun 11
Soft-Prompt Tuning for Fair and Efficient LLM Benchmark Evaluation
arXiv:2606. 12117v1 Announce Type: cross Abstract: Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.
CRAFT: Cost-aware Refinement And Front-aware Tuning of Prompts
arXiv:2606. 04661v1 Announce Type: cross Abstract: Prompts tuned for accuracy often grow long, raising inference cost on every model call.
Related stories
arXiv Computation and Language
2d ago
AdaGEPA: Adaptive Feedback Allocation for Reflective Prompt Optimization
arXiv:2609.39927v1 Announce Type: new Abstract: Prompt optimization improves the performance of language-model systems on downstream tasks by refining their prompts. Classical methods evaluate prompt...
arXiv AI
Aug 12
Optimize Cheap, Deploy Strong: Cost-Aware Cross-Tier Transfer for Evolutionary Optimization
arXiv:2608. 10694v1 Announce Type: cross Abstract: Evolutionary optimization of LLM prompts and agentic programs (e.
arXiv AI
Jun 11
APEX: Automated Prompt Engineering eXpert with Dynamic Data Selection
arXiv:2606. 11459v1 Announce Type: cross Abstract: Large Language Models are highly sensitive to prompt formulation, necessitating automatic prompt optimization to unlock their full potential.
arXiv Machine Learning
Aug 18
PRO-Bid: Pareto-Prioritized Regret Optimization for Constraint-Aware Generative Auto-Bidding
arXiv:2602. 08261v2 Announce Type: replace Abstract: Auto-bidding systems strive to maximize marketing value while maintaining high compliance with efficiency constraints, such as Target Cost-Per-Action (CPA).
arXiv Computation and Language
Aug 28
TRACES: Tagging Reasoning Steps for Adaptive Cost-Efficient Early-Stopping
TRACES (Tagging Reasoning Steps for Adaptive Cost‑Efficient Early‑Stopping) is a lightweight framework that tags reasoning steps of large‑language models in real time, enabling adaptive, cost‑efficient early stopping during inference. By monitoring the types of steps generated, the method identifies when models shift their reasoning after arriving at a correct answer, allowing for interpretable stopping criteria. Experiments on mathematical reasoning benchmarks (MATH500, GSM8K, AIME) and knowledge benchmarks (MMLU, GPQA) show token reductions of 20–50% while preserving accuracy, with more conservative thresholds needed for harder tasks such as BeyondAIME and IMO AnswerBench.
arXiv Machine Learning
Aug 28
$p1$: Better Prompt Optimization with Fewer Prompts
The paper investigates why prompt optimization works better for some tasks than others by decomposing reward variance into response variance and system‑prompt variance. It finds that optimization succeeds when system‑prompt variance dominates, and that adding more user prompts can actually reduce this variance, especially on heterogeneous datasets. To address this, the authors propose $p1$, a filtering method that selects a small set of high‑variance user prompts, which improves optimization on reasoning benchmarks and even allows a system prompt trained on just two AIME 24 prompts to generalize well.
arXiv AI
Jul 9
Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1
arXiv:2607. 06764v1 Announce Type: new Abstract: Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures.
arXiv AI
Jun 4
Not All Errors Are Equal: Consequence-Aware Reasoning Compute Allocation
arXiv:2606. 04402v1 Announce Type: new Abstract: Modern reasoning models can allocate different amounts of test-time computation, such as thinking tokens, model calls, or compute budget, to different tasks.
arXiv AI
Aug 18
CAPO: Constraint-Aware Prompt Optimization for LLM Agents
arXiv:2608. 16068v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed as agents that rely on system prompts to use tools and complete tasks.
arXiv Machine Learning
Sep 22
STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification
arXiv:2609.23716v1 Announce Type: cross Abstract: Textual-gradient methods automate prompt optimization through natural-language feedback, but their iterative updates can be unstable. We identify two...
arXiv AI
Jul 14
OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping
arXiv:2607. 11089v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in complex reasoning tasks through Chain-of-Thought (CoT) prompting.