arXiv AI
Prompt Interference in LLM Post-Training: Why Pass@k Optimization Can Degrade Pass@1
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 9
More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)
arXiv:2601. 21522v2 Announce Type: replace-cross Abstract: The performance of large language models (LLMs) on verifiable tasks is usually measured by pass@k, the probability of answering a question correctly at least once in k trials.
arXiv AI
Sep 15
Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models
The paper introduces Drift-Constrained Optimization (DCO), a framework that treats behavioral drift during fine‑tuning of instruction models as a bounded constraint rather than an uncontrolled side effect. By defining a drift budget, the authors reformulate fine‑tuning as a direction‑selection problem, showing that choosing different update directions can qualitatively change outcomes. Experiments on Qwen3 models demonstrate that carefully selected directions improve scientific reasoning and multilingual translation while preserving reasoning capabilities and general performance.
arXiv AI
Sep 30
Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning
arXiv:2609.37066v1 Announce Type: cross Abstract: Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@1 alone underidentifies what has changed...
arXiv AI
Jun 3
Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verification
arXiv:2606. 03608v1 Announce Type: cross Abstract: Test-time reinforcement learning has emerged as a promising paradigm for enhancing the complex reasoning abilities of large language models in a completely label-free manner.
arXiv AI
Oct 2
Sharpening Tax in Post-Training
The paper investigates how reinforcement learning post‑training of large language models (LLMs) tends to sharpen existing behaviors, improving single‑shot accuracy but reducing solution coverage. It shows that pre‑trained LLMs, when paired with a lightweight inference harness, can outperform post‑trained models in coverage for agentic tasks that require multi‑turn tool use. The authors introduce the Sharpening Tax metric to quantify this trade‑off, analyze its prevalence across 14 model pairs and 42 benchmark cases, and propose a Bayesian sampler, PTGS, that mitigates the tax by adapting sampling temperature to prompt difficulty.
arXiv AI
Jun 16
Understanding Diversity Collapse in RLVR via the Lens of Overtraining
arXiv:2606. 15455v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a key approach for enhancing the reasoning abilities of large language models.