← Back to all news
arXiv AI

Characterizing a Configuration Where Inference-Time PRM-Pruned Fragment Grafting Is Inert: Evidence from Three Reasoning LMs

The paper investigates PRM‑Pruned Fragment Grafting (PPFG), an inference‑time technique that extracts high‑reward prefixes from a pruned chain‑of‑thought and grafts them into a sibling decoding process. Experiments on Qwen2.5‑7B‑Instruct with Math‑Shepherd across 500 MATH problems and multiple seeds show that PPFG performs statistically indistinguishable from a parallel‑CoT baseline, with only 14% of grafts targeting genuinely struggling chains. The study extends across three language models, six benchmarks, and multiple PRM configurations, concluding that PPFG’s inertness is not due to heuristic specifics and providing an equivalence‑testing framework for mechanism nulls.

Related stories

arXiv AI
Aug 20

Selection, Recombination, or a Fresh Solve? A Candidate-Free Control for Single-Pass Test-Time Aggregation

The paper investigates whether providing candidate solutions during test‑time aggregation improves or harms accuracy compared to a fresh solve that does not use any candidates. Using Qwen3‑4B on AIME‑2025 and HMMT‑2025, the authors find that conditioning on multiple correct candidates boosts accuracy (+0.290), while conditioning on an all‑wrong candidate pool reduces accuracy (−0.123); the effect for a single correct candidate remains unclear. The study also explores structured interventions and placebo controls, but the underlying mechanisms of these effects are not resolved.

By Guiv Farmanfarmaian
benchmarks
More like this →
arXiv AI
Sep 1

Locked at the Entrance, Open Inside: Where RLVR Narrows the Solution Space

The paper investigates why reinforcement learning with verifiable rewards (RLVR) reduces the diversity of solutions in reasoning tasks. By analyzing the Countdown task, the authors show that RLVR contracts the solution space mainly at the entrance—before the first arithmetic operation—causing a 67% drop in solution coverage. They demonstrate that providing an unselected entrance prefix or applying entrance‑targeted interventions can restore or even improve coverage without harming accuracy.

By Qiancheng Zhou, Ruizhe Li
llmsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv AI
Sep 1

Moving the Mean Toward the Known Good, Not Beyond It: What Inference-Time Interventions and Weight Consolidation Buy in Open-Ended Generation

The study investigates how inference‑time interventions and weight consolidation affect open‑ended generation in an online bin‑packing task. By iteratively generating, verifying, selecting, and consolidating with LoRA, the model’s outputs shift toward higher value, reducing excess by 1.7 points and outperforming random consolidation by 3.1 points. Across three independent runs, the mean performance remained consistent, and the best candidates converged to the classic heuristic’s level without exceeding it, while consolidation also lowered the proportion of better‑than‑classic candidates but increased their absolute number.

By Roberto I. Ono Filho
fine-tuning
More like this →
arXiv AI
Sep 3

Coverage, Not Targeting: A Structural Regime in Multi-Turn Agent Credit Assignment

The paper argues that in multi‑turn agentic reinforcement learning, credit assignment should be viewed as a coverage problem rather than a targeting problem. It introduces verifier information density (V_d) as a structural metric, showing that terminal‑state verifiers operate in a low‑V_d regime where targeting fails. Experiments on tau^2‑bench, BFCL, and ToolACE‑2‑8B demonstrate that uniformly distributing reward across all turns outperforms sparse, targeted rewards, and that full chain coverage is necessary for optimal performance.

By Chenyu Zhou, Qiliang Jiang, Shuning Wu, Xu Zhou
agentsreinforcement-learning
More like this →