← Back to all news
arXiv Machine Learning By Jonathan Williams, Esin Tureci Karthik R. Narasimhan

Adapter Thickets: Splitting an RLVR Budget Beats Concentrating It

Read the original on arXiv Machine Learning →

The paper investigates how reinforcement learning with verifiable rewards (RLVR) interacts with majority voting over sampled completions, a common test‑time scaling technique. It shows that training a single LoRA adapter with the full RLVR budget can improve single‑sample accuracy but often harms the overall majority‑vote performance because the policy’s errors become increasingly correlated. By splitting the same training budget across multiple adapters—each trained on a disjoint shard, forming an "adapter thicket"—the authors demonstrate that this approach consistently out‑votes the fully trained single adapter and preserves most of the single‑sample gains, especially when the number of votes is large.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Jul 21

MADA-RL: Multi-Agent Debate-Aware Reinforcement Learning for Parameter-Efficient Reasoning in Compact Models

arXiv:2607. 18006v1 Announce Type: cross Abstract: Large language models achieve strong reasoning performance, but often at prohibitive training cost - a challenge that is especially acute for compact models ($\leq 4 \, \mathrm{B}$ parameters) trained under limited budgets.

By Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov, Zifeng Ding, Volker Tresp, Yunpu Ma
llmsagentsreinforcement-learningfine-tuningbenchmarks
More like this →
arXiv Machine Learning
Sep 29

New LoRA Skills Should Read but Never Write

The paper introduces READ, a method for composing low‑rank adapters (LoRA) in large language models. By rewriting each adapter into a balanced canonical form and enforcing a one‑directional coupling, READ allows new skills to read but never write into the output subspaces of existing skills, eliminating interference. Experiments on four benchmark suites and two model families show that READ consistently outperforms existing baselines, improving SuperGLUE scores by over twenty points and domain suite scores by more than seven points.

By Zeyan Li, Panqi Yang, Qirong Guo, Shengda Zhuo, SIyuan Qiu, Hu Xu, Chun Li, Jianfeng Xu
llmsfine-tuningbenchmarks
More like this →
Hugging Face Trending Papers
Jul 30

LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts

Reinforcement learning from verifiable rewards (RLVR) for mathematical reasoning suffers from a structural blind spot: on "cliff" prompts-those on which every sampled rollout in a group fails-the group-normalized advantage is identically zero, so GRPO produces no gradient on precisely the prompts at the frontier of the model's capability. We introduce LoRA Scaffolded Policy Optimization (LSPO), a sampling-time mechanism that recovers this lost gradient.

reinforcement-learningfine-tuningbenchmarks
More like this →
arXiv Machine Learning
Oct 19

Sample, Mix, or Verify? Allocating Test-Time Compute under Imperfect Verifiers

The paper introduces SPLIT, a method that optimally allocates test‑time compute among sampling, model selection, and verification for language models and agents. Using an exact probability formula that accounts for task difficulty, shared blind spots, and verifier correlations, SPLIT is fitted via empirical Bayes on a small labeled pilot and then used to plan budgets for unseen tasks. Across seven diverse benchmarks, 26 models, and six verifier types, SPLIT achieves near‑optimal performance, outperforming rule‑based baselines by up to 2.5 percentage points and providing insights into when additional LLM judges reduce precision.

By Srinivasan Iyengar
llmsagentsbenchmarks
More like this →