arXiv Machine Learning September 22, 2026 By Can Wu, Xinrui Chen, Ou Wu, Yi Du

Token Utility Is Selection-Conditioned: Coupled Selection of Prompt Context and Response Supervision for Efficient Instruction Tuning

Read the original on arXiv Machine Learning →

The paper introduces BRIDGE, a method for efficient instruction tuning of large language models that jointly selects prompt context and response supervision. BRIDGE uses a shared validation-directed interaction surrogate to evaluate token utility conditioned on the other side’s retained state, and employs budgeted alternating selection and structure-aware projection to produce coherent supervision spans. Experiments across three model families show BRIDGE outperforms independent selection methods in mathematical reasoning, code generation, and instruction following, with its advantage increasing as compression grows.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 10

FutureBridge: Token Selection Beyond Local Preference in Collaborative Decoding

arXiv:2608. 06819v1 Announce Type: cross Abstract: Token-level collaboration allows a large language model (LLM) to assist a small language model (SLM) when their predictions diverge.

By Quanquan Li, Hongbo Zhang, Yihe Chi, Jingyu Li, Xidong Xi, Liuyang Song, Hongzhen Zhang, Yuxiang Huang, Jing Ke, Siyuan Ma, Junyi Lin, Guitao Cao

llms benchmarks

arXiv Machine Learning

Jul 9

MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

arXiv:2607. 06974v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly improve their reasoning at test time via additional computation, yet most existing works treat each problem in isolation.

By Ruilin Tong, Dong Gong

llms rag

arXiv Machine Learning

Jul 3

PARTREP: Learning What to Repeat for Decoder-only LLMs

arXiv:2607. 01792v1 Announce Type: cross Abstract: While decoder-only LLMs excel at a vast array of natural language tasks, it suffers from an asymmetric information flow induced by causal attention: later tokens are richer in contextual grounding than earlier ones.

By Andikawati P Widjaja, Yongjun Kim, Hyounghun Kim, Jaeho Lee

llms efficiency benchmarks

Hugging Face Trending Papers

Jul 2

PARTREP: Learning What to Repeat for Decoder-only LLMs

While decoder-only LLMs excel at a vast array of natural language tasks, it suffers from an asymmetric information flow induced by causal attention: later tokens are richer in contextual grounding than earlier ones. A simple and effective remedy is prompt repetition -- just appending a second copy of prompt before generation can redistribute grounding across positions and improve reasoning performance.

llms efficiency benchmarks

arXiv Machine Learning

Jun 19

A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)

arXiv:2602. 14696v2 Announce Type: replace Abstract: Instruction fine-tuning of large language models (LLMs) often involves selecting a subset of instruction training data from a large candidate pool, using a small query set from the target task.

By Nihal V. Nayak, Paula Rodriguez-Diaz, Neha Hulkund, Sara Beery, David Alvarez-Melis

llms fine-tuning

arXiv AI

Aug 3

TAPR: Enhancing LLM Performance with a Task-Aware Prompt Rewriter

arXiv:2607. 28657v1 Announce Type: new Abstract: Large Language Models (LLMs) often require carefully crafted prompts to unlock their full potential, which can be a barrier for non-expert users.

By Oliver Savolainen, Emanuele Bastianelli, Hosein Azarbonyad

llms nlp reinforcement-learning fine-tuning benchmarks

One email a morning, machine-written

Related stories

FutureBridge: Token Selection Beyond Local Preference in Collaborative Decoding

MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

PARTREP: Learning What to Repeat for Decoder-only LLMs

PARTREP: Learning What to Repeat for Decoder-only LLMs

A Critical Look at Targeted Instruction Selection: Disentangling What Matters (and What Doesn't)

TAPR: Enhancing LLM Performance with a Task-Aware Prompt Rewriter