← Back to all news
arXiv AI By Ya-Shi Zhang, Jian Tang, Pierre-Andr\'e No\"el, Oleksiy Ostapenko

TIPS: A Tiny Inference-time Policy for Scaling in Diffusion Language Models

Read the original on arXiv AI →

The paper introduces TIPS, a tiny inference-time policy that controls which tokens are visible during each denoising pass of masked diffusion language models. TIPS, a 600‑K‑parameter shallow transformer, learns to unmask and remask tokens while keeping the language model frozen, using visibility‑aware confidence features and a fixed schedule for expected visibility. Trained via supervised warmup and reinforcement learning, TIPS achieves over 58% accuracy on GSM8K and MATH‑500 with only 32 decoder evaluations, surpassing strong baselines by more than 11 percentage points and remaining competitive at higher budgets without retraining.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

Hugging Face Trending Papers
Jun 10

Re-evaluating Confidence Remasking in Masked Diffusion Language Models

Masked diffusion language models (dLLMs) have recently emerged as a competitive alternative to autoregressive language models, with the promise of faster inference via parallel token generation. A notable limitation of the masked formulation, however, is that once a token has been unmasked it can no longer be revised, leaving dLLMs vulnerable to early sampling mistakes.

llmsdiffusion
More like this →
arXiv Machine Learning
Oct 8

CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models

CoDR (Confidence Drift Remasking) is a training‑free, sampler‑agnostic refinement pass for masked diffusion language models that detects and corrects early token commitments that become unsupported as context grows denser. It estimates confidence drift for all committed positions using only a few forward passes and remasks only the tokens whose confidence has dropped, thereby improving accuracy across multiple backbones, tasks, and samplers with modest computational overhead.

By Yue Wu, Qinghe Zhang, Yu Zhang, Jian Huang
llmsdiffusion
More like this →