← Back to all news
arXiv Computation and Language By Zhuochun Li, Yuelyu Ji, Yiming Zeng, Daqing He

SPEAR: Distilling Domain-Adaptive Reasoning Skeletons via Sequential Symbolic Alignment in Reinforcement Learning

Read the original on arXiv Computation and Language →

SPEAR (Symbolic Process Evaluation and Alignment Reward) is a training‑free, plug‑and‑play reward method for on‑policy distillation in reinforcement learning. It converts natural‑language reasoning traces into domain‑adaptive symbolic milestones and uses the longest common subsequence to align student exploration with teacher milestones, producing a dense, order‑aware reward that enforces logical consistency without an external neural verifier. Experiments on math, science, and commonsense reasoning tasks show that SPEAR effectively bridges the reasoning gap between student and teacher models through sequence‑level distillation with efficient dense process rewards.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Jun 30

Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning

arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.

By Xiaojun Wu, Xiaoguang Jiang, Huiyang Li, Jucai Zhai, Dengfeng Liu, Qiaobo Hao, Huang Liu, Zhiguo Yang, Ji Xie, Ninglun Gu, Jin Yang, Kailai Zhang, Yelun Bao, Jun Wang
llmsreinforcement-learningfine-tuningefficiencybenchmarks
More like this →
Hugging Face Trending Papers
Jun 17

Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation

Post-training of reasoning language models is commonly driven by supervised distillation and reinforcement learning with verifiable rewards. Distillation often relies on chain-of-thought annotations that are expensive to obtain and may themselves be noisy, incomplete, or partially incorrect; even when the final solution is correct, an imperfect rationale can interfere with learning.

llmsreinforcement-learningefficiencybenchmarks
More like this →
arXiv Machine Learning
Sep 14

CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models

CanvasAnneal is a curriculum‑guided reinforcement learning framework designed to improve Diffusion Language Models (DLMs) on complex reasoning and tool‑use tasks. It starts training by injecting reasoning traces from a stronger teacher model into the diffusion canvas, then gradually reduces this guidance so the model learns to generate reasoning independently. Experiments on mathematical reasoning and tool‑use benchmarks show that CanvasAnneal outperforms standard diffusion RL methods such as diffu‑GRPO on tasks like MATH500, Countdown, and Tau2, and accelerates reward improvement, though the gains vary by task.

By Blake Olson, Yuhang Song, Emmett McQuinn, Yuan Shangguan
llmsdiffusionreinforcement-learningbenchmarks
More like this →