arXiv AI
Structural Rationale Distillation via Reasoning Space Compression
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 30
Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.
arXiv AI
Sep 1
MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
arXiv:2608.29623v1 Announce Type: cross Abstract: Recent advances in large reasoning models (LRMs) have shown strong performance on complex problems through long chain-of-thought (Long CoT) reasoning...
arXiv AI
Oct 12
BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation
BoundaryEvo introduces a two‑stage data‑model co‑evolution framework for distilling long‑chain reasoning from large teacher models into small language models. First, Capability Priming distills a small set of teacher trajectories to give the student an initial reasoning baseline. Then, Capability‑Boundary Co‑Evolution iteratively selects questions the student has not mastered but the teacher can solve, updating the training data as the student’s capability grows. With only 1,000 examples, BoundaryEvo improves Qwen3.5‑4B accuracy from 62.9 % to 75.3 % on IMO‑AnswerBench and from 26.8 % to 46.0 % on AMO‑Bench, outperforming static and dynamic baselines across 11 benchmarks.
arXiv AI
Oct 28
From Solver Trajectories to Teaching Trajectories: Cognition-Aligned Reasoning Distillation
The paper introduces Cognition-Aligned Reasoning Distillation (CARD), an RL-based method that converts solver trajectories from large reasoning models into teaching trajectories better suited for smaller student models. CARD uses two reward signals—a Distribution Alignment Reward and a Logical Coherence Reward—to align the teacher’s outputs with the student’s cognitive capabilities and reduce misleading reasoning steps. Experiments show that CARD improves both in-domain and out-of-distribution performance compared to distillation from raw teacher trajectories.
arXiv AI
Jul 1
Distilling the Essence: Efficient Reasoning Distillation via Sequence Truncation
arXiv:2512. 21002v3 Announce Type: replace-cross Abstract: Distilling the capabilities from a large reasoning model (LRM) to a smaller student model often involves training on substantial amounts of reasoning data.
arXiv AI
Jul 14
BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking
arXiv:2602. 17686v5 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) reasoning has significantly improved LLMs' mathematical problem-solving capabilities, but distilling such capabilities into smaller models remains challenging due to the capacity mismatch between verbose teachers and compact students.