arXiv Machine Learning
Collaborative Reasoning Distillation via Cross-Feedback and Coherent Curation
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv AI
Jun 30
Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning
arXiv:2508. 09883v2 Announce Type: replace-cross Abstract: Large language models (LLMs) demonstrate remarkable reasoning capabilities in tasks such as algorithmic coding and mathematical problem-solving.
arXiv AI
Oct 5
Learning to Revise Reasoning with Segment-wise On-Policy Distillation
arXiv:2610.02703v1 Announce Type: new Abstract: On-policy distillation (OPD) improves large language model reasoning by training students on their own rollouts with dense token-wise supervision from...
arXiv AI
Sep 1
MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation
arXiv:2608.29623v1 Announce Type: cross Abstract: Recent advances in large reasoning models (LRMs) have shown strong performance on complex problems through long chain-of-thought (Long CoT) reasoning...
arXiv Machine Learning
23h ago
Recursive Offline Distillation via Entropy Gain Sampling and Answer Mixing
arXiv:2610.20914v1 Announce Type: new Abstract: Distillation of reasoning chains from Large Language Models (LLMs) into their smaller siblings is a standard training approach. Frequently, researchers...
arXiv AI
Jun 19
Reinforcement-aware Knowledge Distillation for LLM Reasoning
arXiv:2602. 22495v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) post-training has recently driven major gains in long chain-of-thought reasoning large language models (LLMs), but the high inference cost of such models motivates distillation into smaller students.
arXiv AI
4d ago
BoundaryEvo: Data-Model Co-Evolution at the Capability Boundary for Long-Chain Reasoning Distillation
BoundaryEvo introduces a two‑stage data‑model co‑evolution framework for distilling long‑chain reasoning from large teacher models into small language models. First, Capability Priming distills a small set of teacher trajectories to give the student an initial reasoning baseline. Then, Capability‑Boundary Co‑Evolution iteratively selects questions the student has not mastered but the teacher can solve, updating the training data as the student’s capability grows. With only 1,000 examples, BoundaryEvo improves Qwen3.5‑4B accuracy from 62.9 % to 75.3 % on IMO‑AnswerBench and from 26.8 % to 46.0 % on AMO‑Bench, outperforming static and dynamic baselines across 11 benchmarks.