Hugging Face Trending Papers
TraceForge: Distilling with Your Own Thinking
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv Machine Learning
Oct 22
TraceForge: Distilling with Your Own Thinking
TraceForge introduces a two‑stage training method that generates reasoning traces in the style of a target model family, using supervised learning followed by adversarial refinement. The system produces traces that are judged as authentic with scores comparable to real traces, and when applied to teacher answers, it improves performance on coding benchmarks such as HumanEval+ and MBPP+. The approach addresses the limitation of existing models that only return answers without detailed reasoning, enabling more transparent and learnable outputs.
arXiv AI
Jul 28
Masked Distillation: Internalizing the Chain-of-Thought in Language Models
arXiv:2607. 22629v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) produce long, explicit chains of intermediate steps before generating a final answer at inference time.
arXiv AI
Oct 19
EKLAV: Elicited Key CoT Learning via Answer only superVision
EKLAV is a method that uses a teacher’s chain‑of‑thought (CoT) reasoning as privileged input while supervising only the final answer. By strategically placing parts of the trace in the prompt, EKLAV improves ranking performance on BRIGHT passage reranking and table‑reranking benchmarks, outperforming full‑trace distillation and answer‑only training. The approach reduces training compute and inference time, and the generated traces show more efficient reasoning patterns.
arXiv AI
Jul 24
Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utility
arXiv:2607. 20440v1 Announce Type: cross Abstract: Proprietary large language models (LLMs) entail substantial intellectual and financial investment, making them valuable intellectual property (IP).
arXiv AI
Jul 17
Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models
arXiv:2607. 14552v1 Announce Type: cross Abstract: A standard recipe for distilling the reasoning ability of large language models (LLMs) is to sample chains of thought from the model, keep those that reach the correct final answer, and fine-tune on the survivors.
arXiv AI
Oct 15
Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models
arXiv:2604.23238v3 Announce Type: replace-cross Abstract: Distillation via reasoning traces exposes frontier models to adversarial third parties, who can bypass their guardrails and misappropriate th...