arXiv AI
Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 24
Answer-then-Edit: Reasoning Skeleton Editing for Anti-Distillation with Preserved Utility
arXiv:2607. 20440v1 Announce Type: cross Abstract: Proprietary large language models (LLMs) entail substantial intellectual and financial investment, making them valuable intellectual property (IP).
arXiv AI
Aug 11
Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs
arXiv:2608. 09542v1 Announce Type: cross Abstract: Large reasoning models (LRMs) achieve remarkable success on complex tasks but remain vulnerable to harmful prompts that induce unsafe outputs.
arXiv Machine Learning
3d ago
TraceForge: Distilling with Your Own Thinking
TraceForge introduces a two‑stage training method that generates reasoning traces in the style of a target model family, using supervised learning followed by adversarial refinement. The system produces traces that are judged as authentic with scores comparable to real traces, and when applied to teacher answers, it improves performance on coding benchmarks such as HumanEval+ and MBPP+. The approach addresses the limitation of existing models that only return answers without detailed reasoning, enabling more transparent and learnable outputs.
arXiv AI
Oct 9
The Distillation Game: Adaptive Evaluations & Efficient Defenses
The paper introduces a minimax framework to analyze the trade-off between model utility and vulnerability to distillation attacks, featuring an adaptive evaluation rule that reweights high-value examples and a teacher-side defense that suppresses outputs most useful for imitation. It proposes a cheap, forward-pass-only defense called Product-of-Experts (PoE) that combines the teacher with a proxy student during generation. Empirical results show a significant passive–adaptive gap, with adaptive students recovering more capability than passive evaluation suggests, narrowing the robustness gap between expensive defenses and PoE while keeping PoE cheaper and preserving higher-quality reasoning traces.
arXiv Machine Learning
Jul 9
Geometric Self-Distillation for Reasoning Generalization
arXiv:2607. 06855v1 Announce Type: new Abstract: On-policy distillation is a practical post-training recipe for large language models, supplying dense teacher supervision on the student's own trajectories.
arXiv AI
Jul 14
What Does It Mean to Break a Distillation Defense?
arXiv:2606. 25059v2 Announce Type: replace-cross Abstract: Black-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs.