← Back to all news
arXiv Machine Learning By Zibo Diao, Jingchu Gai, Xinyue Ai, Zhang Zhang, Zhenyu He, Di He

Lossless Anti-Distillation Sampling

Read the original on arXiv Machine Learning →

Lossless Anti-Distillation Sampling (LADS) is a defense that keeps the generation process unchanged while reducing the effectiveness of model distillation. It achieves this by coupling latent randomness across accounts, so that a single user experiences the same output as without defense, but a multi‑account distiller receives dependent data that hurts its generalization. Experiments on image, math, and code generation show that LADS degrades distilled model performance while preserving statistical fidelity for individual users.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Jul 14

Reference-Based Distillation Detection in LLMs

arXiv:2607. 09692v1 Announce Type: new Abstract: Model distillation -- training on outputs from stronger third-party models -- is widely used to boost performance, but raises concerns about unfair advantages and policy violations.

By Rajat Rawat, Sizhe Chen, Akshay Anand, Michael Duan, Bob Rotsted, Sewon Min
llmsefficiency
More like this →
arXiv AI
Jun 24

MGI: Member vs Generated Inference

arXiv:2606. 23872v1 Announce Type: cross Abstract: As generative models increasingly produce samples that are indistinguishable from human-created content, it becomes difficult to determine whether a given data point was part of a model's natural training set or was generated by the model itself, especially when models memorize and reproduce training data.

By Bihe Zhao, Michel Meintz, Juangui Xu, Franziska Boenisch, Adam Dziedzic
diffusion
More like this →
arXiv Machine Learning
5d ago

Smaller Models, Better Rejects: Preference Distillation Scaling

The paper challenges two common assumptions in preference distillation: that self-generated failures are the best negatives and that rejects must come from large models. Experiments show that smaller frozen models can generate high‑quality rejects with less compute, improving student performance on code generation and math reasoning. The authors provide a theoretical bound on Direct Preference Optimization, identify three practical interventions—mixing rejects, shuffling tokens, and selecting low‑likelihood candidates—that further enhance reject utility, and argue that task structure, not just reference policy coupling, drives effectiveness.

By Rui Cai, Wenhui Zhu, Xiwen Chen, Jincheng Cao, Han Yu, Shayan Mohajer Hamidi, Zelin He, Qiyao Ma, Daiwei Chen, Xuanzhao Dong, Yuanda Xu, Jelena Markovic-Voronov, Kayhan Behdin, Zhengze Zhou, Ran He, Alborz Geramifard, Rohit Jain, Zhe Zhao
efficiency
More like this →
arXiv AI
Jul 17

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

arXiv:2607. 14552v1 Announce Type: cross Abstract: A standard recipe for distilling the reasoning ability of large language models (LLMs) is to sample chains of thought from the model, keep those that reach the correct final answer, and fine-tune on the survivors.

By Jungseob Lee, Seungyoon Lee, Suhyune Son, Dongyub Jude Lee, Sungbin Han, Sugyeong Eo, Heuiseok Lim
llmsfine-tuningefficiencybenchmarks
More like this →