arXiv AI August 11, 2026 By Nuthakki Siva Gopala Krishna, Kanishka Jain

STEMMA: An Adversarial Multi-Agent Framework for Evaluating Self-Identity Consistency in LLMs

arXiv:2608. 08164v1 Announce Type: cross Abstract: Knowledge Distillation is a widely adopted technique in the training and fine-tuning of large language models (LLMs) enabling transfer of structured information and functional behavior from a large teacher model to a smaller student model while significantly reducing computational costs.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jul 21

Why Does Feedback-Augmented Self-Distillation Fail to Improve Retrieval-Interleaved Search Agents?

arXiv:2607. 17558v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) offers a promising approach for training large language models without relying on a separate teacher model.

By Fan Yang, Rui Meng, Yuxin Wen

llms agents efficiency

arXiv Computation and Language

6d ago

User Model Extraction via Belief Self-Distillation

arXiv:2609.31603v1 Announce Type: cross Abstract: Large language models (LLMs) implicitly infer attributes of their users and adapt their behavior accordingly, yet these beliefs remain difficult to i...

By Ali Holmov, Yiran Huang, Kirill Bykov, Zeynep Akata

llms efficiency safety

Hugging Face Trending Papers

Jul 2

Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

Post-training large language models (LLMs) without real-world interaction feedback or human-labeled supervision remains challenging, particularly in specialized domains where expert annotations are costly to obtain. Recent annotation-free self-evolution methods address this by using the model's own outputs as supervision signals, constructing a teacher via additional context and aggregating predictions across multiple rollouts through majority voting to produce pseudo-labels.

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jul 3

Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

arXiv:2607. 02460v1 Announce Type: cross Abstract: Post-training large language models (LLMs) without real-world interaction feedback or human-labeled supervision remains challenging, particularly in specialized domains where expert annotations are costly to obtain.

By Zhuowei Chen, Xiang Lorraine Li

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Aug 14

Latent On-Policy Self-Distillation

arXiv:2608. 13040v1 Announce Type: new Abstract: Enabling agents to learn from experience and internalize it into their policy has become a central problem in self-evolving AI.

By Guibin Zhang, Jiayang Lyu, Ran Sun, Xinlei Yu, Haoyu Zhao, Qibing Ren, Shuicheng Yan

agents efficiency benchmarks

arXiv AI

4d ago

Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models

arXiv:2609.37132v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) improves large language models by letting a self-teacher with privileged information provide dense token-level super...

By Zheng Zhang, Xinyue Tan, Lufei Li, Xinyi Zhang, Yexin Li, Kan Ren

llms efficiency

One email a morning, machine-written

Related stories

Why Does Feedback-Augmented Self-Distillation Fail to Improve Retrieval-Interleaved Search Agents?

User Model Extraction via Belief Self-Distillation

Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

Latent On-Policy Self-Distillation

Train Ahead, Distill Back: Bootstrapping On-Policy Self-Distillation for Large Language Models