arXiv Machine Learning June 25, 2026 By Chengheng Li-Chen, Zhiqian Zhou, Hao Chen, Nicolas Chauvin

WinDOM: Self-Family Distillation for Small-Model GUI Grounding

Read the original on arXiv Machine Learning →

arXiv:2606. 25964v1 Announce Type: cross Abstract: Small ($\sim$2B) GUI-grounding agents are attractive for on-device deployment, accessibility tooling, and low-cost iteration, but at this scale they face two open recipe questions: how to obtain bounding-box training data without expensive human annotation, and how to combine supervised fine-tuning with reinforcement learning.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Aug 6

On-Policy Self-Distillation without Any Supervision

On-policy (Self-)Distillation (OPD / OPSD) has shown strong potential for post-training large language models (LLMs). However, existing methods still rely heavily on external supervision, including ground-truth signals, environmental feedback, or guidance from larger models, and therefore fall short of genuine "self"-distillation.

llms efficiency benchmarks

arXiv Machine Learning

Aug 7

On-Policy Self-Distillation without Any Supervision

arXiv:2608. 06296v1 Announce Type: new Abstract: On-policy (Self-)Distillation (OPD / OPSD) has shown strong potential for post-training large language models (LLMs).

By Yijiang Li, Bingyang Wang, Yijun Liang, Yunjie Tian, Di Fu, Nuno Vasconcelos

llms efficiency benchmarks

arXiv AI

Sep 1

Learning to Follow In-Context Watermark Instructions via Self-Distillation

arXiv:2608.29030v1 Announce Type: new Abstract: In-context watermarking (ICW) prepends an instruction to a query asking the model to embed a statistically detectable signal in its response. It thus e...

By Yepeng Liu, Tianyi Chen, Xuandong Zhao, Dawn Song, Yuheng Bu

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Sep 22

PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents

arXiv:2609.23449v1 Announce Type: cross Abstract: Memory systems are becoming a core component of LLM agents, but constructing and maintaining memory remains expensive because it relies on repeated c...

By Pirzada Suhail, Menglin Xia, Xuchao Zhang, Mayukh Das, Chetan Bansal, Saravan Rajmohan

llms agents efficiency

arXiv AI

Jul 14

The Verifier is the Curriculum: Execution-Gated Self-Distillation for Cross-Family Game Generation

arXiv:2607. 09709v1 Announce Type: new Abstract: Post-training a code generator against a learned judge can optimize proxy features that raise the score without improving the artifact.

By Chenyu Zhou, Qiliang Jiang, Shuning Wu, Xu Zhou

fine-tuning efficiency

arXiv AI

Sep 25

From Self-Distillation to Self-Practice: Privileged Information for Multi-Turn Agents

The paper introduces Privileged Self-Practice (PSP), a method that retains privileged information (PI) in the prompt rather than the loss during on‑policy self‑distillation for multi‑turn agents. PSP injects short per‑task instructions from an analyzer model when rollouts fail, sampling again with the instruction in context and training with the unchanged GRPO objective. Experiments on AppWorld and SWE‑bench Verified show PSP consistently outperforms plain GRPO, boosting task‑goal completion by up to 65% and resolved rate by up to 61% across three student models.

By Xingyu Su, Abhishek Kumar, Qing Ping, Youzhi Luo, Jonathan Buck, Zach Zhang, Subramanian Chidambaram, Vinayak Arannil

llms agents reinforcement-learning efficiency

One email a morning, machine-written

Related stories

On-Policy Self-Distillation without Any Supervision

On-Policy Self-Distillation without Any Supervision

Learning to Follow In-Context Watermark Instructions via Self-Distillation

PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents

The Verifier is the Curriculum: Execution-Gated Self-Distillation for Cross-Family Game Generation

From Self-Distillation to Self-Practice: Privileged Information for Multi-Turn Agents