← Back to all news
arXiv AI By Leon Sick, Lukas Hoyer, Yannick Str\"umpler, Christian Reisswig, Timo Ropinski, Federico Tombari

On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models

Read the original on arXiv AI →

The paper introduces On-Policy Self-Training, a method that allows large language models to improve themselves without external teachers or curated datasets. By generating its own question-answer pairs and privileged reasoning demonstrations guided only by coarse topic descriptions, the model creates a synthetic curriculum that enhances performance across multiple domains. Experiments show consistent self-improvement on both text and multimodal benchmarks, including Visual Question Answering, while preserving abilities on non-self-trained domains.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Aug 28

Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models

Video-OPSD introduces a post‑training framework for Video Large Language Models that leverages privileged visual evidence to enhance on‑policy self‑distillation. The method constructs a self‑teacher conditioned only on annotated evidence frames, while the student processes the full video, allowing the teacher to provide more focused supervision. Additionally, an evidence‑guided token optimization scheme weights distillation based on each token’s reliance on privileged evidence, improving perceptually grounded reasoning. Experiments demonstrate consistent gains over standard OPSD and comparable performance to GRPO with less training time.

By Ziyue Wang, Shiqi Huang, Weiwen Xu, Bihan Wen, Xudong Jiang
llmsefficiencybenchmarks
More like this →
arXiv Computer Vision
Aug 27

Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding

The paper introduces Clue-OPSD, a clue‑privileged on‑policy self‑distillation framework that improves long‑video understanding by focusing on short, question‑relevant clue intervals rather than the entire video. Experiments on multiple benchmarks and Qwen3.5 model scales show that this approach consistently outperforms standard backbone models and competes strongly with supervised post‑training baselines, all while requiring fewer input frames and no additional inference modules.

By Kaishen Wang, Dongdi Zhao, Yijun Liang, Dingqiang Ye, Ruibo Chen, Heng Huang, Di Fu
llmsefficiencymultimodalbenchmarks
More like this →