Rethinking On-Policy Self-Distillation for Thinking Models

📰 ArXiv cs.AI

Learn how to improve thinking models using on-policy self-distillation and avoid degradation on long reasoning traces

advanced Published 7 Jul 2026

Action Steps

Apply on-policy self-distillation to a thinking model using privileged information
Test the model on long reasoning traces to identify potential degradation
Configure the self-distillation process to balance improvement and degradation
Run experiments to compare the performance of the model with and without self-distillation
Analyze the results to determine the optimal approach for improving thinking models

Who Needs to Know This

ML researchers and engineers working on thinking models can benefit from this knowledge to improve their models' performance and avoid common pitfalls. This is particularly relevant for teams developing language models that require test-time reasoning.

Key Insight

💡 On-policy self-distillation can improve thinking models, but may degrade performance on long reasoning traces if not configured carefully

Key Takeaways

Learn how to improve thinking models using on-policy self-distillation and avoid degradation on long reasoning traces

Full Article

Title: Rethinking On-Policy Self-Distillation for Thinking Models

Abstract:
arXiv:2607.05184v1 Announce Type: new Abstract: Self-distillation is a promising recipe for self-improvement in language models. In this setting, a model can serve as its own teacher when given privileged information, such as a solution to a math problem. This seems especially appealing for thinking models, which can use test-time reasoning to absorb the privileged information. Surprisingly, we show that privileged self-distillation degrades thinking models on long reasoning traces: across five

Read full paper → ← Back to Reads