dOPSD: On-Policy Self-Distillation for Diffusion Language Models

📰 ArXiv cs.AI

Learn how to improve diffusion language models with on-policy self-distillation, a technique that enhances post-training reasoning without exposure bias

advanced Published 7 Jul 2026

Action Steps

Implement on-policy self-distillation for diffusion language models using dOPSD
Apply dOPSD to your existing diffusion language model to reduce exposure bias
Compare the performance of your model with and without dOPSD
Fine-tune your model using dOPSD and evaluate its reasoning capabilities
Test dOPSD on different diffusion language models and tasks to assess its generalizability

Who Needs to Know This

NLP engineers and researchers can benefit from this technique to improve the performance of their diffusion language models, especially when fine-tuning is challenging

Key Insight

💡 On-policy self-distillation can enhance post-training reasoning in diffusion language models without suffering from exposure bias

Key Takeaways

Learn how to improve diffusion language models with on-policy self-distillation, a technique that enhances post-training reasoning without exposure bias

Full Article

Title: dOPSD: On-Policy Self-Distillation for Diffusion Language Models

Abstract:
arXiv:2607.04428v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) generate text by iteratively denoising a masked sequence, offering a parallel alternative to autoregressive models, but eliciting strong reasoning through post-training remains difficult: supervised fine-tuning is off-policy and suffers from exposure bias, while reinforcement learning gives only sparse, sequence-level rewards and is hard to apply without tractable sequence likelihoods. On-policy self-distil

Read full paper → ← Back to Reads