DP-OPD: Differentially Private On-Policy Distillation for Language Models

1. 1Apply differential privacy to on-policy distillation for language models
2. 2Use DP-OPD to balance record-level protection and utility loss in autoregressive generation
3. 3Implement DP-SGD as a baseline for comparison
4. 4Evaluate the trade-offs between privacy and efficiency in language model compression

This research benefits AI engineers and ML researchers working on language models, as it provides a method for protecting sensitive information while maintaining model performance

💡 DP-OPD balances formal privacy guarantees with efficient deployment through model compression

🔒 DP-OPD: Differentially private on-policy distillation for language models 📚