Distilling LLM Feedback for Lean Theorem Proving

📰 ArXiv cs.AI

Learn to improve LLMs for theorem proving by distilling feedback for lean training, enhancing reasoning capabilities

advanced Published 1 Jun 2026

Action Steps

Apply self-distillation techniques to LLMs for theorem proving
Use Feedback Distillation to match token-level distributions
Implement GRPO with modifications to address sparse rewards and mode collapse
Train models with privileged information to enhance exploration
Evaluate model performance on theorem proving tasks using metrics such as accuracy and efficiency

Who Needs to Know This

Researchers and engineers working on LLMs and theorem proving can benefit from this technique to improve their models' performance and efficiency

Key Insight

💡 Distilling feedback from LLMs can enhance their reasoning capabilities for theorem proving

Key Takeaways

Learn to improve LLMs for theorem proving by distilling feedback for lean training, enhancing reasoning capabilities

Full Article

Title: Distilling LLM Feedback for Lean Theorem Proving

Abstract:
arXiv:2605.30861v1 Announce Type: new Abstract: Post-training for reasoning models typically combines supervised fine-tuning with reinforcement learning from verifiable rewards, most commonly with GRPO. However, this algorithm suffers from sparse rewards, limited exploration, and mode collapse. Building upon recent works on self-distillation, we propose Feedback Distillation, a training method where the model is trained to match, at the token level, its own distribution conditioned on privileged

Read full paper → ← Back to Reads