Self Distillation Explained in like 1min.

Key Takeaways

the core idea of self-distillation. We take an LLM and we operate it in two modes. The first mode is the student mode. We just get the input prompt, some question from a user, X, and it's output a response, Y. A second mode is a teacher mode, where in addition to the input prompt, the same LLM also get an extra context, C. This extra context can be expert demonstration, instruction, feedback, whatever, but the important thing is that now the model is conditioned on another input, the context. And this automatically change its output distribution. Now, the output distribution, the responses that the teacher would have produced, uh if we sample from it, are kind of different from the student. And therefore, the core idea in self-distillation is just to use this teacher and do teacher-student learning, a old paradigm in machine learning, where you just minimize the sum distributional measure, and this in our case we chose the reverse KL between the teacher and the student. Um and take the gradient, of course, only through the student and not through the teacher, because the teacher is the one that guided the learning. So, this is a distillation algorithm. Um and why we like it is that this is much more similar to how humans learn.