'How neural networks learn' - Part III: Generalization and Overfitting

In this third episode on "How neural nets learn" I dive into a bunch of academical research that tries to explain why neural networks generalize as wel as they do. We first look at the remarkable capability of DNNs to simply memorize huge amounts of (random) data. We then see how this picture is more subtle when training on real data and finally dive into some beautiful analysis from the viewpoint on information theory. Main papers discussed in this video: First paper on Memorization in DNNs: https://arxiv.org/abs/1611.03530 A closer look at memorization in Deep Networks: https://arxiv.org/ab…