Applied Deep Learning 2025 - Lecture 2 - Neural Networks, Optimization, and Backpropagation
About this lesson
In this unit, we talk about biological neurons, artificial neurons, how they can be arranged into neural networks, and how those networks can be trained efficiently for various tasks, including classification and regression. Complete Playlist: https://www.youtube.com/watch?v=vlTnIjhhmzA&list=PLNsFwZQ_pkE8H1o874cZbiwnNRJ6hCDJI 00:00 - Start 00:28 - Recap 02:36 - The Model 04:43 - Assumption of Independent Identical Distributions 07:29 - Classes of Models 08:05 - Outputs of the Model 08:31 - Neural Networks 09:15 - The Biological Neuron 09:55 - The Artificial Neuron 10:56 - Parametric Models 11:39 - The Activation Function 15:21 - The Neural Network - A Universal Approximator 17:38 - The Loss Function 21:39 - Optimization - Getting to a small loss 22:53 - Gradient Descent 25:36 - Stochastic, Batch, and Mini-Batch Gradient Descent 27:03 - Global and Local Minima and how to avoid them 27:53 - Momentum 32:02 - Backpropagation 35:06 - Path Factorization 36:43 - Summary == Literature == 1. Pramerdorfer, Deep Learning for Visual Computing, 2016 2. Lopez et al. Skin lesion classification from dermoscopic images using deep learning techniques. 2017 3. Fei fei Li et al. Deep Learning for Visual Computing 4. Araujo dos Santos, Artificial Intelligence 5. Deep Learning: https://www.deeplearningbook.org 6. Ruder. An overview of gradient descent optimization algorithms 7. Colah. Calculus on Computational Graphs: Backpropagation. 2015 8. Ramachandran et al. Searching for Activation Functions, 2017. 9. Hendrycks et al. Gaussian Error Linear Units (GELUs), 2023.
DeepCamp AI