Applied Deep Learning 2025 - Lecture 9 - Preprocessing, Augmentation, Regularization, Visualization
About this lesson
In this lecture, we're discussing how preprocessing can help our networks to learn better or even enable efficient processing in the first place. Data augmentation is another very important technique that helps us to deal with situations where we don't have a lot of data, or want our models to become more robust to small variations. Regularization is another important trick in our toolbox to prevent overfitting and finally, we have a look at how we can visualize what the models are actually learning. Complete Playlist: https://www.youtube.com/watch?v=vlTnIjhhmzA&list=PLNsFwZQ_pkE8H1o874cZbiwnNRJ6hCDJI 00:00:00 - Start 00:00:33 - Recap 00:06:26 - Preprocessing 00:08:39 - Input Normalization 00:12:46 - Preprocessing to unlock problems 00:14:58 - Data augmentation 00:22:38 - Adversarial training 00:24:38 - Adversarial Attacks 00:26:35 - Data augmentation for audio 00:29:38 - Data augmentation for text 00:33:06 - Regularization 00:35:59 - Weight Decay 00:38:18 - Early Stopping 00:40:52 - Cutout 00:41:19 - Dropout 00:43:08 - Batch Normalization 00:48:57 - Layer Normalization 00:50:36 - Normalizing activation functions 00:51:31 - Stochastic Weight Averaging 00:55:28 - Visualization 00:55:39 - Tensorboard 01:00:14 - Netron 01:00:44 - Class Activation Maps 01:08:47 - Visualizing the Gradient 01:09:45 - Summary == Literature == 1. Chatzimichailidis et al. GradVis: Visualization and Second Order Analysis of Optimization Surfaces during the Training of Deep Neural Networks. 2019 2. Nikolenko. Synthetic Data for Deep Learning. 2019 3. Pramerdorfer. Deep Learning for Visual Computing. 2016 4. Zoph et al. Learning Data Augmentation Strategies for Object Detection. 2019 5. Wu et al. Making and Invisibility Cloak: Real World Adversarial Attacks on Object Detectors. 2019 6. Goodfellow, et al. Explaining and Harnessing Adversarial Examples. 2015. 7. Nicholas et al. DocCreator: A New Software for Creating Synthetic Ground-Truthed Document Images. 2017 8. Ma. Data Augmentation for A
DeepCamp AI