Back Propagation Through Time (BPTT) in RNN | Vanishing Gradient | Exploding Gradient | Explained

RoboSathi · Beginner ·🧬 Deep Learning ·2mo ago

About this lesson

📘 Notes: https://robosathi.com/docs/natural_language_processing/rnn/#bptt 🎥 NLP Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxcDlHCeNiKbRhLWKVunQaxn 🎥 Deep Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxe749nPGDV2cd6SR6zIZIJl 🎥 Machine Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxeydAqz2lsSMFYinbrJy9mu 🎥 Full Maths Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxen-R6NytSMigAri7piPhFp 🎥 Back Propagation: https://youtu.be/iDQTFJCOkHk 🎥 RNN: https://youtu.be/jSj1STepMA8 ✅ This video describes in detail with Maths how the gradient is Back Propagated Through Time in a RNN to update the weights. ✅ We will also understand the reason for Vanishing/Exploding gradient problem and solutions for them. 🕔 Time Stamp 🕘 00:00:00 - 00:01:56 Introduction 00:01:57 - 00:04:11 RNN Architecture 00:04:12 - 00:05:37 RNN Parameters 00:05:38 - 00:12:20 What is Back Propagation Through Time (BPTT) ? 00:12:21 - 00:17:22 RNN Layers, Error & Gradient (Maths) 00:17:23 - 00:20:25 BPTT Gradient Calculation 00:20:26 - 00:24:15 Vanishing/Exploding Gradient Problem Example 00:24:16 - 00:28:11 Solution: Vanishing/Exploding Gradient Problem 00:28:12 - 00:29:16 Next: Applications of RNN

Original Description

📘 Notes: https://robosathi.com/docs/natural_language_processing/rnn/#bptt 🎥 NLP Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxcDlHCeNiKbRhLWKVunQaxn 🎥 Deep Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxe749nPGDV2cd6SR6zIZIJl 🎥 Machine Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxeydAqz2lsSMFYinbrJy9mu 🎥 Full Maths Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxen-R6NytSMigAri7piPhFp 🎥 Back Propagation: https://youtu.be/iDQTFJCOkHk 🎥 RNN: https://youtu.be/jSj1STepMA8 ✅ This video describes in detail with Maths how the gradient is Back Propagated Through Time in a RNN to update the weights. ✅ We will also understand the reason for Vanishing/Exploding gradient problem and solutions for them. 🕔 Time Stamp 🕘 00:00:00 - 00:01:56 Introduction 00:01:57 - 00:04:11 RNN Architecture 00:04:12 - 00:05:37 RNN Parameters 00:05:38 - 00:12:20 What is Back Propagation Through Time (BPTT) ? 00:12:21 - 00:17:22 RNN Layers, Error & Gradient (Maths) 00:17:23 - 00:20:25 BPTT Gradient Calculation 00:20:26 - 00:24:15 Vanishing/Exploding Gradient Problem Example 00:24:16 - 00:28:11 Solution: Vanishing/Exploding Gradient Problem 00:28:12 - 00:29:16 Next: Applications of RNN

Watch on YouTube ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

Related Reads

Help Choosing Neural Network Architecture for Matrix Classification

Learn to choose a suitable neural network architecture for classifying matrices with variable row sizes

Reddit r/deeplearning

How to Choose the Best Deep Learning Model for Medical Imaging

Learn how to choose the best deep learning model for medical imaging to ensure the success of your medical AI project

Medium · Deep Learning

Another Way to Read Neural Geometry

Learn to read neural geometry from first principles using Goodfire's discovery and apply it to your deep learning projects

Medium · Data Science

Another Way to Read Neural Geometry

Learn to read neural geometry from first principles using Goodfire's discovery

Medium · Deep Learning

Chapters (9)

00:01:56 Introduction

1:57 00:04:11 RNN Architecture

4:12 00:05:37 RNN Parameters

5:38 00:12:20 What is Back Propagation Through Time (BPTT) ?

12:21 00:17:22 RNN Layers, Error & Gradient (Maths)

17:23 00:20:25 BPTT Gradient Calculation

20:26 00:24:15 Vanishing/Exploding Gradient Problem Example

24:16 00:28:11 Solution: Vanishing/Exploding Gradient Problem

28:12 00:29:16 Next: Applications of RNN

RNNs Explained in 60 Seconds #ai #coding #machinelearning