Root Mean Square Propagation (RMSProp) Optimization | Deep Learning | Vanishing Learning Rate

RoboSathi · Beginner ·📐 ML Fundamentals ·3mo ago

About this lesson

📘 Notes: https://robosathi.com/docs/deep_learning/optimization-method/#rmsprop 🎥 NLP Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxcDlHCeNiKbRhLWKVunQaxn 🎥 Deep Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxe749nPGDV2cd6SR6zIZIJl 🎥 Machine Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxeydAqz2lsSMFYinbrJy9mu 🎥 Full Maths Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxen-R6NytSMigAri7piPhFp Pre-Requisites: 🎥 AdaGrad Video: https://youtu.be/mlNASt_0tNw 🎥 Optimization Methods Video: https://youtu.be/-q5IZkJT3nE 🕔 Time Stamp 🕘 00:00:00 - 00:00:13 Introduction 00:00:14 - 00:03:12 Vanishing Learning Rate Problem of AdaGrad 00:03:13 - 00:04:09 RMSProp 00:04:10 - 00:07:42 RMSProp Algorithm 00:07:43 - 00:10:33 RMSProp Explanation 00:10:34 - 00:11:43 Limitation of RMSProp 00:11:43 - 00:12:01 Next: Adam Optimizer

Original Description

📘 Notes: https://robosathi.com/docs/deep_learning/optimization-method/#rmsprop 🎥 NLP Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxcDlHCeNiKbRhLWKVunQaxn 🎥 Deep Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxe749nPGDV2cd6SR6zIZIJl 🎥 Machine Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxeydAqz2lsSMFYinbrJy9mu 🎥 Full Maths Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxen-R6NytSMigAri7piPhFp Pre-Requisites: 🎥 AdaGrad Video: https://youtu.be/mlNASt_0tNw 🎥 Optimization Methods Video: https://youtu.be/-q5IZkJT3nE 🕔 Time Stamp 🕘 00:00:00 - 00:00:13 Introduction 00:00:14 - 00:03:12 Vanishing Learning Rate Problem of AdaGrad 00:03:13 - 00:04:09 RMSProp 00:04:10 - 00:07:42 RMSProp Algorithm 00:07:43 - 00:10:33 RMSProp Explanation 00:10:34 - 00:11:43 Limitation of RMSProp 00:11:43 - 00:12:01 Next: Adam Optimizer

Watch on YouTube ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

Related Reads

Python's Object Model in Depth: Why Two Lines That Look the Same Behave Differently

Understand Python's object model to explain differing behavior of two similar lines of code

Dev.to · Ameer Abdullah

QuantzAI: Wie ich eine Machine-Learning-Forschungsplattform für Marktdaten entwickle

Learn how to develop a machine learning research platform for market data with QuantzAI

Dev.to · Yones Alizai

Introducing NumPy4J: Bringing NumPy-Style Computing toJava

Learn how NumPy4J brings NumPy-style computing to Java, enabling efficient numerical computations in Java applications

Dev.to · Darius Nica

Edmonds-Karp Max Flow with Go

Learn to implement the Edmonds-Karp algorithm for max flow problems in Go and understand its application in network flow optimization

Medium · Programming

Chapters (7)

00:00:13 Introduction

0:14 00:03:12 Vanishing Learning Rate Problem of AdaGrad

3:13 00:04:09 RMSProp

4:10 00:07:42 RMSProp Algorithm

7:43 00:10:33 RMSProp Explanation

10:34 00:11:43 Limitation of RMSProp

11:43 00:12:01 Next: Adam Optimizer

The Adam Optimizer is Just Momentum + RMSProp