Adaptive Gradient (AdaGrad) | Parameter Specific Optimization | Vanishing Learning Rate Problem

RoboSathi · Beginner ·📐 ML Fundamentals ·3mo ago

About this lesson

📘 Notes: https://robosathi.com/docs/deep_learning/optimization-method/#adagrad 🎥 NLP Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxcDlHCeNiKbRhLWKVunQaxn 🎥 Deep Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxe749nPGDV2cd6SR6zIZIJl 🎥 Machine Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxeydAqz2lsSMFYinbrJy9mu 🎥 Full Maths Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxen-R6NytSMigAri7piPhFp Pre-Requisites: 🎥 Optimization Methods Video: https://youtu.be/-q5IZkJT3nE 🕔 Time Stamp 🕘 00:00:00 - 00:00:33 Introduction 00:00:34 - 00:02:28 Problem with Sparse Features 00:02:29 - 00:05:20 Adaptive Gradient Algorithm 00:05:21 - 00:07:30 Adaptive Gradient Detailed Explanation 00:07:31 - 00:08:38 Vanishing Learning Rate Problem 00:08:39 - 00:09:04 Next: RMSProp

Original Description

📘 Notes: https://robosathi.com/docs/deep_learning/optimization-method/#adagrad 🎥 NLP Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxcDlHCeNiKbRhLWKVunQaxn 🎥 Deep Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxe749nPGDV2cd6SR6zIZIJl 🎥 Machine Learning Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxeydAqz2lsSMFYinbrJy9mu 🎥 Full Maths Course: https://www.youtube.com/playlist?list=PLnpa6KP2ZQxen-R6NytSMigAri7piPhFp Pre-Requisites: 🎥 Optimization Methods Video: https://youtu.be/-q5IZkJT3nE 🕔 Time Stamp 🕘 00:00:00 - 00:00:33 Introduction 00:00:34 - 00:02:28 Problem with Sparse Features 00:02:29 - 00:05:20 Adaptive Gradient Algorithm 00:05:21 - 00:07:30 Adaptive Gradient Detailed Explanation 00:07:31 - 00:08:38 Vanishing Learning Rate Problem 00:08:39 - 00:09:04 Next: RMSProp

Watch on YouTube ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

Related Reads

AI Training Data Services: The Complete Enterprise Guide to Building Accurate AI Models (2026)

Learn how to build accurate AI models with the right training data services for your enterprise, and why it matters for AI success

Medium · Machine Learning

Machine Learning Concepts Explained #4: Features and Labels

Learn the difference between features and labels in machine learning and their role in supervised models

Machine Learning Concepts Explained #4: Features and Labels

Learn the difference between features and labels in machine learning and their role in supervised learning

Medium · Machine Learning

Machine Learning Concepts Explained #4: Features and Labels

Learn the difference between features and labels in machine learning and their role in training supervised models

Medium · Data Science

Chapters (6)

00:00:33 Introduction

0:34 00:02:28 Problem with Sparse Features

2:29 00:05:20 Adaptive Gradient Algorithm

5:21 00:07:30 Adaptive Gradient Detailed Explanation

7:31 00:08:38 Vanishing Learning Rate Problem

8:39 00:09:04 Next: RMSProp

Dropout in Deep Learning