The Algorithm that Helps Machines Learn [AdamW]

Jia-Bin Huang · Beginner ·📐 ML Fundamentals ·1y ago

About this lesson

How do machines learn? In this video, we review the basic ideas of optimizers, algorithms that efficiently update the parameters of deep neural networks and minimize the loss function. We will cover gradient descent, momentum, RMSProp, Adam, and AdamW. References: [RMSProp] https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf [Adam] Adam: A Method for Stochastic Optimization https://arxiv.org/pdf/1412.6980 [AdamW] Decoupled Weight Decay Regularization https://openreview.net/forum?id=Bkg6RiCqY7

Original Description

How do machines learn? In this video, we review the basic ideas of optimizers, algorithms that efficiently update the parameters of deep neural networks and minimize the loss function. We will cover gradient descent, momentum, RMSProp, Adam, and AdamW. References: [RMSProp] https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf [Adam] Adam: A Method for Stochastic Optimization https://arxiv.org/pdf/1412.6980 [AdamW] Decoupled Weight Decay Regularization https://openreview.net/forum?id=Bkg6RiCqY7
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related Reads

📰
Introduction Data Science and Machine Learning
Learn the basics of data science and machine learning to stay ahead in the emerging field
Medium · Data Science
 📰
AgriScore: An Explainable AI Credit Scoring System for Smallholder Farmers
Learn how AgriScore uses explainable AI for credit scoring to help smallholder farmers access loans, and why this matters for financial inclusion
Medium · Machine Learning
 📰
AgriScore: An Explainable AI Credit Scoring System for Smallholder Farmers
Learn how AgriScore uses explainable AI for credit scoring of smallholder farmers, and how to build a similar system using machine learning and Streamlit
Medium · Data Science
 📰
The Sophistication Trap: Why the Smarter AI Technique Keeps Losing
Smarter AI techniques don't always guarantee better results, and understanding the sophistication trap can help you optimize your approach
Medium · Machine Learning
Up next
How to start learning AI | Complete AI Learning Path | Roadmap For Beginners (With No Background)
Career Talk
Watch →