The spelled-out intro to neural networks and backpropagation: building micrograd

Name: The spelled-out intro to neural networks and backpropagation: building micrograd
Uploaded: 2022-08-16T22:44:26+00:00
Channel: Andrej Karpathy
Description: This is the most step-by-step spelled-out explanation of backpropagation and training of neural networks. It only assumes basic knowledge of Python and ...

Andrej Karpathy · Beginner ·📐 ML Fundamentals ·3y ago

This is the most step-by-step spelled-out explanation of backpropagation and training of neural networks. It only assumes basic knowledge of Python and a vague recollection of calculus from high school. Links: - micrograd on github: https://github.com/karpathy/micrograd - jupyter notebooks I built in this video: https://github.com/karpathy/nn-zero-to-hero/tree/master/lectures/micrograd - my website: https://karpathy.ai - my twitter: https://twitter.com/karpathy - "discussion forum": nvm, use youtube comments below for now :) - (new) Neural Networks: Zero to Hero series Discord channel: https:…

Watch on YouTube ↗ (saves to browser)

Chapters (19)

intro

0:25 micrograd overview

8:08 derivative of a simple function with one input

14:12 derivative of a function with multiple inputs

19:09 starting the core Value object of micrograd and its visualization

32:10 manual backpropagation example #1: simple expression

51:10 preview of a single optimization step

52:52 manual backpropagation example #2: a neuron

1:09:02 implementing the backward function for each operation

1:17:32 implementing the backward function for a whole expression graph

1:22:28 fixing a backprop bug when one node is used multiple times

1:27:05 breaking up a tanh, exercising with more operations

1:39:31 doing the same thing but in PyTorch: comparison

1:43:55 building out a neural net library (multi-layer perceptron) in micrograd

1:51:04 creating a tiny dataset, writing the loss function

1:57:56 collecting all of the parameters of the neural net

2:01:12 doing gradient descent optimization manually, training the network

2:14:03 summary of what we learned, how to go towards modern neural nets

2:16:46 walkthrough of the full code of micrograd on githu

Playlist

Uploads from Andrej Karpathy · Andrej Karpathy · 3 of 17

← Previous Next →