Build your own neural network, Exercise 1

Brandon Rohrer · Advanced ·🧬 Deep Learning ·5y ago

Key Takeaways

This video demonstrates building a neural network framework from scratch using NumPy, covering the basics of neural networks and optimization, and provides a walkthrough of the solution to the first coding exercise.

Full Transcript

hey welcome to the course we're going to be using code walkthroughs as a way for me to explain my solutions to the coding exercises that i pose to you this is only to show one possible way to do it there are lots of ways to go about this lots of really good ways to go about this this i'm not showing you the right way or even the best way but i'm showing you a way and the benefit of that is that at each step along the way you're free to either do it in your own style to differ from what i write as much as you want but also if you get a little bogged down or sometimes the changes can be hard to keep track of you can always revert back to whatever my solution was to the previous exercise and build from that so the goal is to give you maximum flexibility but to always have a get out of jail free card in case you get tripped up the walkthroughs themselves are actually entirely optional it's just a way for me to verbally step through what i've done but you're welcome to just look at the code of my solution and for some people that actually is a much easier way to ingest the ideas and to understand the work that was done so they're not gonna you're gonna be blocked from progressing to the next example if you don't watch the video they're just here for in case it's helpful here we're taking a look at our very first test data set this is our as simple as possible example we've named it data loader 2x2 dot pi the very first thing we've done in line one is import numpy and numpy will let us do all kinds of numerical computations has a great set of tools the most important thing we'll use it for right now is it's very useful data structure called a numpy array we define the function get data sets and within it we create this variable examples which is a list of arrays and each of these arrays is two rows by two columns and the way that we initialize the array here is to do a list of lists it's a little bit confusing because we have examples which is itself a list and then within it we have a bunch of arrays which are each initialized as a list of lists so there's a bunch of square brackets running around but the way i've uh spaced it here you can see the rows and the columns and you can if you use your imagination imagine each z zero as a black pixel and each one as a white pixel then what you see in this first example is an image with a black bar on top and a white bar on the bottom the second example has a white bar on the left and a black bar on the right the third example has a white bar on top and a black bar on the bottom and so forth and as you go through here you can see that we have diagonals we can have corners and altogether there's lots of variations that we can have of this in fact because we have four pixels each of which we're allowing to have two values we can have two to the fourth or 16 different examples which we enumerated most of them here if not all and then we just close that out for this example that's all we had to do was create a list of these numpy arrays each of which is a very tiny two by two matrix or two by two two dimensional array which you can imagine as an image so this is our silly simple image test set as we get things up and going the goal being that if we can run this ridiculously simple example through we know what answer we should get then once that's if that doesn't work we know something's wrong once it does work then we can swap this out for a slightly more complex example until we step our way up to the full problem that we're trying to solve

Original Description

Get the full course experience at https://e2eml.school/312 In this course we build a neural network framework from scratch. By the time you are done, you will have a simple but fully functional neural network framework. You will understand every important concept, including optimization, normalization, backpropagation, and gradient descent. Writing the code has been broken out into 28 separate coding exercises. In each one, we discuss the principles involved, list the goals, and then you get turned loose to write the code. You get to work at your own pace and on your own timeline. If you get stuck, there are discussion threads within each lesson so that I can help you get pointed in the right direction. After you’re done, I walk through my solution line by line, describing exactly what I did and why. This sets you up to experiment on your own, to try more exotic architectures, and to better understand what frameworks like PyTorch and TensorFlow are doing behind the scenes.
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Brandon Rohrer · Brandon Rohrer · 0 of 60

← Previous Next →
1 Robot Learning with a Biologically-Inspired Brain (BECCA)
Robot Learning with a Biologically-Inspired Brain (BECCA)
Brandon Rohrer
2 BECCA talk at AGI 2011
BECCA talk at AGI 2011
Brandon Rohrer
3 Robot Learning with a Biologically-Inspired Brain (BECCA), The Sequel
Robot Learning with a Biologically-Inspired Brain (BECCA), The Sequel
Brandon Rohrer
4 BECCA listens to The Hobbit
BECCA listens to The Hobbit
Brandon Rohrer
5 Learning the building blocks of speech: BECCA extracts a hierarchy of audio features
Learning the building blocks of speech: BECCA extracts a hierarchy of audio features
Brandon Rohrer
6 BECCA listens for sound effects in The Hobbit
BECCA listens for sound effects in The Hobbit
Brandon Rohrer
7 BECCA finds movie trailers while watching the Big Bang Theory
BECCA finds movie trailers while watching the Big Bang Theory
Brandon Rohrer
8 Listening for unexpected sounds: BECCA detects anomalies in audio data
Listening for unexpected sounds: BECCA detects anomalies in audio data
Brandon Rohrer
9 Learning the building blocks of vision: BECCA extracts a spatio-temporal hierarchy of features
Learning the building blocks of vision: BECCA extracts a spatio-temporal hierarchy of features
Brandon Rohrer
10 Watching for the unexpected: BECCA detects anomalies in video data
Watching for the unexpected: BECCA detects anomalies in video data
Brandon Rohrer
11 BECCA finds a stationary target
BECCA finds a stationary target
Brandon Rohrer
12 BECCA finds a stationary target at 3X speed
BECCA finds a stationary target at 3X speed
Brandon Rohrer
13 BECCA watches the X-men and Bruce Lee
BECCA watches the X-men and Bruce Lee
Brandon Rohrer
14 BECCA plays Quidditch
BECCA plays Quidditch
Brandon Rohrer
15 BECCA chases a ball
BECCA chases a ball
Brandon Rohrer
16 BECCA chases a ball, part 2
BECCA chases a ball, part 2
Brandon Rohrer
17 Becca chases a ball, part 3
Becca chases a ball, part 3
Brandon Rohrer
18 BECCA creates features from MNIST
BECCA creates features from MNIST
Brandon Rohrer
19 How reinforcement learning works in Becca 7
How reinforcement learning works in Becca 7
Brandon Rohrer
20 Deep Learning Demystified
Deep Learning Demystified
Brandon Rohrer
21 How Data Science Works
How Data Science Works
Brandon Rohrer
22 How Convolutional Neural Networks work
How Convolutional Neural Networks work
Brandon Rohrer
23 How Bayes Theorem works
How Bayes Theorem works
Brandon Rohrer
24 How Deep Neural Networks Work
How Deep Neural Networks Work
Brandon Rohrer
25 Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)
Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)
Brandon Rohrer
26 How Support Vector Machines work / How to open a black box
How Support Vector Machines work / How to open a black box
Brandon Rohrer
27 How autocorrelation works
How autocorrelation works
Brandon Rohrer
28 Getting closer to human intelligence through robotics
Getting closer to human intelligence through robotics
Brandon Rohrer
29 A minimalist's guide to slicing and indexing pandas DataFrames
A minimalist's guide to slicing and indexing pandas DataFrames
Brandon Rohrer
30 How decision trees work
How decision trees work
Brandon Rohrer
31 Data scientist archetypes
Data scientist archetypes
Brandon Rohrer
32 How to use python's datetime package
How to use python's datetime package
Brandon Rohrer
33 How optimization for machine learning works, part 1
How optimization for machine learning works, part 1
Brandon Rohrer
34 How optimization for machine learning works, part 2
How optimization for machine learning works, part 2
Brandon Rohrer
35 How optimization for machine learning works, part 3
How optimization for machine learning works, part 3
Brandon Rohrer
36 How optimization for machine learning works, part 4
How optimization for machine learning works, part 4
Brandon Rohrer
37 How convolutional neural networks work, in depth
How convolutional neural networks work, in depth
Brandon Rohrer
38 How to pick a machine learning model 4: Splitting the data
How to pick a machine learning model 4: Splitting the data
Brandon Rohrer
39 How to pick a machine learning model 3: Choosing a loss function
How to pick a machine learning model 3: Choosing a loss function
Brandon Rohrer
40 How to pick a machine learning model 2: Separating signal from noise
How to pick a machine learning model 2: Separating signal from noise
Brandon Rohrer
41 How to pick a machine learning model 1: Choosing between models
How to pick a machine learning model 1: Choosing between models
Brandon Rohrer
42 How to pick a machine learning model 5: Navigating assumptions
How to pick a machine learning model 5: Navigating assumptions
Brandon Rohrer
43 What do neural networks learn?
What do neural networks learn?
Brandon Rohrer
44 Interview with iRobot's Director of Data Science Angela Bassa
Interview with iRobot's Director of Data Science Angela Bassa
Brandon Rohrer
45 How Backpropagation Works
How Backpropagation Works
Brandon Rohrer
46 Evolutionary Powell's method: A discrete optimizer for hyperparameter optimization
Evolutionary Powell's method: A discrete optimizer for hyperparameter optimization
Brandon Rohrer
47 1D convolution for neural networks, part 1: Sliding dot product
1D convolution for neural networks, part 1: Sliding dot product
Brandon Rohrer
48 1D convolution for neural networks, part 2: Convolution copies the kernel
1D convolution for neural networks, part 2: Convolution copies the kernel
Brandon Rohrer
49 1D convolution for neural networks, part 3: Sliding dot product equations longhand
1D convolution for neural networks, part 3: Sliding dot product equations longhand
Brandon Rohrer
50 1D convolution for neural networks, part 4: Convolution equation
1D convolution for neural networks, part 4: Convolution equation
Brandon Rohrer
51 1D convolution for neural networks, part 5: Backpropagation
1D convolution for neural networks, part 5: Backpropagation
Brandon Rohrer
52 1D convolution for neural networks, part 6: Input gradient
1D convolution for neural networks, part 6: Input gradient
Brandon Rohrer
53 1D convolution for neural networks, part 7: Weight gradient
1D convolution for neural networks, part 7: Weight gradient
Brandon Rohrer
54 1D convolution for neural networks, part 8: Padding
1D convolution for neural networks, part 8: Padding
Brandon Rohrer
55 1D convolution for neural networks, part 9: Stride
1D convolution for neural networks, part 9: Stride
Brandon Rohrer
56 The Four Grand Challenges of Robots in the Home
The Four Grand Challenges of Robots in the Home
Brandon Rohrer
57 How Convolution Works
How Convolution Works
Brandon Rohrer
58 The Softmax neural network layer
The Softmax neural network layer
Brandon Rohrer
59 Batch normalization
Batch normalization
Brandon Rohrer
60 Getting ready to learn Python, Mac edition #1: Files and directories
Getting ready to learn Python, Mac edition #1: Files and directories
Brandon Rohrer

This video teaches how to build a simple neural network framework from scratch using NumPy, and provides a walkthrough of the solution to the first coding exercise. The goal is to understand the basics of neural networks and optimization, and to be able to implement a simple neural network.

Key Takeaways
  1. Import NumPy
  2. Define the function get_data_sets
  3. Create a list of 2x2 matrices
  4. Initialize the matrices as lists of lists
  5. Imagine each matrix as an image
  6. Create a list of these matrices
  7. Close the example
💡 The key insight is that by starting with a simple example, we can build our way up to more complex problems and ensure that our neural network framework is working correctly.

Related Reads

Up next
RNNs Explained in 60 Seconds #ai #coding #machinelearning
Ascent
Watch →