PyTorch Crash Course: Deep Learning in Python

Key Takeaways

Today we're going to do a crash course on PyTorch, the leading machine learning framework out there. There's oftentimes discussions like TensorFlow versus PyTorch. Sometimes even Jax is put into this comparison, but for the most part, everyone is aligned at PyTorch is the superior framework. Most machine learning projects either directly or indirectly use PyTorch. So if you have to learn one package for deep learning, that is definitely the one you want to pick. My goal for this video today is to cover the basics of PyTorch as quickly as possible and to give you a good introduction into the framework. If you like this video, let me know by hitting a like button and subscribing. But now, let us get right into it. [music] >> All right, so we're going to do a PyTorch crash course in this video today. And as always with my crash courses, I'm going to try to focus on simplicity, speed, and being concise. It's going to be very beginner friendly. I don't expect you to know anything about PyTorch. I also don't really expect you to know a lot about neural networks and the theory behind them. Although I will make one or two comments about this. I'm not going to go very deep into this. I do however expect you to be comfortable coding in Python. So I'm not going to cover Python basics and also it helps to have worked with NumPy and the basic data science stack before just because I'm going to make some comparison but that's not a necessity. Now for those of you who don't know what PyTorch is at all, it is a deep learning framework for Python. I would say it's the deep learning framework for Python. Almost every machine learning project is built on top of PyTorch in a way either directly or indirectly. And what deep learning framework basically means is that this is a framework that allows us to easily create, train, use and evaluate neural networks in Python. And neural networks for those of you who don't know what that is, basically this is a architecture type or an algorithm type for machine learning. So a specific way of doing machine learning with neurons and edges and weights and biases parameters that we're training that allows us to do a bunch of different tasks. So recognizing images or classifying images, generating text, the transformer-based architectures are also neural networks in the essence or in their essence and pietorch is the framework for working with these types of architectures. So let us briefly talk about the structure of this crash course today. My plan is to roughly split it up into two sections. The first one is going to be focusing more on the low-level aspects of PyTorch. So we're going to talk about the two components here which is on the one hand PyTorch is aiming to be a replacement of NumPy for the GPU and on the other hand it also has an automatic differentiation engine which automatically takes a derivative of functions which is extremely useful when it comes to working with neural networks. So we're going to explore the lower level aspects of PyTorch. What is happening behind the scenes and in the second part we're going to go through a practical example. So we're going to load a data set. We're going to use the PyTorch data loader and data set. We're going to then create a neural network. We're going to train it. We're going to use it and evaluate it. So that is going to be the plan for this video today. Starting with a little bit of low-level theory and then going into the practical use case. So let us get started with the environment setup. For this, open up your terminal. Navigate to the directory that you want to be working in. And now it depends on your setup. The easiest way to install it is to just use pip. So pip or pip 3 install and then torch. It's not pietorch. The package name is torch. So you can just run this command. If you want to install it on your global python installation. In my case, I'm going to use uv which is a rustbased python package manager. So I'm going to go a different route. I'm going to say uv init here. And then I'm going to say UV add which is the same as pip install just focused on this particular directory. So I don't alter my packages on a system level but I have a specific environment here in this tutorial directory. If you also want to use UV you can easily install it by saying pip or pip 3 install uv. This will then install it as a system or python system package so to say even though it's rust actually. Uh but then you can also use just uv init and then uv the package names. If you just want to use pip just say pip install and the same packages that I'm typing here now. So I'm going to use torch of course. This is what the crash course is about. Then also I'm going to install jupit lab for the development environment. So this is what we're going to be using for coding here. And in addition to that I'm going to use numpy as well to show you the differences between numpy and pytorch. And we're also going to install scikitlearn not because we're going to use it for machine learning but because we're going to load a data set from there. So run this either pip install or uv at and once you have this you can just open up jupit lab ei ei ei ei ei ei ei ei ei ei either by saying jupit lab or if you're also using uv you have to say uv run jupit lab this is going to open it in your browser so take that window I'm going to put this to a different workspace now and here now I have jupitter lab opened up for those of you who don't know what jupitter lab is or what interactive python notebooks are at all uh basically instead of running a script from top to bottom like ordinary Python files. What we can do here is we can run individual cells. So I can define a variable A here. I can print the value down here without running the first cell again. Then I can do something with A like add five to it and then I can go back and print it. Now it's 15. I don't have to run the code from top to bottom every single time. That's the basic idea. And when we do machine learning or data analysis, data science, that is a very convenient uh setup. So let us get started by first importing Torch. So by importing Torch, we import PyTorch. And the first thing I want to show you is how to get information about your hardware. This can also show you if you set up everything correctly because you can also have some problems with CUDA. I'm not going to cover troubleshooting here because in my case, it just works. But Google and your favorite LM is going to help you with the different uh issues that you can have here with CUDA. But in order to see if you have even CUDA GPUs available, what you can do is you can say torch.cuda.is_available and that is going to give you a boolean which tells you whether CUDA is available or not. So whether you have a CUDA compatible GPU what you can also do is you can see how many CUDA devices you have. This can be useful if you are on a server that maybe has three GPUs on it. Then you can say here torch.cuda.device device_count. In my case, that's going to give me one because I only have a single GPU. And then I can also say torch.cuda get device name and the index. Since I only have one, the index is going to be zero for the first GPU. And in my case, that's an Nvidia GeForce RTX 3060Ti. So that's 8 GB of VRAM. And if you want to actually get the device, not just the device name, you can also say torch cuda device zero or actually I'm not sure if it's device or get device. Yeah, it's actually device. So this gives you the actual device which is my GPU. So that is the GPU part. Now let us create a heading here using markdown just so we have a new section. I'm going to say now here tensors. Tensor is basically just a word for a highdimensional array. And we can use numpy to do basically what we also do with PyTorch here. But numpy only works on the CPU. So if I say import numpy as np, I can go ahead and define an array to be equal to np array. And I can then say I want to have a two-dimensional array 1 2 3 and 4 5 6. That is the numpy way of doing things. Then I can look at that array. It's as simple as that. Now with PyTorch I can do the exact same thing. I can say tensor is equal to torch.tensor and I can pass the same list. So I can say 1 2 3 and then 456 and then I can take a look at this tensor. The idea is the same. Now in this case we automatically have floats here. This is also very uh important because when we do machine learning stuff when we do any derivative stuff or differentiation stuff we're going to work with floats here. But essentially it works like a numpy array. So in numpy what I can do is I can say array x* 5 and I will get all the values multiplied by five. So this is elementwise multiplication. I can also do the same thing with a tensor. I can say tensor x* 5 and I have the same effect in numpy. I can say arrays sum and I can also do the same thing with a tensor here. Tensor dots sum. So for the most part we can basically do with tensors with torch tensors the same things that we can do with numpy arrays. In fact we can also create tensors from numpy arrays. So I can say tensor is equal to and then torch from numpy and I can pass my numpy array here and that is going to result in the same tensor. This time we have integers just because we explicitly created it from an integer array. But that is also what you can do here. And what you will also notice is that there are many numpy functions or methods that you like to use. For example, np1s by passing a shape here. Uh 2 * 4 for example, that gives you an array full of ones with 2 * 4 elements. I can also do the same thing with torch. Torch ones and then two four will have the same effect. And same with zeros. And with random we also have I think it's np random random and then we can pass 24. And with torch, I think it's just torch dot rant. And then we pass the shape to four. And then we also have random numbers. And of course, we also have the attribute. So I can have an array and I can get the shape of this array. And I can get the data type of this array. And I can also see what device this array is on. And I can also do the same thing with a tensor. Tensor shape, tensor dype, and tensor device. But where you start to see the difference is when you try to put this onto the GPU. So if I say array.2 device and I say CUDA, it will tell me unsupported device CUDA only CPU is accepted because numpy doesn't support GPU parallelism. However, if I go and say tensor 2, now here we don't use two device, we just use two. and I say CUDA then you're going to see that now this is on my GPU this tensor now works on my GPU whatever I do will happen on the GPU and of course I can do the same calculations now actually I need to save this if I want this to be on a GPU I need to assign it back to tensor so tensor is equal to the tensor on the GPU and now I can still do the same stuff as before and we always see devices CUDA now I can also get it back from the GPU to the CPU and this is also necessary if I want to let's say convert it back to a numpy array because I want to do something else with it that is now outside of the pietorch world. So if I say tensor.numpy I will get a problem because it tells me it cannot convert a CUDA zero device type tensor to numpy. What we need to do first is we need to say CPU to get it back to the CPU and then numpy and now we have a numpy array from the tensor. And one thing that you will always see in PyTorch is that not just the tensors but also the models are sent to the GPU. And often times we don't know who's going to be using our script. So what we often times like to do is we like to define a device to be equal to CUDA if torch. CUDA is available the function that we that we looked at in the beginning and otherwise CPU. And then we just do dot to device. Whenever we have a model or a tensor, we just send it to the same device, which of course is necessary because if you have a tensor on one device and the model on another device, they cannot work together. But let us now get rid of all this numpy stuff and let's move on to the automatic differentiation. So what is actually happening when we're doing back propagation, when we're doing an optimization step? The magic behind PyTorch is the automatic differentiation. So let's say I have a tensor here a torch.tensor and let's say in here I'm going to have uh the values let's say two and three and I'm also going to add a special keyword argument here requires gradient. So requires grad is equal to true. Now I will do the same thing for a tensor called B. And here we're going to say six and four for example. And now the idea is that I'm going to come up with a formula. Let's say I have a function that is going to be called f which is going to be 3 * a to the power of 3 for example minus b to the^ of 2. So that is basically our function and I also added some markdown here to show you what it looks like in the math syntax. We have f being equal to 3 * a ^ 3 minus b ^ 2. Now if you take the partial derivative of f with respect to a you're going to get 9 a^ 2 just because we take the 3 down. So it becomes 3 * 3 * a ^ of 2 and here it becomes -2b. These are the derivatives and we can do this by hand of course but pietorch can do it for us with the automatic differentiation engine. So if we take a look at f we're going to see it has concrete values -12 and 65. This is just a result of taking 3 * and then 2 ^ 3 - 6 ^ 2 that is going to be -12. And this number is the result of taking 3 * um 3 ^ 3 minus 4 ^ 2 that is going to be 65. And now what we can do is we can do the differentiation. So we can back propagate the gradient so to say and we do that by saying f.backward backward. This is the function here. And we can provide a gradient which is going to be a torch tensor. And this torch tensor here will just say how important the individual um values are. So I'm just going to pass one one to treat both positions equally. That is just something that we need to pass here. And that is going to now back propagate the gradient. So I can actually go into a and print the gradient. That is going to be 36 and 81. So that is going to be the gradient with respect to a. And we can also do some math here. So this is supposed to be 9 a^ 2. So we can check what happens if I say 9 * a to the power of 2. We can see it results in the exact same thing. Same thing for b. If I print b.gradient that is going to give me certain values. And if I say what was it 2 * b that is also going to give me the same values here. If I may for a second, I would like to plug myself in as the sponsor of my own video. If you go to my website neurall9.com, you will find a tab services and a tab tutoring. Here you can hire me for all sorts of stuff like data science, machine learning, web development. If you need help with something in a project, here you can book me for one-on-one tutoring. If you want me to teach you personally something that you don't understand, if you like my teaching style, on both pages at the bottom, you can contact me via mail and also via LinkedIn. Just wanted to let you know about this. So this is just a peak behind the scenes. Now we're going to move on to the actual deep learning stuff to the actual neural network stuff. And for that we're going to do a couple of more imports. We're going to say import torch.n for neural networks snn. This is just an alias. Then also import torchn functional sf capital f. This is just what people usually do. These are the conventions. And then we're also going to say import torch.optim as optim. Now we're going to use nn for basically everything for all the layers and all the neural network stuff. Functional is going to be used for activation functions. These are functions that are applied after the layers are applied to break linearity. I also have a video where I explain in detail mathematically why this is needed. And optim is for the optimizers that we're going to use. In addition to this, we're also going to need some utils to load data. So from torch.youutils dot data we're going to import the data loader and also the tensor data set. So that is everything we need from pietorch. But we also need the data sets themselves. In my case now I'm going to take them from scikitlearn. So I'm going to say from sklearn.data sets. If I get some autocomp completion here I don't get it. Okay. We're going to load breast cancer. We're going to use load breast cancer. Then from sklearn.mmodel selection I want to import the train tests split function to easily split into a training set and a testing set. And I also want to use from pre-processing here the standard scaler. So these are the imports and we're going to start by getting the data set from scikitlearn. Then we're going to make sure it fits into the pietorch world. We're going to transform it into a tensor data set into a data loader as well. We're going to use a data loader, but we're going to start by saying that the X data and the Y data is going to be load breast cancer data set with the keyword argument return XY equal to true. This basically just means that we get numpy arrays X and Y that contain a bunch of information about a tumor and Y is going to be zero or one depending on whether this is a malignant or benign tumor. So whether it's cancer or not. uh then we're going to split this into a training set and a testing set. So we're going to use one portion of the data for training the model and then we're going to use the test data set to evaluate the model on unseen data so it's not biased. And then we're going to do this like this. So xrain x test y train y test is the order train test split on x and y with a test size of 20%. So 0.2. This means we're going to use 20% of the data for evaluation, 80% of the data for testing. And what we also need to do since neural networks are scale sensitive, we need to scale the data. So xrain scaled is going to be equal to or actually first of all I need to create a scalar. So scaler is equal to standard scaler and then scalar.fit transform xrain to have the scaled x data. And then we're going to say test scaled is going to be just transform. So we're not going to fit transform again. We want to use the parameters that are learned from scaling the training data set in order to scale the test data set because that is a realistic scenario. So we're going to run this. This is now our data our scaled data. We can take a look at it. X train scaled looks like this. just a numpy array and y train is just 0 or one depending on whether it's cancer or not. And now we want to take this and turn it into tensors and then also into a tensor data set and then we want to use a data loader to easily load the data for our training process. So we're going to use something that we already know about. We're going to say x trained tensor is going to be equal to torch from numpy and then xrain scaled is the array that we're going to base this on. And then we want to make sure that this is converted to float for the training. So dotflat in the end. We're going to do the exact same thing for the testing data. So x test scale tensor is going to be the same thing from x test scaled. And now for the Y data we need to add an additional dimension just because PyTorch expects a certain shape. So we need to say Y train tensor is going to be equal to torch from numpy and then Y train. But then we also need to make sure that this is actually unsqueezed. So there's squeeze to reduce dimensionality and unsqueeze to add additional dimensions. So we're going to do unsqueeze here uh one dimension and also we're going to transform to float before that. And same thing is going to be done with the test uh numpy array here. So run all of this and now we can easily create our train data set. I can say train data set is a tensor data set which is going to be created by using the xrain scaled tensor and the y train tensor. And now finally we can turn this into a loader. So I can say train loader is equal to data loader from the train data set. We can specify a batch size. Now if you don't know what to go with, we can take a look at the dimensionality of our data. I can say X train scale tensor shape. We have 114. Actually, this is test. So let's go with train. We have 455 entries. So maybe let's go with something like 32. And we're going to shuffle it for the randomness effect. So that is how we create the data loader. And now we can just iterate over the data by using the train loader as we're going to see in the training loop. But before we get to this, we need to first define our neural network. So here now we're going to create our neural network as a class by saying class BCNET for breast cancer inheriting from NN module. So this is what you have to inherit from if you want to create a neural network in PyTorch. We're going to define the constructor. So the init method here self and in this case we don't need to take any parameters besides that and we're going to call the super constructor. So super bcnet self and then dot_nit and now we can define our layers. So the first layer that I want to define here is a fully connected layer. That is the basic layer that you use if you don't have a reason to use something else to keep it simple. So self.fully Fully connected layer one is going to be an nn.linear layer. For those of you who know TensorFlow, that would be the dense layer. So this is just a fully connected simple dense linear layer. Now the input is going to be 30 because we have 30 features per instance. So 30 is the input. And now we can choose how many neurons we want to have, how many outputs we want to have for the next layer. Um let's go with 128 for example. Then I'm going to say the second fully connected layer is going to come afterwards. So it's going to take as input the output of the previous layer. So the 128 neurons and let's say here we want to uh produce 64. And then the third fully connected layer in this case to keep it simple is going to take 64 as input and it's going to produce our actual output which is of course going to be a single value zero or one or actually it's going to be a probability is this breast cancer or not how much is the likelihood or how high is the probability and maybe these numbers are a bit overkill so let's actually go with 64 and 32 I think it makes more sense here because the task is not very difficult so that is our architecture now but this is just the structure now we actually need to say what happens if we put data through this neural network and for this we're going to use the function called forward forward takes self as a parameter and some input I'm going to call it x because input is a reserved keyword in Python even though the pytorch docs always use input um and we're going to say that this input x is going to be used we're going to reassign the value here x is going to be selffully connected layer applied onto X. This basically means we take this input vector or matrix if we have a batch. We feed it through the network and we get the result of just taking the input and multiplying it with the weights with the weights of fully connected layer 1 adding the bias and then we get some output. Now in order to break linearity and as I said I have a video where I explain why this is necessary in an intuitive way but in order to break linearity we now need to apply a nonlinear activation function. And for this we can just say f do relue rectified linear unit. This now gives us a new value for x after we fed it through the first layer and an activation function. Then I can do the same thing with fully connected layer 2. And finally I can do the same thing with fully connected layer 3. But here I'm not actually going to use relu here. I'm going to use sigmoid because sigmoid produces a value between 0ero and one which is what we need because we're using or we want to produce a probability not just any value and after all this is done we can just return x. So that is the forward pass through our network. So let's create this and now we're going to define two things on the one hand the criterion. So our loss function and then also the optimizer. So our criterion is just going to be the binary cross entropy loss. This is what you always use if you just have a yes or no question. If you do binary classification, the binary cross entropy laws. We also have the cross entropy laws in general and also sparse categorical cross entropy. These are for uh for different types of classification. But today since we're using binary classification, I'm going to just go and say nn.bce loss, binary cross entropy loss. Now for the optimizer, we're going to use Adam. I'm going to say optimizer is equal to optim atom and I'm going to choose a learning rate of 0.001. So these are fairly standard things that I'm using here. We can of course also add regularization layers, dropout layers and so on. But I want to keep it simple. I just want to show you how to do a very basic use case here. So we just have dense layers, linear layers, fully connected layers, whatever you whatever you want to call them. And we apply reli to break linearity twice and then sigmoid to get a value between 0 and one. The loss function binary cross entropy loss the optimizer atom with a learning rate of 0.001. So this just tells us how uh how much we react to the changes. Uh now of course I forgot one important thing. We cannot just define atom. We need to define atom on something. And the something is going to be our model the parameters of our model. So for this we need to create it. I'm going to say model is equal to bcnet and then I'm going to say that the optimizer is optimizing the parameters of the model. And now we're almost there. What we're going to do now is we're going to define the training loop. So in TensorFlow what you do is you define the architecture and then you just do fit and evaluate. Here we actually need to define what fit means. We need to implement the training loop, the training logic manually. So I'm going to say epoch is equal to 20. We're going to do 20 iterations on the same data. And then I'm going to say for epoch in range epoch. We're going to put the model into training mode. So model train. We're going to also keep track of a running loss which is going to start at 0.0. And then we're going to iterate over our data loader to get the individual batches. So I'm going to say for Xbatch Y batch in train loader. These are now the batches. And what we're going to do is we're going to feed the data through our model get the prediction which in the beginning is going to be completely random. Then we're going to take the ground truth. So the actual true classification and we're going to calculate the loss. So how wrong is the model and then we're going to take that loss which is a function. We're going to back propagate the loss which means we're going to take the derivative of that function and we're going to try to find a minimum in that function a local minimum. Of course it would be great to find a global minimum but we usually end up with a decent local minimum. So I'm going to say here optimizer.0 zero gradients reset all the gradient calculations. Remove all the gradients and then we're going to say predictions is equal to model being applied onto X batch and the loss is our criterion when we compare the actual Y batch to whatever our model predicted. Actually, I think we need to do this the other way around to have the best practice convention here. Pretz and Ybatch. So, this is now just a raw loss. What we also need to do is we need to back propagate it. So loss.backward the function that we used before to do the automatic differentiation and then we need to take a step with the optimizer into that direction depending on how much the learning rate is. This is going to be a very large step or a small step but we say basically optimizer step and then to keep track of the loss we can also optionally add the loss to the running loss. So loss item and at the end of an epoch we can print epoch number epoch + one and that is going to then show us the loss loss was and then running loss and we want to of course divide that by the length of the train loader. All right so that is going to run very quickly. You can see the loss starts here and goes down very very quickly. And now we can just do the same thing again and define our evaluation logic. So how can we evaluate this model on unseen data? This is not that difficult. We can just say with torch dot no gradient. So no gradient calculations, no grat. Then we're going to put the model into eval mode again because we don't need gradient calculations. Then we're basically doing the same thing. We can say predictions is equal to the model being applied onto our entire testing data. So x test scaled tensor and then we have the loss which is going to be the criterion applied onto the predictions compared to the y test tensor. Now to get an accuracy score which is more interesting than just the loss we can also say accuracy is equal to and then we can define all the values where we have above or equal to 0.5. We're going to consider them a one. So I'm going to say where the predictions are greater than or equal to to 0.5. If those are also one in the uh actual ground truth. If that is the case then we're going to consider this to be uh correct to be a match. And so we're going to take this float and to get it across all the samples we're going to say dot mean and then dot item. So we can run this and then I can print the accuracy. We get 98.2. 25% accuracy. So this is a very good performance and this is the architecture behind it. Now of course you can do a bunch of more things here. You can add more linear layers. You can add a dropout layer. So for that you would do self.ropout let's say drop out or actually the dropout would be somewhere here is equal to nn dropout with a probability of 0.5 for example. And then you could also apply this let's say in between here. So x is equal to self.ropout onto x. And in this case now I can run this too. We can go through the whole process again. We have more regularization. We even get better accuracy because it doesn't overfit the training data. We get 99.12% accuracy. So yeah this is the very very basic crash course very quick crash course on PyTorch. We can go a lot deeper into that. If you want to have a full tutorial series where I actually make multiple episodes, then I can cut them together to a long form course. Or if you want to have an advanced crash course for PyTorch, let me know in the comment section down below. But that is a good start. So that's it for this video today. I hope you enjoyed it and I hope you learned something. If so, let me know by hitting a like button and leaving a comment in the comment section down below. Also, in case you're interested, on my website, you will find a services tab and a tutoring tab. that you can contact me if you need help with a project, if you need help from a freelancer, or if you need something explained to you one-on-one, you can contact me at the bottom of the pages using LinkedIn or via email. Besides that, don't forget to subscribe to this channel and hit the notification bell to not miss a single future video for free. Other than that, thank you much for watching. See you in the next video and bye.