CuPy is NumPy on The GPU

Key Takeaways

what is going on guys welcome back in this video today we're going to take a look at CPI which is a python package that allows us to use numpy on the GPU and thus allows us to speed up certain calculations so let us get right into it [Music] not all right so we're going to take a look at Cai in this video today now as I already mentioned coui is a python library that allows us to use numpy on the GPU and it can lead to certain performance increases to speed ups when it comes to certain calculations and operations it's based on Cuda and what I want to do in this video today is I want to show you how to use it how to use cpai on a basic level how to use it like numpy how to take numpy arrays and move them through the GPU do calculations move them back to the CPU and then I want to move on to the performance comparison I want to show you how much faster cpai is in certain situations but also that it's not necessarily always faster especially when you consider all parts that are involved in calculations so we're going to explore that you can see I have a bunch of images here and visualization scripts that I ran before recording this video I want to explore the results there but first of all I want to show you how to use cpy so the first thing is we want to open up a command line and we want to install cpai so pip or pip 3 install cpy you can either do it like this or you can say install ci- Cuda and a specific version so ci- Cuda 11x for example or cy- Cuda 12x now interestingly on my system I'm using Cuda 12 but I still have to use Cuda 11x I don't know why but the package doesn't work with Cuda 12x so I don't know why that's the case the best thing is you try out for yourself which one works either just cpai cpai Cuda 11 cpai Cuda 12 you can see what Cuda version you have by running the nvidia-smi command at least on um Linux you can see I have Cuda 12 but I still use C CI Cuda 11 because the other one doesn't work now once you have CPI installed you can import it by saying import CI as CP which is quite similar to numpy as NP and the interesting thing about CPI is you can basically use it like numpy so you can go ahead and say array equals cp. array and that's the same thing sorry that's the same thing as running num. array or np. array and if you have autoc completion you can type CP Dot and you can see that basically you have all the numpy functions available so that's one way to use uh Cy by just using it like numpy another way to use it is to take a numpy array like this one here and move it to the GPU turn it into a cpai array so to say by using the as array function so I can say C array or actually let's call it GPU array is equal to CP as array array that basically moves the array to the GPU and you can do then matrix multiplication or whatever you want to do and then you can take the result whatever that result is which is going to be a GPU array and we can turn it back to a CPU array by saying whatever the array is as numpy so or actually CP as numpy and we pass the GPU array in here that turns it back into an umpi array and uh moves it back to the CPU that is already what you need to know about this the rest is going to be a performance comparison so what I have here is three visualization scripts which are slightly different uh it's not too important to read through all of the code if you don't if you don't want to uh but basically what we're doing here is we're generating random numpy arrays um of a pretty high dimension so these are matrices n * n uh we start with a th we go up until 11,000 with a step size of a th000 so we do 1,000 2,000 3,000 and so on um and what we do then is we move these arrays to the GPU we perform a matrix multiplication using the dot method here or the dot function uh we do this one time with npy and one time with qy and then we compare the time it takes to do that now what you probably noticed here is this warm-up function now this warmup function I have my Mals open up great uh this warmup function is basically a very simple operation just to get CPI going because the first time you use cie in a script it's going to take some time to initialize if you don't do a warm-up um to initialize coupie you're going to skew the results so I'm going to show you the result of this this is what you get if you do the warmup you have the numpy time or actually the numpy time is the the one so how long it takes in seconds to do a matrix multiplication with these Dimensions here or with this size here uh you can see that it grows quite a bit and we have more than 8 seconds when we get to the 10,000 size um and for cpai it's basically 0 seconds almost so it's instantaneous uh doesn't matter what size we have here it's very very very fast um if I don't do the warmup you're going to see something like this so it's basically the same result but the first time we do it it takes some time to initialize like around 1 second and because of that it seems in the beginning that numpy is faster but this is only because of the initialization the very first time you run it and then it basically doesn't have this overhead anymore and you get the same result as before um another thing here is or one thing that you might want to consider is in the first visualization here uh what I do is I convert the numpy array to to a coupie array but I don't consider that as work so moving the array from the CPU to the GPU is not measured um In the comparison here which of course can be a problem because it's not faster if it takes super long to take a npy array and move it to the GPU to turn it into a coupy array so what I did then is I included that as the measurement here so I did the same multiplication uh or I did the same matrix multiplication but I then also included the time um that it takes to get the two arrays to the GPU and get them back from the GPU to the CPU and when you look at that you can see that the speed up is actually not there anymore because cpai is now slower than numpy because of this uh moving from the CPU to the GPU and back however this is only the case if you don't do a lot of work on the GPU so for example if of course if the time that it takes to move an array or multip AR raay from the CPU to the GPU is longer than the work itself then it doesn't make a lot of sense but if you have to do a lot of work if you do a lot of complicated things on the GPU and the transmission of the result is just one array being transmitted it can be faster so if you have a lot of work to do on a GPU and very little data to transmit it makes sense to use cie and this is what you can see now in the last image here after a certain size with a certain amount of calculations cpai is still way faster than numpy and the higher the problem size uh the less you have to transmit the less you have to convert from CPU to GPU um the more important this effect is going to be so the faster coupon is going to be if you do a lot of work on the GPU but that is basically what cpai does is it allows you to use the GPU for numpy operations specifically for Matrix multiplications this is very very useful because the gpus are optimized for that because of parallel Computing um and and yeah this is something that you can use intelligently to speed up your numpy calculations so that's it for today's video I hope you enjoyed it and hope you learned something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye