How to create free GPU-powered ML apps in Python

Key Takeaways

hi this is Patrick from assembly Ai and in this video you will learn how you can build your own machine learning app that is powered with a free GPU by using flask a Google collab and ngrok in the video we built a concrete example with stable diffusion so in the end you have your own stable diffusion app that can generate images from prompts the approach is pretty simple we built a flask app then run it in the Google collab and use engrock to expose our app to the public and get a public URL that we can use so let's get started first let's Implement our flask app and we can do this on our machine so here we implement this main.pi file so let's import all the functions we want from flask and now the new thing is to also use flask ngrok and import run with engrock then we import the functions we need for our model in this case we use pytorch and the diffusers library and then we have some more utilities and now we can start loading our model in this case we set up the stable diffusion pipeline and then we can push this to the Cuda device since we can use the free GPU in the Google collab then let's create our flask app so this is like we normally create our flask app and the only new command we need here is to call run with engrock and pass the app to this function and this is the only new line you will need for your flask app then we have to set up some endpoints so we here we have our home M route where we render a template I already prepared this here so you can find this on GitHub the link is in the description and then we need another endpoint with a post request so here we can submit our image or we submit the prompt and generate the image so here first let's get the prompt then we can call our Pipeline and get an image back then we can save this to a buffer and then we need to convert this to base64 and then we can again display this so we pass this to our template and and this is all that we need so now the last thing is to call app.run and this is the full main.pi file that you need so again the only new command that we need here is this run with engrock and also make sure to run your model on the GPU next I recommend to add all the dependencies to a requirements txt file and here make sure to include pi endrock and flask androck and then you can push everything to a GitHub repository and then you can access this GitHub repository from your Google collab so I already did this here I prepared this repository with the code that I just showed you and all the other files now let's run this code inside a Google collab so let's open a new collab and here I already prepared this these are all the commands that you need so let's go over them very quickly first we say git clone and then clone this repository here that we created so that we can access all the files then we change into this directory then we install all the requirements then make sure to also set your ngrok authentication token you can get this for free at ndrock.com and then the last thing is to call python main.pi then also make sure to set the runtime type to a GPU you can get one for free and then the last thing is to say run all and now we can run everything and now this should work so let's wait until this is done so now all cells are executed and as you can see the last cell is still running because our flask app is running all the time and you should see this output here so our app is running on the localhost but also you should see an additional URL that ends with ngrok.io this is a public URL where we can use our app so let's open this and here you get a quick warning but we can simply continue to visit the site and here we have our app up and running so this is what we implemented here we can pass a command so let's say Iron Man in Hawaii for example and then click on Create and this should take approximately 10 seconds and if we head to the notebook then you see our endpoint was hit and here it says generating an image of Iron Man in Hawaii and now let's wait until this is done and the image was generated so if we switch back we should see this and as you can see that the fusion model worked so here we see the image and yeah this is how we can run a flask app with a free GPU inside a Google collab and then access this from a public URL and that's it I hope you enjoyed this tutorial and then I hope to see you next time bye