Automate Machine Learning with ChatGPT

Key Takeaways

in this video I'm going to show you how you can automate machine learning using chat GPT and python now let's imagine you're working as a data scientist and a bike rental company ask you to create a prediction model that can for any given day predict how many bikes are going to be rented out now this is a typical problem that you can solve using machine learning and in this video I will show you how we can set this up in Fierce code using a data set that I will provide to you as well link is in the description and chat TPT for generating the code to basically make our workflow really fast okay so let's Now set up officialstudio code workspace that we can use to start writing python code we first download the data set and put it into our directory then if you're new to working with vs code and data science you can check out these videos by following that you will have the exact same setup that I will be using then in Visual Studio code I make sure that the data set is on the data raw the next step is to open up an empty python file called make dataset that is in Source data next make sure that you select the proper python environment that you can use for this project okay with the setup completed we are going to follow the data science lifecycle to create a prediction model and along the way we'll be asking chatgpt to write code for the specific elements that we need Okay so let's get started so for this business to understanding we already know because of the problem definition over here we want to create a prediction model to optimize Revenue then second understanding the data and for this we're going to ask chatgpt to import our data set so I've given the problem statement to give it some context and I'm also going to ask to read the data and I specify the path and then also asking to write python codes to load it into a pandas data frame okay so we get a nice piece of code there even with some examples of how we can tweak it but for now let's just copy and paste this into a file and see if it works so I'm going to fire up an interactive python session over here that is trying to load the data frame alright that is looking good and we can have a look at the data frame over here so that is great so now in order to get a better understanding of this data I'm going to take the columns from this data set and I'm going to copy and paste those and to get a better sense of what we have to do in the data preparations we need to understand the data a little more so let's just ask jgpt to explain what we're dealing with here so I'm just providing it with all the columns and I'm asking how we can use these columns to create a prediction model that can predict rentals okay so this is actually pretty amazing so it went ahead and created a trained test split and a linear regression model already so I was just asking it to explain the columns to get a sense of what we can do with this data but then it said okay so we can use certain columns to make predictions for rentals we have to create a train test split boom here's how you do it so let's just go ahead and copy this code as well and come back to official Studio code so I'm just going to put this over here don't change anything and import the linear regression any trained test split then it knows that our data frame is called DF and we we can split up our variables into an X and A Y so I think that looks really solid then we're going to create a train test splits this also works linear regression model train X and Y train on the model then we're going to test score the model and print the R2 score and boom we have a prediction it's not perfect but we have without changing a single character have a prediction model that can help us make predictions for the company now let's tweak it even further okay so typically in regression problems like this it's best practice to convert the categorical features to dummy variables so here you can see we have four numerical features those are fine as is but we want to convert the categorical features to dummies so I went ahead and asked chatgpt how we can do this I just copy and paste it all the categorical columns and then asked two categorical features and then create in Ponder's data frame it provided the code just perfect so here we can take this selection over here and then I'm going to insert that right at the top of my file save it to let black do its formatting and then let's just run this see if it runs okay so now if we look at the data frame dot info we should see that the columns we've provided are now as a category and then the next step is to convert them to dummies so this was all using one prompt from the last one and it also gave us the dummies so we can copy this as well and now come back and then store that into our data frame again and let's run this save it again and see what we currently have so we are now dealing with a data frame that has a lot of dummy variables based on all of the columns over here now in order to speed things up for the training process I'm going to copy and paste this and put it down here so we first load our data frame then we split our X variable then our Y and then I'm going to change the JF over here and change that to our X in this way and now we will apply everything to the X only and then using the get dummies we can convert our X variable to all the dummy variables while keeping our original data frame as is and this should enable us to create a train test split and now we can create the same linear regression model again try to fit train it score it and then print the result okay so we can see that we have an increase from the 66 R score that we just had and we are now at 73 so that is a nice increase but this is of course a very simple model let's now try and ask chat GPT if we can explore some more fancy models so let's come back to chat GPT okay so I said the regression model is too simple can you provide me with five algorithms and code to train evaluate and compare them okay so we got a couple of algorithms that we can choose from and this is actually really useful but but I want to go one step further and actually automate this process so I've asked Can you combine all code into one function that tests all the algorithms and save the scores to pick the best one in the end sure here's an example so here we can see that it Imports all the various models and then creates a loop and then scores everything so let's just see how it work how this works we can come back to the X and UI so with our X and Y defined we are going to get rid of this block of code over here and then copy and paste this we are going to put the Imports at the top let me quickly install XG boost all right that is also how you can install packages if you don't have them in your environment just do a quick pip install in the terminal and then we can come back to our algorithm over here basically our function let's just make sure that everything is nice and clean up here make sure to run everything and we also have some duplicates over here we don't need that so we have the import ports and now we have our compare algorithms function that takes the X and the Y let's just put this over here so make sure we run this and then we can run the algorithm over here so now it's training looping over everything and we can print everything and wow look at that so it's giving us scores but we have to check so it's using a mean squared error in this sense and then okay so it's trying to find the best model by minimizing the score then print best model boom random Force how awesome is that so with just a couple of prompts to chat GPT we have created all this code over here that can Loop over over various algorithms and compare the scores okay now let's take it one step further we know that the random Forest is the best model out of all the algorithms over here now let's try and tweak the hyper parameters to get an even better performance okay so now I'm going to ask it to optimize the random Force so I ask can you create a function to perform a grid search over the most important hyper parameters in order to tune the model use cross value relation 5 volt cross validation and use the mean squared error and the R2 as evaluation metrics so we can compare them both okay so here it goes tune random Forest it's giving us an estimated max depth also there's actually quite an extensive grid search and now it's going to apply the cross validation all right and we have another beautiful function over here so let's copy and paste this see how this works so we have some imports again that I'm going to put at the top and the random force regressor is already there let's just import the grid search Decay fold and the error metrics now I'm going to store this into memory and then we're going to tune the random Forest let's go okay and after running for a while the grid search completed and look at these results we have an R score of 97 over here so we are getting close to a perfect model so now let's take these best parameters and try and create the model make predictions and then visualize the results okay so I'm just going to ask it hey these are are the best parameters okay now create a random force model with these parameters create predictions for y and then visualize the result here it goes all right and it's finishing up it's now providing description of what it has just created and it looks really good so let's copy and paste this and see what we get so here underneath the last function we are going to fill in the code over here then come back and do a few more I think the only one is we need the mud plot lip in order to create the visualizations then come back over here and we have to split the data again because uh yeah we can still use the X and the Y and now let's run this using the best parameters let's save this and run it okay so these first two plots look really good but this is kind of messy but it's actually not jet gpt's fault because I know that the index here is probably messed up and when you get prediction shoot there is no index so we should be able to fix that by calling just the values and get rid of the index and then everything should line up correctly again so let's visualize that one more time and this is starting to look real good so here we can see a scatter plot so it created the scatter of the Y test any predictions and we can also see that now using the train test split our R score is a little lower so it performed better with the when we did the Decay fold cross validation but it's also why it's always good to create a proper train test split train on one and then test on the other at 20 in this case but we have a really solid model over here okay so I'm happy with how the model turned out and I think we can help our client with this so now let's ask check GPT what the next steps are to bring this model into production okay so now I ask how to export the model to a specific directory in my project and also what are the next step to put this into production alright awesome so we have some code over here that we can use to export the model so let's come back over here make sure the import is at the top and import the job lab Library and then come back over here and we are going to export the model so we're going to dump it and then it should be in here awesome and then we can also use the same model uh to load that again using the job lab library and now it recommends that we can use uh this model on a server and then start make predictions for the client this is actually really awesome now I'm putting this model on actual server might be something for a future video but this is how you automate machine learning using chat GPT and python now and this is pretty scary right because you are probably wondering like what's even the point of learning all this machine learning data science python if chat GPT can do this and this is only version three what happens when we are first in 10 what will happen but don't worry this is not going to replace you but it is going to drastically change how you approach work and not just for data science for every industry and I believe we're currently at a point in time where you're going to fall behind if you don't start to leverage AI for your work because someone else will Bill and that person is going to be 10 times faster than you look at what we've just created in only a couple of minutes and I'm now already fully utilizing the power of jet GPT within my work because I'm working as a freelance data scientist and that means the faster I can complete a project the more projects I can take on the more I earn within a given year so this is huge for me and of course for you as well so what I think you should do if you want to set yourself up for a successful career in data science and machine learning is you should focus on a holistic approach to solving business problems and you do this by getting a really good understanding of the data science lifecycle all the individual steps that you need because what you've seen in this video is my understanding of the data science lifecycle and projecting this projecting it onto this problem and then asking chat GPD to fill in the Nitty Gritty details so it's not so much more about understanding all the algorithms and really knowing the specifics of this and the syntax of the code but it's much more about understanding how to solve a problem as a whole and then asking the right questions to Ai and now in my opinion the only way to actually develop a sense of how you solve business problems using data is by actually getting your hands dirty and doing projects so not just following textbooks and reading up how all the algorithms work but actually getting to work coding and solving problems and now when I was studying data science and machine learning I found it very hard to find actually good examples of how to learn this so that's why I created this series here in this playlist free on YouTube where I uncover an entire machine learning project so if you're learning data science and machine learning I would highly recommend to go check that out next