I Built A Python Project WITHOUT Coding

Key Takeaways

dad nerds I built a data science project in under hour with chat BT Not only was it able to perform some in-depth analysis with SQL and python to generate some pretty insightful visualizations but it was also able to provide some deep insights into this analysis it even covered some things I was previously unaware of and I'm pretty blown away by this cuz I'm hearing all the time of people saying that chat gbt can't do their job so we're going to be walking through my step-by-step process so that way you can do the same Ed Luke here and unfortunately I'm here too early in the video I'm not saying that chat GPT is going to take your job I'm just saying that chat GPT can be a good co-pilot to your job all right let's continue oh and the project is listed below along with my new course on how to use chat gbt for data analytics which this video isn't sponsored but it is supported by those of you that have bought my course so thanks to those that support the Channel all right so let's get into the master plan of how we're going to actually do this the first part involves connecting chat gbt to my data source which is a big query database from there we're going to have it write the code to analyze the data chat gbt will then provide me with the insights that it finds and I can reprompt it as necessary to guide it on where to go next now I can hear the keyboard Warriors now complain about how this is a security concern connecting chat GPT to my database and well if you haven't heard the news chat GPT Enterprises solves this problem it meets the same security requirements of these Cloud providers so should be good enough for you back to the project after we gone through and extracted all the insights we need to move into showcasing the results and as good as they look in a chat window this really is is the best way to share work so one of the plugins that we're going to be using with chbt to perform all this analysis actually has the ability to export all these findings to GitHub and then have this beautiful project so the world can see our work where it captures all our different analysis and insights so what is the problem we're going to be solving with this project well let me show you I built this app data nerd. Tech that tells you what are the top skills of data nerds and it does this by looking at almost two million job postings that I'm pulling on a daily basis from around the world and then Aggregates what the top skills are and you can learn more about how I built this app in this video but we're continuing on so about that problem I can look at a specific job title such as data analyst in the United States and it tells me what are the top skills here we can see that at SQL and Excel additionally this app analyzes to find out what is the salary for a given skill so we can see here something like SQL it's around 990,000 middle of the pack whereas something like Excel is bottom of the pack so I'm not lie like most people I'm going to want more paying money so why wouldn't I go for something like Oracle here that's the highest paying skill well when we go back to the skill rankings we can see that Oracle is down here at the bottom at only around 8% which in my opinion it's pretty low odds that you're going to get a job that's going to pay that high but if we look at something like Python and Tableau which are the second and third highest paying we can see with the popularity of these that our chances of getting it are much higher so that's the problem I want to solve with Chad gbt helping to find out out what is a skill that is not only popular but also that pays well and doing this for not only data analysts but all data nerds so spend 3 minutes into this damn video we haven't still jumped into this problem so let's get to it the first thing we need to do is select the model so we can use GPT for the core model which has analysis or Advanced Data analysis built into it and this is great if I have some sort of data set that's a file such as in this case here where I have a free data set on J postings of data analyst in the United States you can get it from kaggle this would be great for this and I could download it and then upload it into chat gbt however going back to data nerd. Tech we can see we have over 2 million jobs which is like 8 gigs of data if I tried to upload this into chat gbt I'm going to get an error so instead we're going to be using a plugin specifically this plug-in right here notable let me show you what it does by starting this project so I prompted chat GPT to start a new notebook with notable and provides me this link to it and inside of it is nothing right now cuz we got to add to it so I can just prompt it with this of make a fancy visualization in Python and I'm seeing that it's using the notable plugin and I can scroll over here and actually watch the work it actually just generated this python code inside of the notebook and now it's running the cell on its own and we get this final visualization that's calling fancy bar chart all right cool this is what it actually does it runs and then has all your visualizations here and then if we go back to chat GPT it actually exports the results into it now the other great thing of notable besides actually just keeping all of our different Python codee and visualizations in one place that we can go to it's also that we can go in and add different data connections unlike Advanced Data analysis this can connect to a host of different databases in our case this big query database that I have on my job postings in it can connect to all I have to do is provide it this key data to access this database and the last last thing that I really enjoy using notable for all right this is sound like a damn sales pitch right now I am in no way sponsored by notable nor to have any affiliation with them I just really enjoy their product that they've made to integrate with Chad gbt and I'm trying to share with you so just want to iterate that all right let's get back to it and the last last thing that I really enjoy using notable for is being able to actually connect to a GitHub repository when I go into create a project it has the ability to clone from a repository and so I can go into GitHub create a new repository get this URL that this repository is in and then provide this along with another user access token that I'm getting from GitHub to then go in and clone this repository and now my new project inside of notable is connected to my repository inside of GitHub and we can see this by this read me has my new repository going into my new project into the read me we can see it's also saying my new repository so we've now connected notable to our big query database and also connected it to GitHub all we need to do now is go back into chat GPT and start analyzing data so now that we have that project established let's get through and have Chachi BT create a new notebook inside of this project and let's actually perform exploratory data analysis the first step that we need to take so after I've gone through and connected to that data nerds job connection that we previously established I've V promp chat gbt to go in and perform descriptive statistics on the numerical Columns of the data set basically I want to find out what's going on so we end up with this which shows us different insights into the mean median and the core tiles of the salary data but I want to visualize it and we get this beautiful visualization showing the salary distributions for data analysts data engineers and data scientists this is pretty good visually to show what is going on inside this data and if we go into notable to see what's going on behind the scenes to build this thing we can scroll up here and see that first it did a SQL query in order to extract out the information for these three different roles and then from there I use some pretty hefty python to actually go through and visualize those seel results so it's pretty awesome that you're able to do all of this with Chad gbt without having to write a single line of code yourself but let's move on by looking at some of the other columns in this data set that are non-numerical here we have a breakdown of all the different major job titles Within These job postings we can see from this we have a majority of those three that previously showed in the histogram of data analyst engineer and then scientist and then Senior roles and Engineering roles are a little bit less frequent in this let's do something similar for all the skills now in these job postings and we get this showing the top 10 skills along with their associate count in those job postings as expected SQL and python are some of the highest so doing a quick recap of what we've done for Eda so far we've been able to go through and analyze not only that salary data but also that job title data and all those skills now we need to move forward with actually solving this problem all right so we need a way to combine these three graphs we'll talk about this one in in a second for skills I'm not going to lie this one ain't too bad it's great at showing an order and magnitude from high to low now this one I haven't shown you yet it shows for the 10 most requested skills what are their median salaries engineering skills are at the top with analytical skills down at the bottom so what do you need to do when you need to combine two bar charts well you make it into a scatter plot the x-axis down here is going over what is the salary higher paying to the right and then the Y AIS is actually going into the job posting count how many jobs actually have an Associated skill to that paying salary diving into it we can see we have these two outliers up at the top that's you can probably gu us what they are yeah Python and SQL and with this we can see not only the salary associated with it but also the job posting count now it does have a purpose remember we want to get the most optimal skill so we want something that's high paying and also that has a high amount of requests just realized I'm giving two thumbs up so we want to be up in this right-hand quadrant if we can and we want to stay away from this area down here so things like smart sheet digital ocean and monday.com sorry you're not going to make the list we want to be focusing on skills like SQL python AWS Spar and even Scala now don't forget we also want to break this out for all the different data nerds so I took this scatter plot a step further and actually broke it down by these different job titles now this one's showing data analyst data scientists and data Engineers some insights from it are that data analysts are group down here at the bottom with lower paying jobs unfortunately data Engineers which are in red appear to have a lot more technical skills requested in it and then the green for data scientists are sort of just a mix between data analysts and data Engineers anyway this is really great doll but there's just one major problem most people don't know how to read a scatter plot specifically this one how are people supposed to know intuitively how to interpret which skills are the most optimal well here's what I'm thinking just by looking at these three different job tiles we can see there's big disparities between pay and skill count so the first thing we need to do is actually normalize these vales normalize these values between Z and one and so that way we have a standard metric across all the different job titles so way we can actually compare them for a most optimal comparison all right let's jump into Chachi BT to do this all right so that scatter plot that we built actually has a pretty good data frame behind it if we look at here this data frame has these Columns of skills and then the job title itself for each one of them it has what their median salary is and then also a salary count column or a count of the number of values this whole portion right here is how we got that scatter plot these two values are what we're going to be normalizing so I prompted Chad GPT with this we need to come up with some sort of grading mechanism that takes into account a high rate of job postings for a skill and a high salary let's call this thing a skill multiplier and assign it to every value that needs to be normalized based on that job title column now chachu BT got to work and actually it went through and it helped explain how it was going to do this first it talked about that normalized value and what formula you'll be using for that this is what's going to be setting a value between zero and one for both the skills and salary additionally we came out with that skill multiplier which is that normal skill count times that normalized median salary and now we end up with this data frame that has the original columns that we had before but also these three new columns on that skill multiplier and those normalized values anytime you do anything with chat GPT should double check that it made sure did it correctly and going through it I can see that it was as an example if we're sorting this salary from high to low as i' expect for data analyst if I went and check the normalized salary it's going to be a one value because that is the highest amount conversely if I went to the lowest paying data analyst so this one in this case of $50,000 I would expect to see Zero and also did this for the count and then verification of the skill multiplier everything worked out so I was able to get chat gbt to graph all the different job titles that we have and then rank it high to low and provide it in this bar graph now I want to just focus on the top three roles in data science and I can interpret it but I'm going to let chat gbt do it so I'm going to upload this plot to chat gbt and ask it to interpret it it was able to go through and actually extract out a lot of insights from this graph now you may be like Luke why didn't you just prompt it in your last chat with notable and actually ask it about the visualization more and unfortunately whenever you have Chad gbt with plugins enabled currently you can actually use chat BT's capabilities to see images so it came up with some pretty interesting insights that SQL is a crucial across all professions and python is also essential particularly for data scientists the other thing it points out that visualization tools like Tableau are important for analysts and scientist and then finally goes in Big Data Technologies like spark and dupe are more relevant for engineers and scientists now it also came with this final suggestion for data nerds says focus on Python and SQL regardless of the specific data role and then goes into each role in specif specifies what they should maybe hone on further to basically excel in their career and have the most efficient opportunities this pretty great cuz I was going to summarize this graph for you and chat gbt did it instead but me demonstrating that had a purpose I actually went through and provided chat gbt with each of those graphs that we previously generated and then had it from there summarize it in a markdown format and I did this for all the different graphs that we generated going back into notable in our readme file I then took all of those different markdown answers that provided along with those visualizations and put them inside of here so now this read me tells all the different work that we just did and basically overviews our entire project and then because we synced our GitHub repository with our notable project I now have a full data science project that details everything that we did inside of GitHub to actually share with the world and somebody can go in and potentially see for like our exploratory data analysis they could go in and see all our different python code behind actually generating these visualizations and running the code now one disclaimer that I highly recommend you put into this if you're using chat gbt to generate anything is putting a disclaimer that chat gbt generated this that you didn't do the code yourself and this project took me about an hour of work there's a lot of things that I went into in this project that I didn't show in this video to save some time but if you're curious about learning more about how I actually built this project fully then be sure to check out my course on chat gbt for data analytics this course bundles up all my best practices and saves me on average around 20 hours a week now I cover everything from Custom instructions to prompting best practices I even go into detail of how to build this project with stepbystep instructions I've been working on this course for the past year so I'm super excited to finally get a share it with you all right as always if you got value out of the video smash that like button with that I'll see you in the next one