Data Science Project from Scratch - Part 1 (Project Planning)

Key Takeaways

hello everyone that can hear if you're new to my channel I'm a data scientist that produces data science and sports analytics content today I'd like to introduce you to a new series that I've been working on as you probably know I think that data science projects look great to potential employers and are the best possible way to learn the field I've been getting quite a few requests for me to do a basic project from scratch you guys asked and I listened in this series I'll go through a data science project from start to finish this includes the planning phase which is idea generation data collection data cleaning exploratory data analysis model building and the production process like actually putting the code into production I'm gonna try to do a minimal editing so you can see the areas where I get stuck and how I work through them I'm also going to build out a github repo and keep it up keep it up to date you're welcome to follow along but I recommend you trying to build your own project by loosely following the steps rather than copying mine if you enjoy this video and would like to see more similar to it please like subscribe and turn on notifications also please let me know if I can improve this in any way in the comment section below okay let's start the data science project planning process first you have to come up with an idea which can be very difficult there are generally two approaches to coming up with a good idea you can either look first at what data is available and then do a project on what you find or you can start with the problem that you want to solve and then go out and collect or find the relevant data I'll leave their brainstorm problems I'd like to solve or I'll read about them and try to apply them for the second the second way of finding a project I personally think that finding our own data for a problem that you're interested in is a more interesting approach in general and also for employers still let's quickly go through both methodologies so for the first style you can basically go through kaggle data sets or Google data sets and find something that's interesting you can generally just download these data sets and bordel and skip the data collection phase so let's do that real quickly I'm going to log on to cattle here so and then I'm already logged in you'll just have to login for your own thing you can go to data here you can search by the hottest data sets so obviously the the pandemic is is front of mine right now and there's a lot of data sets associated to that so that could be a really potentially interesting project you know we're looking at hospital beds air pollution really a lot of interesting stuff there obviously I like sports so let's look at sports here data set filtering so let's look at Olympic sports medals that could be pretty interesting and we can see a sample of the data here if I want to actually just use this I would click download and then I could get started in my in my Python notebook so again that's a really quick and fast way to get data they have a lot of good data on kaggle but for the intents and purposes of this project I think I'd like to actually you know go in and collect my own data so for this project I want to come up with my own problem to solve and I also want to scrape my own data I come up with project ideas by choosing a few high level areas and then whittling them down to more specific outcomes let's go through kind of a whiteboard brainstorming session now real quickly alright so let's kind of go through a brainstorming process for data science projects here real quickly I really recommend that you doing project that you do projects that you're you're interested in and in terms of the subject matter or that a potential audience is interested in so you know if you want to get a job in agriculture you know working as a data scientist you're probably well off to do projects related to agriculture for your portfolio that would probably really help you out so for my use case I'm always interested in sports I haven't done anything on tennis and that would be an interesting learning experience for me I also haven't done anything on soccer so that could be interesting I think it's kind of meta to do data science projects on data science so we could look at like some career to career type stuff resume like types of positions that's pretty promising there's also like a pandemic going on and we could try and understand that better I don't really want to do it on the pandemic I think that that's a little bit grim for this video sports I do a lot of stuff on sports and I kind of want to mix it up and I think that something that would give the most value to you guys is if I were to do in actual projects that can help you understand at the data science career landscape better so let's kind of dive further into this one here so something that I personally think is relevant is that you know data science positions are different across the country and that's generally reflected in salary so maybe we could try and predict the salary of data science positions based on the like maybe the job title like some info about the company and geography cool I think that that's something that data probably shouldn't be too hard to find and that would be meaningful because we could probably build a cool project or a product where you plug in that information and it spits up maybe a range of salaries so if you're trying to negotiate you're trying to do something like that that would be a cool tool that would be useful to you okay so after some whiteboard chicken scratch we've now got a project idea so in implementing this project idea we next have to figure out what data we want to collect I know that there's probably some salary data and some company data on LinkedIn and Glassdoor so those are probably the places we're gonna look first to to try and collect data from in my next video I'll look in how to actually scrape that data and I'll try and implement it for you guys please stay tuned for the second installment and good luck on your day science journey