Data Engineering on Google Cloud Certification Progress | Learning Intelligence 48

Key Takeaways

oh yeah check that out that was that was very strange actually I my headphones died as as I took them off so good timing all right we've passed part four of the Google Cloud ml data engineering certification this one the how we went through was all about how you deploy a machine learning model Ryan how do you get it into production that that is what I'm focusing on that environment is it's not not so much they want to do it now actually working on it we're doing it at maxsa Kelson long story short if you build a great machine learning model but it lives in Jupiter notebook it's not necessarily helping anyone what I personally want to do and what what probably a lot of you guys want to do as well I hope you should want to do this is get your code out there into the world and into into production what do I mean by into production that that could be anything right but it means it means to to get someone using it so like I don't have my iPhone on me because that's in my drawer because I've been staying so when you use your iPhone and you use an app or something chances are Facebook Instagram you know the common ones they all use machine learning but how cool would it be to have one of your own out there that that use machine learning to to to do something incredible like for me I've got an idea of using machine learning to help people learn about nutrition by taking photos of foods so I want to mean my brother have worked on something similar so there will be a future project but we're going to certificate on the way by the time this video goes up and probably be here so high five foot for the certificate we'll get it back up there we go how shiny does that look [Music] I'm gonna go take a break and then we're gonna get into pop live let's do it tonight sup brother what's happening so much what game is this it's like a knock off a little smash you know how much I love movement been breaking up study recently with with skipping I find if I do like a Pomodoro or something so like 25 minutes of study and come out outdoors usually it's a bit sunny out but it's it's late in the afternoon now it's I don't have a watch on it's like 4:30 5:00 p.m. or something 100 skips and then that I find gets everything flowing gets the brain going because you sit down too long and get a bit stagnant so you move around get bored jumping and then back into it now wonder if I can get a hundred straight I probably just change myself [Music] 20 all right heart rates up we'll go back in I'll get into part five do come for a Doros I got a few left with today I think we do 8 8 or 10 let's see how today goes see thing about machine learning models and she probably know they love data so the Titanic problem that we did had if out a thousand rows now what if your data said as close to a billion rows so a million times more well that we're the pandas with pandas right that's what we use for the Titanic one or that's what what gets commonly used with smaller data says the data frame or the table you work with stores in memory so right that the RAM of your of your computer now storing a billion rows in RAM might not be the best option so what can you do that's when you need distributed systems that's when you need cloud platform says we need giant machines help still still puff from those skips if you don't go to skip and get a skipping right pin that that that point there the more data thing was there's a simple one right but that's what I like about about this causes if it broke it into there was three steps to improve machine learning I've got my little notepad here and so number one was more data that's what the example I just had what if you had a billion rows of data and not just a thousand like our Titanic little baby sample number two is feature engineering so we didn't do any feature in engineering in the Titanic example but what is feature engineering I think the camera is slightly moving that's all right I'll just hold it future engineering how could you combine two features into one and what do I mean by feature imagine a column a column is a feature so imagine Titanic it's an easy example so you have a column for age cabin class what if you combined age and class so you had people who are let's say over around a 50 all right this is a rudimentary example over under 50 and in first class or second class so you've combined the age column with the class column now that's just one one example but that's another way to improve your machine learning models as if if the features you get don't sort of give you the best results you can create your own and then the final one is hyper parameter tuning so more data feature engineering and then hyper parameter tuning so if you do build a model and you want to test a whole bunch of different different settings on that model if you imagine if you imagine you've got a different combination of let's say learning rate let's say optimization rate let's say the number of of batch sizes that you use what if you wanted to test a whole bunch of different combinations right and that is where cloud and auto ml comes in or you can do like hyper rot which is a library fill high parameter tuning I mention that in the Titanic video research that searches a whole bunch of different parameters and now that's enough talking that's three ways to to to blend your machine learning goal to improve your machinery let's not blow them out that's what we wanted to improve [Music] more skipping oh yeah I think it started rain they better make this quick skipping in the pain I think it's a song about that skipping and singing in the rain even care miss me so far all right one more Pomodoro you notice how notice how it's stuffed up I lost concentration similarly when you study focus on the one thing that's why I said the timeline and don't do anything else except just got really dark really done back into the light and don't do anything except for this we're going to finish off this par 5 and then I think happy for today [Music] I forgot to turn the microphone on so there's no audio but we're gonna just essentially sum that up in a lot quicker better fashion of course of course five have been doing on on the Google cloud platform data engineering certification a mouthful has been on streaming data now you have your regular data which maybe say it's a data just in a box doesn't really change it stays in that box streaming data is like stuff going through a river so imagine YouTube or Spotify or or whatever or something or Google they always have data flowing through it's not just a it's not just a box full of data it's always getting updated like a river on it saying you say you put your your hand in the river you put it in again it's never the same river pretty sure that's is saying somewhere anyway what's important to know this well it's as data grows and grows and grows and sensors become more prevalent it's a problem that that has to be taken taken on right you're not always gonna have just a a box of data to work on to build great models off you're gonna have constant input of data now all this stuff I've been I've been learning on here like all different services like data flow and and pub/sub and Google bigquery and BigTable they're all new to me right so I'm kind of kind of like still like what do I do here so if someone was to ask me could I build up into n pipeline with it not yet but I'd probably know where to look I know where to find out and that that segues into a little question I got I'll answer or answer this to wrap up the video I think that's enough study today someone asked reach out to me when two things when do you start to understand things and he's been coding Python for a couple of months but struggling a bit with some of the concepts well I'll I'll I'll come straight with that I'm I'm in the same boat right I've been coding for 18 months two years or let's just say eighteen months whatever the timeframe doesn't really matter because I've spoken to people who were coded for 10 years and still look up basic things you'll find blog post of people who are experts in their in their field one of the some of the highest quality developers and they say they'll say are the honest ones the honest ones will say they look stuff up every day that on stackoverflow trying to piece together things and that's that's what I do - that's where I'm at with this Google cloud platform services because it's still all new I mean there's so much out there right there's always a new framework a new way of doing things what I think is important is to get some foundation skills and never lose that curiosity or that eagerness to try and figure something out right yeah that's that's the most important part because when you first start out it's gonna be frustrating I go through it I go through it every day you'll have failures the other day I failed at committing some github code and/or some code to github and lost 500 lines of Python code so yeah sorry Michael that was my colleague we were working on that with so we all have failures we will have struggles but we learn and we figure out how to have a look for the right questions not necessarily the right answer look for the right questions of where to go next so if you're struggling with with coding or with data signs with any of these concepts trust me I'm in the same boat that's why I swam I'm here studying every day I'm studying along with you try and try and best the best we can to learn but I'm gonna go do a work out in the backyard and have some dinner and then get back into this tomorrow keep learning team see you soon see I'm trying to say see you [Music] brain operates too fast man Kalin and I say keep the keep moving see you next week [Applause] [Music]