Data Engineering on Google Cloud Platform Specialization Progress | Learning Intelligence 47

Key Takeaways

[Music] wrote to our first lab of the Google cloud data engineering certification otherwise known as service data analysis with big Grinnell you're down there I mean I mean standing up let's let's come down to the same level Seth like the standing this needs have been toiling now first of all I want to give a big shout out to whoa hello it up here this is not where I'm supposed to be I'll be back later but cool Sarah for sponsoring this video you know I love Coursera because they partner with people like Google just have a look at this right got that little Coursera you see that Coursera for Google symbol they're going through a server less data sip what's it called service data analysis with bigquery they haven't said it before I can't remember it here now what exactly is bigquery well according to the formal definition it's an ERV ops data warehousing and analytics now I just read that because I pure Lea copied copied it down but to me I understand it as it's a place where you can if you have a big amount of data you can set it up to the cloud and if you want to store it and leave it there in a safe manner you can it's it gets cheaper over time as you store it there for a longer period of time because Google do something smart with their their storage - I mean if you're not accessing all the time you don't have to use that electricity and all that sort of stuff so storage gets cheaper over time but it means that if you want to do it do analysis on that like run some queries on it which is literally why it's called big query you can do it fast and you can do it effectively with SQL if sequel sequel like queries what's why is that so handy well the fact is that it's service right and to me I don't know anything other than data warehousing that's service which which means living in the cloud you don't have to necessarily boot up a computer to run stuff on it whereas in the part you'd have to have your own series just going crazy anyway in the past my phone should be in the drawer that's a note to self right keep your phone in the drawer in the past you'd have to have this big computer on site and run all this data stuff you'd have three different people to do this to do a job that you and I can do right now in in uh in the bedroom I guess but we're gonna jump into the first lab now with this with this course is the reason why I love it is because is structured labs it actually shows you what to do if you actually want to start running it on your own data so I'm gonna I'm plan I'm gonna get three labs done there's six labs in total three labs and then break and then we'll do another three laps [Music] alright we just did our first it's really dark in here I think there's a storm brewing we'll get to that in a second actually there's enough light for this we just finished our first quiz on the the Google certification data engineering we've we've been through three labs so we've done the goal we've done goal for now it's it's it's break time I think but lab one you can see was building building it I think that should be and and building a bigquery query lab was loading and exporting data so how do you how do you get data from from your computer to to upload it to the bigquery you can do it I the two methods we went through was the command-line interface of terminal and then otherwise just that regular upload button and then it was a lab on advanced capabilities so advanced SQL queries I'm finding SQL SQL seems a bit like a bit like Python and Pam is right but it's it's it's more to me seems no I don't want to say simple simples not the right word for it but it's it's like you're literally going select this from here where this and groupid by that so I mean it's similar and there's a saying when I went to Bali once like when you when you go over there and you buy like a t-shirt but it's a it says it's it's a supreme t-shirt but it's not supreme they would say it's same same but different that's what I feel our sequel is I think we've got a storm coming check it out that sky is looking gray looking dark the rains are coming or actually the rains a rains are carne already here seven you got a new toy there you go to tour their - bill we were gonna come play in the backyard but I guess since it's raining I still work out inside yeah that's a good idea now went for a bike ride earlier this morning in the Sun that was amazing just finished riding right not riding with riding now now we're up to got BJJ training and the Google cloud certification on Coursera it's a big day so it doesn't physically say oh yeah a BJJ training I got a tournament in like ten days which I'm really excited for but I like to break up study with movement so I've been sitting down for a bit now it's time to move body the party's not moving - it'll be something nice and simple we'll start off five minutes of stretching and then we're gonna do a hundred push-ups and a hundred squats and then we'll get back into there are there three labs that we have to do that's Carl we'll start off with one of my favorite ones for the hip no sitting down for so long studying all day sitting on a computer screen hips can get real too high and that's that's not a good time so we want to do is get one leg back like this a leg up forward hands shoulder-width apart and then you're just gonna straighten your back leg thrust your hips towards the ground all right and if you really want to step it up you can just hold it here and go down like this right keeping that the same posture or what I like to do is give upper back stretch as well looking to do this for five five each side should feel it all through the hip all through the leg really make an effort to keep that back leg straight alright for this one we're doing a posing movement to what you usually do when you sit down right what do you usually do usually sitting at a computer desk hunched over like this alright so spine is curved and the shoulders around and forward so this one is to to counteract that I'm gonna come in just like this you can start on your knees if you want well I prefer start flat on your stomach and we're just gonna pretend like we're on a surfboard keep your hips on the ground push up like this so now we're doing the reverse of that curve that happens when you're sitting at a computer desk for a long period of time I'm just gonna hold this in then make the arms nice and straight spread the fingers and hold it there about twenty or thirty seconds or so just to feel that stretch should feel it right through the spine right through the hips all right push up the squat let's do it we call this one the lounge room workout because you can do it any way so do these hundred squats and a hundred push-ups go do 20 push-ups and straight into 20 squats and then we're gonna have a minute rest do that five times alright I like breaking up things into intervals study intervals workout in tools let's go set one [Music] Photobucket don't try and make ya'll comfortable for the record you in truck Road and stopping you for the Brecon lab on me going on away for the record a trial eight notes on away [Music] I got out pricing on these for dates for your shoulders so I see y'all hey I will get a bar today 100 US hundred squats 5 mins the switch stretching you like that bill yeah yes London you feeling good now a little bit added race well that's good it's definitely raining outside now check this out 7 you made a made an absolute mess there didn't you alright we had some water wipe this sweat off I'll get back in the last relapse we'll finish this certification tonight oh my gosh is it dark in this room oh then yeah don't say that again oh yeah ok let's get back into so Google engineering labs wonder what we remember what was this I think it's I think it's MapReduce which I'm gonna have to look up the definition to as well because there's a lot of terms in here one of these stuff is is new to me right and I think it's important to recognize that even though it's it's all new and you're going through I'm going through these labs you might be going through these labs or whatever course you're not gonna get it the first time around like I find that a lot in jujitsu right so I go in I do a move and I'll stuff it up and then the next time I go in and be the exact same move and I make the same mistake and my coach is always like who's a black belt who's wise awesome dude and he's like you know make the same mistake a hundred times 500 times but eventually you'll realize hey I should probably stop putting my arm there so I don't get down next time anyway let's get back into it we've got three labs left data flow data pipeline or there we go lab data pipeline there was MapReduce that's what I was talking about before gonna have to that I think it's reducing the dimensionality of your data or if you know what MapReduce is leave a comment below and explain it as best you can and so you can help me and others understand and then we have another lab MapReduce in dataflow so you've got dataflow pipeline I'm producing data flow and then a final lab which is lab no lab inputs side inputs and then we finish up with the last quiz easy money it's 6:42 we're gonna finish this in time for dinner well alright catch on the flip [Music] when the last lab what do we got side inputs bow society inputs are like different sources of data into your main main pipeline so imagine you've got the stream of data here side inputs another one coming in over here that's how I'm thinking of them anyway all right let's get into it finish this lab go get some food wrap it up all right now it's my last attempt on this quiz for the last 8 hours or for the next 8 hours if I don't get this quiz right I always I wrote an article the other day on how I always stuffed up in exams exams get me mad they get me all right so which of the following statements are true I think it's this one dataflow execute Apache being pipelines and data flow transforms support both batch and stream and this one I know is BCA if not we're completing this tomorrow morning jump in oh boy oh boy oh boy some resources here you can check those out if you want actually there's a great medium article which is here I'll link that below so you can check it out it's it's a it's one of the sample projects right it's so Google put all this all this github data on bigquery so you can test it out so it's two billion files and then we did one before would you like which Java programs need help so the Java open source libraries which have to do Zoar fix means in them and there was two million one hundred ninety six thousand two hundred forty-seven files something you don't really want to process on you on your little baby laptop all right you want to you want to use bigquery for that so we've gone through a lot of stuff today let's see if let's see if we were completed all data prep well done preps a new one actually they've just learned about bigquery dataflow MapReduce have we completed the required stuff for today oh no what is this we haven't passed one of these things I'm gonna have to go back it was that dang sorry inputs lab the last lab best lab alright well we're gonna have to go back I'll see you in a sec alright we're back so I don't know what's going on there must be an error I need to reach out because Sarah because refresh this page sanim haven't completed this lab then we go into here the exact lab lab completed elapsed time seven minutes in seven minutes because I've done it four times now and I'm really quick in it you don't need a certificate right now you think so yeah you can fix the bug tomorrow whatever but what's most important is you got the skills yeah you're on you got an incredible platform like go Sarah it helps out 35 million people worldwide partners with incredible companies incredible universities what you want to use it for is to get skills that'll take you to where you want to go now you the thing about studying you can spend the whole weekend in this room all right hours in that chair there then you go to work on Monday or you see your friends no one knows about the hours you put in here but that's okay cuz you're on the journey I am the journey to where I'm from 20:48 I want to thank you I want to thank you for taking care of your health and take care of you knowledge step by step I made you may not notice it now but it all adds up over time trust me keep learning and keep moving to true future Dan yeah I gotta keep that in mind you heard man keep learning team special shout out to to call Sarah for sponsoring this video and special shout out to my boy lack lacks back baby LOC lat I still get his name wrong it's lack it's definitely lack lack you're the man man Suzie my gonna get some dinner keep learning oh yeah and if you want to try the course there's a link below as well as a link to that medium article which was a tutorial on bigquery but I'm enjoying it except for this little one part that's not letting me pass but we'll get through it have fun I mean you can do it tomorrow but the certificates not the be-all and end-all let them take advantage I was pilot already offering deals tunnel chocolate I still want to ask not the ghost motor running don't try and make your truck Road and stop me [Music]