Hands on with Cloud AI | Workshop

Key Takeaways

[Music] hello everybody welcome to build and deploy a model with vertex ai my name is sarah robinson i'm a developer advocate on the google cloud team focused on machine learning you can find me on twitter at s wrap tweets in this session i'm going to walk you through how to train a custom model how to deploy that to an endpoint in vertex ai and how to get predictions on that model endpoint if you are registered for this session at google i o you can ask questions using the q a feature and i have some awesome tas that will be there to help answer your questions so just wanted to give a shout out to them boris dale mark and mikhail thank you a bunch for answering questions so anything you have questions on if something isn't working if you're following along step by step just ask uh in the q a so now to the fun part we're going to get into the workshop uh what will you learn in this workshop so yesterday we just announced vertex ai which is our new end-to-end machine learning platform on google cloud helps you with every step of the ml process from gathering your data training deploying models and managing models in production so you're going to learn how to train and deploy a custom texture model on vertex ai the finished product will be a deployed model endpoint that you can use to get predictions so if you'd like to follow along please head over to the url on this slide g dot co slash code labs slash cloud ai dash lab so if you could all go there i'm assuming you're following along you can also just watch i'll be going through all of the steps i'll leave that up for just a moment so folks can get to that url and from there let's roll to the demo so if you went to that url you should be at a page that looks like this um this is the code lab that we'll be following step by step i'm going to go through everything and you'll want to also have your cloud console open in a second tab you can access that by going to console.cloud.google.com what you see here so i'm not going to show the lab i'll just assume that you have that up i'm going to be accessing it from another tab so this way you can focus on the console um so if we if we look at the overview step in the lab um we'll get started we're going to use vertex ai as i just mentioned this is our newly announced ml platform on google cloud super excited to have all of you try it out we are going to use this to build a custom model so we're going to build a tensorflow model see how to kick off a training job on vertex ai and then take the results of that training job deploy that to a model endpoint and get our predictions um to interact with most of the tools in this lab we're going to be using a cloud shell which is a command interpreter that is powered by a vm that's running on google cloud um the great thing about cloud shell is that it lets you interact with all the different uh products in your cloud console so you can leave the shell window up click around and access a ton of different products uh the cloud shell uh window also has an ide so we're also going to be using the code editor in cloud shell to generate some of the files that we're going to be using for training in this lab one other thing the cost of the lab as i mentioned is about two dollars just make sure you follow the instructions at the end to clean up the resources so that you're not charged assuming that you don't want to use them after the workshop um so with that let's proceed to step two um quick intro to vertex ai so vertex ai which we can see in our console if we go here just click on the navigation menu i have pinned it to the top which you can do by scrolling down to find vertex under ai and then just hit that pin button so i've pinned it to the top so we can see vertex ai here as you can see vertex ai includes lots of different tools to help you with each step of the ml process and you can use as many or as few of the as few of those as you want so if you only want to use it for training you're welcome to that if you want to use it for prediction you can do that or you can use it for your end-to-end ml workflow in this lab we're going to be focused on the training models and endpoints products here in vertex ai um so with that we are going to start setting up our environment um so if you don't have a project i mentioned this before you'll want to just click on this at the top and then you want to hit new project here you'll give your project a name it'll just take a few seconds to create the project and you'll be good to go and just make sure that you associate that project with a billing account that you've set up so we are in vertex ai um as i mentioned the primary way we're going to be interacting with vertex is through this tool called cloud shell so you'll want to find this activate cloud shell button at the top of your console and we're going to click on that okay so we can see we now have access to this command interpreter that we can use to access all of our cloud resources the first command that you run in cloud shell it'll ask you to authorize it so you'll want to click authorize so just to make sure that we're authenticated correctly i'm going to run gcloud off list if you've never used gcloud before gcloud is the cli tool that you can use to interact with many different cloud projects so we can see that i am authenticated and this is the project that i want to use for this workshop if this if the project you see here isn't the one you want to use just run and this is in the the lab instructions as well just run gcloud config set project and then the project id that you want to use i'm all set so i don't need to run that there is a environment variable that cloud shell gives us with our project id and i'm going to be using this in several places throughout the lab so you'll want to run echo google cloud project that's the name of the environment variable that we get through cloud shell and you should see the id of your project there so that means you're good to go so now that we know how cloud shell works one other thing i want to show you is this handy ide that i mentioned another cool thing about cloud shell is that you can expand the window uh to be as big as you want and you can click around within your console and the session will stay active so you can also click on this open editor button this is opening the cloud shell ide and we're going to be using this to create some files throughout the lab you can have the terminal and the editor side by side i kind of prefer just bouncing back and forth between them so whatever files i create here will be saved if i go to the terminal and vice versa so i'm just going to be going back and forth between them um okay so now that you are oriented with cloud shell and the editor that's in cloud shell um we're going to get started so the first thing you'll want to do is you'll want to copy that snippet which will use gcloud to enable all the services that we're going to be using in this workshop so you'll run a command that looks like this um this is enabling three services for us compute engine we're going to be using this to run some of our training jobs and when we deploy our model container registry is a place where you can store all of your docker container images securely and share them amongst different people on your team we're going to be using that to package up our training code so one option that vertex ai provides for training and i'll get into this more in a moment is packaging your code as a docker container so we are going to package up our code we're going to push it to google container registry and then we'll use that to run training so the last service we're enabling is uh called ai platform that will give us access to all of the different tools that we'll be using on vertex i am going to zoom in a bit so that you can all see this a bit more um hopefully that makes things a little easier to see um okay so once this completes um you should see this message operation finished successfully um the next thing we want to do is create a cloud storage bucket um so google cloud storage is our object storage solution on google cloud we'll be using this to save some of the assets that are created in our training job so when we run model training the last step of that training job will be to save those tensorflow model assets somewhere we need to save those into a cloud storage bucket so we are going to create a bucket which is what this command does um i have already created this bucket buckets across google cloud storage need to be globally unique so i the reason i put this here is that your bucket name is just going to be um your cloud project name with slash bucket or dash bucket appended um so it should hopefully be globally unique um and then you're going to run this gsutil command which will make the bucket for you in a specific region i'm using the u.s central one region throughout this lab you can use other regions if you'd like just keep in mind that you need to keep it consistent since the region that your bucket is in is also the region that you need to use when you create an endpoint let me just make sure that that font size looks good i think we're good but let me know if you can't see that um so this uh gsutil is the cli we'll use to interact with google cloud storage um i have already created this bucket so i'm not gonna run this command but if you run that command it should create a bucket for you and then when you go to cloud storage you should see that bucket created it'll look like this with your project name there and notice that i'm clicking around to different products here in my console but my cloud shell is persisting so you should have created a bucket with this bucket name environment variable and the last thing that we're going to do in this step is we're going to just make a quick alias to make sure that when we run python scripts throughout this lab we're going to be using python3 so that concludes the setup portion of this so you should have created a bucket and enabled all the services that we'll be using throughout the workshop um and now we're going to get into the machine learning aspect of this the model that we're going to be building is we're going to be using a public data set of car data the data set is called auto mpg i'll show it to you real quick it's available on kaggle also provided by uci machine learning so this just has some metrics on different cars along with their fuel efficiency miles per gallon so we'll be building a regression model which means that we'll be predicting a new a numerical value which will be the fuel efficiency of a vehicle um so that's the data set we'll be using we're going to be building a tensorflow model to do that fuel efficiency prediction but the focus here is mostly on the tooling available in vertex ai for training and deploying models so hopefully once you've completed this workshop you'll be able to take what you learn and apply it to your own data set that you're working with at your organization so um next we're going to move on to step four which is where we're going to get into the fun ml parts of this so in this step we're going to be containerizing our training code to do that we want to create a couple directories and files that we're going to use to put our training code into a doctor container so i'm going to run these commands to create some files within a directory called mpg just calling it mpg because that's what we're predicting miles per gallon so once i've created these i should see them show up in the editor and i do so i have this mpg directory um let me just make this a tad bigger i have an mpg directory with a trainer subdirectory and this trained.pi file the reason i've chosen to use a container for this to to put this all in a docker container is because this handles a lot of the headaches of dependency management and version management of all the different dependencies that my training code uses by containerizing them and i can easily share that with other folks on my team if they want to reproduce my training experience so we've created all these files they are currently empty so you just need to double click on them in cloud shell to open each file so we have this docker trainer directory and this train.pi file is where we're going to be writing our training code so the next step is to create our docker file the docker file sets up the commands that will be run when this container is run so the first line here is notice that we're using a deep learning container image provided by google cloud um so there's many different options to choose from you can look at the google cloud docs if you want to learn more about what's available there we're using a tensorflow 2.3 image one of the great things about using this image is that it comes with a lot of popular ml and data science frameworks pre-installed so it includes pandas scikit-learn tensorflow along with some others um so then we are going to set up our entry point to invoke our training code which is in this trainer dot train uh file so once we've done once we've copied that in we can save that in our docker file and the next thing we want to do is we want to take that long code snippet in step four of the lab that will contain all of the training code that we want to be run when we run this training job on vertex ai so i've just copied that and i'm going to paste it into my train file so let's just walk through what this does briefly there's a reference to a bucket at the top we're going to replace that in a moment so don't worry about that right now first we're just importing some libraries that we're going to be using for training and i've made a version of this data set public in google cloud storage so in the next a few lines we're just downloading that data set into a pandas data frame we're going to just clean up the data a bit do some feature engineering to make sure that the data is all in the right format that we'd like next we are splitting the data into train and test sets um and this is adapted from a tutorial in the tensorflow documentation if you'd like to learn more about it so we're splitting our data into training and testing which is uh best practice in machine learning you want to make sure that you set aside some of your data to use for testing that wasn't used when you trained your model the next thing we're doing is uh which is really important in machine learning as well as we're normalizing our data so when you're dealing with numerical data often that data can range and have many different values and so what you want to do is typically normalize that within a common range so that all of your values are within say negative one and one or zero and one um and then we build our model using the keras sequential model api which is part of tensorflow and so here we're just the sequential model api lets us build our model as a stack of layers and remember that this is a regression model so the output is just a single numerical value and then we run training and the final line of code here is going to save our trained tensorflow model to a bucket in google cloud storage so if you remember we had that bucket variable way at the top of the file up here so now we want to replace that with your cloud storage bucket so let's do that so if you go back to the lab um in step four you will see that there is a little said command there so what we want to do is we want to bounce back to the terminal um let me just make this a little bit bigger so you can i'll see the terminal code okay we're going to run this said command all this is doing is it should replace the bucket name variable with the name of our bucket at the top of the file which it did so that's a success um and now we have all of our code uh ready to go so the next thing we want to do is um build and push the docker container so we'll first we'll build the container locally then we'll push it to container registry so we'll be doing this back in the terminal when you run these commands you want to make sure that you're within the mpg directory whatever directory has your docker file at the root so if you copy that command that sets the image uri variable um when we push this to container registry this should be it should use your google cloud project id um as part of your url and we're calling this image mpg and we're tagging it as v1 uh next we want to run docker build so this will run the commands we've specified in our docker file and this might take a few minutes for for you if you're following along might take about five minutes to run i've already run this a few times so shouldn't take that long for me i'm just going to make this a bit smaller while that runs so once the build command completes the next step is going to be to push this to google container registry um and so i'm gonna head over to google container registry while this is going and you can see that i have one version of this here just a saved version of of this image that i'm building right now but once this build command completes we will run docker push with that image uri and we will we should then see that appear in container registry um so if that doesn't finish up soon i'll probably just use the saved version that i have and yep so after this you will push this image to container registry again if you have any questions or running into issues just use the q a feature to ask a question to any of our wonderful tas and so once our container registry image is available we can then point vertex ai to that image to run training if you prefer not to work with containers you can also package up your training code as a python package and you can provide your custom training code to vertex ai that way if you'd like to do that so a couple different options for training i'm just showing you one here you can also take advantage of automl for for training your models um so while that's running i'm just going to pop back over to vertex ai and show you where we'll kick off a training job i'm going to make this just a little bit smaller so that you can i'll see so you should all be able to see that the training section of the console in vertex ai so once this is ready to go we will click create to create our training job um we will tell vertex the url this container and container registry and then we will be ready to go um so because this is taking a bit i'm just going to use a saved version that i have of this image so i'm going to go over here to create once you're just remember once your docker image finishes building finishes the build step you'll just want to push it so there's a command in the code lab for you to run docker push with the url of your image that'll push it to container registry and you will see it there um so next step here is to kick off a training job and just a reminder that all of the things that i'm showing you within the vertex ai ui here in the console you can also do any of these things programmatically so i'm showing you how to kick off the training job um in the console but you can also use the vertex ai sdk to write code to automate all of your ml processes from creating a data set creating a training job and kicking off that training job so the first step is just to tell vertex ai which data set we'll be using um i'm not going to get into the manage data sets product in this workshop but if you would like to upload your data set to vertex ai via manage data sets that's what you'll want to do if you want to use automl for training so as you can see here there's a few different options for training we are downloading that data set within our training code which is why i've selected no managed data set here so there's a few different options here primarily auto ml or custom training so if you don't want to worry about writing that model code yourself you can take advantage of automl which uses a technology called neural architecture search under the hood and it searches across many different model architectures to find the best one for your prediction task so you don't have to write any of that model code at all automl will handle finding the best model for you it actually looks like our docker build may have completed so let's head back there and see if we can use that the other option so if you want to write the training code yourself which is what i'm showing you in this workshop um you would select custom training on this step um but let's just see if this completed hold on one second ah it's still cranking here there we go all right so now i'll run docker push this should be faster than the build there we go okay so now i'm just going to go back to container registry to show you that hopefully we see that new image there yes we have this mpg image which is the one that i just pushed so now i'll be able to use that image that i just created when i kick off my training job so the next step is to go into vertex ai select training i'm going to make cloud shell a bit smaller because we're going to be using the ui to kick off the training job and just for those following along with the code lab the um we're now on step five running a training job in vertex ai um so we will hit create here we are not using a managed data set we are running custom training so we'll hit continue we're going to give our model the name mpg you can give it whatever name you'd like i'm just using mpg here we'll hit continue so when you're working with uh custom training there are essentially two options for how you want to provide your custom training code to vertex ai the first option is a pre-built container so if we have a number of supported runtimes that give you access to containers that have already been created so if you want to submit your custom training job in this way all you need to do is package up your training code as a python package um and the supported runtimes are for xg boost scikit-learn tensorflow and pi torch and there's various versions of each of those that are supported if you're if you prefer to use a docker container which is what we're doing here maybe because your code is written in um you're using a different machine learning framework or a language other than python um you'd want to go the custom container route so we will click on custom container here um let me just make sure everyone can see my screen okay yes we're good um okay so in the custom container settings we can click browse here and we should see that container that we just pushed yes two minutes ago we're gonna select that um so that we don't have to paste that container image url it's just handy right there in the browser um we are going to leave this model output directory field blank we would use this to specify a directory where our saved model assets are located when we deploy our model just for in the interest of time in this workshop i've made a public version of these model assets available and we're going to use that to deploy the endpoint just so that we don't need to wait for the training job to complete so you'll specify your container image here you'll want to leave the rest of the fields blank step 4 is hyper parameter tuning in this workshop we are not going to use hyper parameter tuning but just a little bit about what that is if you'd like to have vertex ai handle tuning or hyper parameter values for you things like learning rate or the number of neurons in a specific layer in your machine learning model all you need to do is in your code you would provide a config file that specifies the different hyper parameter values you want to tune the range of values you'd like vertex ai to try for each of those hyper parameters um and the metric that you're trying to optimize for and using this um this will use bayesian optimization as the technique for hyper perimeter tuning there's also a couple of other algorithms available so you can read more about hyper primer tuning in the docs we are not going to use that for now okay so now we're going to specify some compute settings for our model training job um remember when we selected the region for when you created your cloud storage bucket you we used us central one or at least i did in this tutorial so if you use a different region that's where you would want to click this drop down and um change it to the region that you used when you created your bucket and now we can specify the compute settings for this training job there are a lot of different distributed training options available on vertex ai the model example that i'm showing you here it trains really fast so we're not going to take advantage of distributed training for this but if you wanted to you would add more worker pools so here we're just going to select a standard n1 standard for a machine we're not going to use any accelerators here so we're not going to use a gpu for this job but if you'd like to do that you can specify that if you kick off your training job programmatically via the sdk you can also specify all of these replica account options using the sdk so if we wanted to run distributed training we just need to make sure that we had specified a distribution strategy in our training code with whatever framework we're using we would then choose how many worker pools we wanted to be allocated for this job and whether we wanted to add an accelerator to any of those worker pools like a gpu as i mentioned just to keep things simple in this case we're going to just use one worker pool with an n1 standard 4 machine um and then last step so as i mentioned you don't need to use uh vertex ai for training and serving if you only want to use it for one piece so one example maybe you have a very computationally expensive training job and you want to take advantage of the cloud for running that but then you want to download the resulting saved model assets and deploy that uh run it in an on-device environment you can absolutely do that um you can also maybe your model trains really fast and you'd like to just train it on your laptop but then you want to deploy it to the cloud to use online prediction or batch prediction you can do that as well so we're going to leave this prediction container option blank as i mentioned i wanted to show you how to kick off a training job in vertex ai when we deploy this model to an endpoint we're going to use a version of the save model assets that i already have available so with that we will click start training and we should see this training job kick off here so we can monitor this training job in the console and after the workshop i encourage you to go back and take a look when the training job completes it should write the tensorflow save model assets to the storage bucket that you created at the beginning of the workshop so you're welcome to go back and look at that you can also go take a look at the logs for the training job as it's running um so with that we are going to move right along to the next step of the lab so this training job should take about 15 minutes to complete so again when it's done you can go back and check it out you can even deploy a new model using the model assets that were created from your training job but just in the interest of time i've made these model assets available in a public cloud storage bucket for you so if you're following along with the code lab tutorial um make sure you are on step six at this point next we are going to um be getting back to cloud shell we're going to be creating a model in vertex ai we have the concept of model resources and a model can be deployed to an endpoint the endpoint is what you would use to get online predictions which is what we want to do so we will make our cloud shell a little bit bigger because we're going to be using that for the uh next steps of this lab so let me just clear that um okay so next step is we want to install the vertex ai sdk so i'm grab i'm grabbing the pip3 install command from the code lab and we can use this sdk to do all sorts of things in vertex ai we can use it to create data sets and i just want to make sure i've installed the latest version correctly looks like i have we can create data sets training jobs deploy models get predictions get batch predictions lots of things that i haven't listed there and you can take a look at the docs if you want to learn more about how to use the sdk in this case we're going to be using the sdk to create a model and deploy that model to an endpoint um and so to do that we're going to head back to the editor in cloud shell and i'm going to just copy this code before i do that so we'll want to go back to our editor and we don't need this we're not going to be using the mpg directory now since we've already run our training job um we will want to create a new file which we can do via this file menu and we're going to call this deploy dot pi okay you should have a file that looks like this uh let me let me just make it a bit bigger so you can all see what i'm doing in here okay so in this file um you want to paste that code that is in step six of the tutorial and what this is you don't need to make any changes to this this is going to upload your model and then deploy that model to an endpoint so we're just importing the vertex ai sdk that's what this line is doing and then here we're using it to upload our model here notice we have this artifact uri parameter that we've passed these are the public model assets that i've made available so once your training job is done if you'd like to create a different version of this model um what all you need to do is change the artifact uri to your own bucket and make sure that the directory you provide is just the directory where you have your savedmodel.protobuff file so you can absolutely go try that after this with your own saved model assets and then if you remember when we went through the training steps we had the option to specify a prediction container we had we said no prediction container in this case we're specifying it here so just like for training how there is an option to use either a pre-built or custom container we have the same options for model prediction on vertex ai we can use a pre-built or a custom container so for serving our model for this lab we're going to use a pre-built container we're using this tensorflow 2.3 image which is available from google cloud we could also create our own container for serving and one of the benefits of that would be if we had any pre or post processing code that we wanted to run so maybe let's say we have a text model we're doing some nlp maybe text classification and we want the client to be able to send raw text to the model but our model obviously needs that in a different format maybe word embeddings a numerical format so in our container we could handle transforming that raw text input into word embeddings and then if we wanted to do any post-processing on the output we could do that in our custom container as well to keep things simple for the purposes of this workshop we are going to use a pre-built container um and then once the model's created we're going to deploy that to an endpoint the only parameter we need to provide here is uh the machine type that we want that endpoint to be on and one of the cool things you can do in vertex ai is you can split traffic between different models so maybe you want to roll out a new version of your model incrementally to a small subset of your users you this is where you would specify a traffic split so maybe you only want to serve one model to five percent of your users and serve a different version to the other ninety-five percent um you could specify that here so once you've saved this file again you don't need to make any changes to it you will go back to the terminal and we want to make sure we are one directory up so not in our mpg directory just make sure you're in the directory where that deploy file you just created is okay um and then the next thing we're doing is we're just running that deploy file and we're piping the output to a text file which we'll use when the endpoint deploy has completed to grab the url of our deployed endpoint and copy that into our prediction file so that will take probably about 10 minutes to deploy but don't worry i've got a version that's already deployed for me to show you and for us to try out so i'm going to make this a bit smaller as you can see if you go back to your editor um you should see this deploy output file created which is logging the output of that command to this file that's what the t did in the command um so let's minimize this we'll go over to our models if we hit refresh we should see that a new model was just created and we want to click on that and we should see a pending endpoint deploy which we do so that's a good sign um so once this endpoint is deployed we can use it to get online predictions let me just make sure everyone can see the screen okay again looks good um so while we're waiting for that endpoint to deploy um i'm going to show you an example with a version that i have already deployed so if we go back to our models page um we see this saved version and we see an endpoint that has been deployed um and so the next step in the lab is to finally get a prediction on your deployed endpoint so that was the goal of this workshop teach you how to train a model deploy that model to an endpoint and then finally get a prediction on that model so next step is to create a file called predict.pi so let me just copy what's going to go in there so we're back in our cloud shell editor file new file we want to go predict up high and we see that file created here and we'll want to copy the code from step 6 into this file into this file so let's just look a little bit at what's going on here so again just like when we deployed our endpoint we are importing the vertex ai sdk and the next thing we're doing is creating a reference to that endpoint here it just has a placeholder endpoint string in the lab you'll see instructions for how to update that with your specific endpoint string once it's finished deploying next we have a test example that we're going to send to our model this test example has already normalized data so this has been normalized so it's in the format that our model is expecting the output that we're expecting is a predicted uh fuel efficiency value miles per gallon so that's our test example all we need to do to get a prediction um via the vertex ai sdk is just run this endpoint.predict uh with that test example and we're going to log the response and just the actual miles per gallon value to the console um so let's check on the status of our endpoint um go back here okay looks like it's still deploying so in that case um we're just gonna use the version that i have ready to go um which is this predict save file so as you can see um this has the url of i believe the endpoint that we're looking at here yes you can see that this id matches the one here um so we're creating a reference to that endpoint that i've already deployed um and then we're just going to run that to get a prediction on our model once your endpoint deploy completes just make sure that you go back to the lab there is two commands that you need to run just some said commands that will replace this um endpoint string placeholder with the string of your endpoint so don't forget to run those two commands otherwise your prediction will not work so we are ready for the final piece of this which is getting a prediction so all we're going to do is back to the terminal i'm just going to open a new tab and make sure i'm in the right directory yes so i'm going to run python3 predict saved dot pi this should paying my deployed model which it just did and good news is we have a prediction so for this particular input um our model has predicted that based on these car metrics uh the predicted miles per gallon is 16.3 roughly so just to recap all the things that i've demonstrated here um we started by enabling our these services um packaging up our tensorflow model training code into a docker container uh pushing that container to container registry then we used vertex ai to train our model pointing it to that uh container registry image and then we deployed it to an endpoint and finally got a prediction um as you can see was able to do all of that um in under an hour and again the cost of of this was just about two dollars um it's important so that you're not charged for this end point assuming that you don't want to use it when you're done with this is that you undeploy it um and you also want to delete that storage bucket that you created um so instructions for that are in step seven of the lab um so that is that basically concludes the lab if you have any questions um definitely use the q a feature um since we do have a little extra time i will um tell you a little bit more about vertex ai so if we could go back to the slides where i left off on those okay yeah so just to give you a little overview on vertex ai and introduce you to some of the features that i wasn't able to cover in this live workshop um so again at a high level what is vertex ai vertex is a managed machine learning platform uh for practitioners to accelerate experiments and deploy ai models um so what does that mean as you saw it's an end-to-end machine learning platform so it's designed to support you through each step of your ml process um and provide a unified environment for doing that so what no matter what uh tool you use for model training all of your models will show up in the same place along with all of your endpoints and you can use the same api the same sdk to access all those machine learning resources and again this is just announced yesterday so you can be one of the first to try it out just to give you a little bit of background and set some context for where vertex ai fits into a typical application development workflow so as you may have guessed vertex ai sits at your application back-end and provides this end-to-end complete platform for machine learning model development deployment and monitoring so you would then probably have vertex ai you probably have your application logic let's say you wanted to build an application that serves predictions to end users in that case you would have your application logic communicate with your deployed model in vertex ai if that's what you want it to do so just as we did that code that you just saw the prediction code to call our deployed model endpoint that would be maybe your application logic talking to vertex ai and then you would surface that prediction to your end users this doesn't uh describe every use case so maybe you only want to uh generate batch predictions on your model you can absolutely create a batch prediction job in vertex ai either through the api or right in the ui so we saw just a few pieces of vertex ai demonstrated in this workshop um but this slide is just meant to give you an overview of all the different parts of vertex that are included so it takes you from all the way from gathering your data that you're going to use to train your model tools to help you with feature engineering hyper parameter tuning training serving your models and monitoring those production models to make sure that they are staying accurate in this lab we focused on the training and prediction services so you can use vertex ai for your entire machine learning workflow if you'd like you can use it to upload your data train your model um scalably deploy your model to an endpoint um and you can also use it to streamline workflows with pipelines so some new products that are available in vertex ai can help you with ml ops which is the process of productionizing standardizing and making your machine learning workflows reproducible and we have many different products to help you with that from pipelines to model monitoring to feature store we have documentation and blog posts for you to learn more about that i touched on this during the workshop but just to reiterate vertex ai supports both ui based model development and code based development so whichever you prefer if you'd like to go into the console to kick off your model training jobs you can absolutely do that but you can also use the sdk we saw a little bit of both here we use the console to create our training job and then we use the sdk to create our model and deploy that model to an end point and get a prediction and this is just an exercise that i use when i'm working with customers and external folks to help them figure out the right tool to use for their machine learning job so once you've identified that you want to use machine learning for a particular task the next step the next question that i'll typically ask is do you have your own training data if the answer is no we have a set of pre-trained apis for you that give you access to pre-trained machine learning models that have already been trained on lots and lots of data so all you need to do is pass it some data and it works just like simple rest api and you'll get some analysis back so this is a great choice if you want to perform a common machine learning task maybe by analyzing what's in an image um getting the sentiment of text or translating text we have a number of different rest apis to help you do that if you don't have your own training data this workshop assumed that you did have custom data that you wanted to use to build a machine learning model so train your own custom model so if if the answer to the first question is yes then the next thing you want to think about is whether or not you're writing that model code yourself and summary here is that whether or not you are writing that code yourself we have tools that can help you on vertex ai so it does support varying levels of machine learning expertise so if you don't want to worry about writing model code you can use automl for training which i didn't show in this lab but i do have another lab that shows you how to do that and then if you are writing the model code yourself using um frameworks like tensorflow pytorch xgboost socket learn or any other framework as we saw there's lots of tooling available um to support building custom models on vertex ai so that's that's the path we took in this workshop a little bit more about automl training so automl gives you access to state-of-the-art machine learning models in less time and doesn't require you to write any model code and when your model is done training you get access to detailed evaluation metrics and for certain data types you also get access to feature attribution so this tells you that signals that influenced your model most in making a prediction various data types are supported for automl including image data video data text data and tabular data um we i showed you how to do custom training so you can train models with a framework of your choice um using custom containers or you can use one of our pre-built containers you can run distributed training jobs and accelerate your training via gpus and you can also make use of hyper perimeter tuning which i talked about a little bit then as we just saw we can scalably deploy your model to an endpoint and whether you train your model with automl or custom model code all of your endpoints show up in the same place and you can access them with the api that i just demonstrated you can split your traffic between models and you can also customize the machine type on your endpoints as you saw i passed that parameter when i created the endpoint via the sdk so that's all i got for today thank you for joining thanks the tas for helping answer questions if you want to get started uh cloud.google.com vertex ai if you want to dive into documentation and start building your own custom models with vertex ai and then one more thing just want to plug some upcoming events um opportunities to learn more about all the tools that i covered today and some that i didn't get a chance to cover we have the google cloud data summit coming up on may 26th and the google cloud ml practitioner summit where i'll be going more in depth on some of the features that i covered in today's session um that is on june 10th and you can register uh via the link on the slide so encourage all of you to check that out so thanks everybody for watching and i hope you enjoy the rest of google i o you