Deploy Hugging Face models from Vertex AI Model Garden

Key Takeaways

[Music] you can deploy thousands of AI models with Google cloud and hugging face without leaving the Google Cloud Web console and without setting up complex infrastructure I'm sure you want to know how so let's get started this is the Google Cloud Web console let me use the search bar to go to vertx AI now in vertex ai go to the model garden and has many machine learning model the models that are available in the model Garden can handle various types of data including language images and table data they can also perform a wide range of tasks including generating text classifying data and translating languages so you can find models from Google here and from other providers and one of those providers is hugging face let's explore here's a list of thousands of machine learning models straight from the face up you might see some models here that you're familiar with here's llama there's mistr and there's Google's Gemma too so let's deploy Gemma you'll need to fill in a hooking face access token now let me show you how to create a token you'll need to do this only once so first go to the hugging face Hub hugging face. and create an account once you're logged in choose settings then access tokens and click create a new token give the token a name and allow a read access to all public gated repositories you can access make sure to copy the token and you also need to accept the Gemma license right search for Gemma 2 and acknowledge the license agreement now back to vertex AI fill in your hugging face access token click deploy wait for a while and make a prediction request alternative ly you can also deploy using Google kubernetes engine make sure you have a gpe autopilot cluster with gpus ready or if you're using standard enable note out of provisioning then copy and apply the following manifest it creates a deployment with the hugging face deep learning containers and a service to expose the deployment it also creates a secret with your hugging phase token because it needs to pull those models from up so that's that you can thousands of models from the hugging face up without leaving the Google Cloud Web console [Music]