Run Hugging Face transformers on GPU enabled Cloud Run functions

Key Takeaways

we create a GPU enabled Cloud run function that will directly host a Jamma 2 2B model downloaded from the huging face website I'll also show you how easy it is to leverage huging face Transformers python library to rapidly build AI powered applications with Cloud run let's get started first first I'll create a cloud run function via the UI let's go ahead give the service a name and we're going to choose python as our runtime let's go ahead make the public uh endpoint public for the demo purpose and choose the CPU always allocated under the containers tab I'm going to go ahead configure the memory and CPU required for this container instance and we attach an Nvidia L4 to this to this let's go ahead and set concurrency to be one all right next let's go to take a look at the Vari variables tab here I'm setting up two environment variables the first one uh it will be a LD Li Library pass that's going to point to the pass to for me to access to the Cuda drivers next environment variable I'm using the secret manager to store my huging phas token which is needed later on for me to download the models from the huggingface website let's go ahead and create this Cloud run function now it's going to take a few second all right so the cloud run function service is created let's Jump Jump Right In to edit our code first let's put in some of the requirements needed for this app and then for the main logic of this application instead of writing any new codes let's go ahead go to the huging face website here we're going to copy the code over into our functions event handler all right so let's copy the code here let's go ahead fix the indentation and then we'll move the inputs to the top of the file let's update the device wise map to Cuda so here I uh we are leveraging the Transformers python library to interact with the Jamma model and then I'm asking JMA to help me write a poem about machine learning and also know the maximum output token here is set to 32 so expect the poem from the Jamma model be truncated all right so so far most of the code is based on the existing samples except we made a few tricks here and there and we didn't even need to write any new code everything looks good let's go ahead and save and deploy This Cloud run function in the interest of time uh let's go ahead and switch to this pre-deployed pre-build Cloud run function here you can see the source code is identical to the one we just deployed now let's go ahead actually invoke this Endo since this is a warm instance the Jamma model responds quite quickly as expected poem is truncated because of the token setting in the sample code all right so we've done it we have successfully demonstrated the Jamma model running on a GPU enabled Cloud run function