Deploying 🤗 ViT on Kubernetes with TF Serving

📰 Hugging Face Blog

Deploying ViT on Kubernetes with TF Serving for scalable model serving

advanced Published 11 Aug 2022

Action Steps

Containerize the ViT model using Docker
Push the Docker image to a registry
Provision a Kubernetes cluster on a cloud provider like GKE
Write Kubernetes manifests to define the deployment
Perform the deployment and test the endpoint

Who Needs to Know This

This tutorial is beneficial for machine learning engineers and DevOps teams who want to deploy and manage AI models in a cloud environment. It helps them understand how to containerize and deploy models using Docker and Kubernetes.

Key Insight

💡 Using Docker and Kubernetes enables scalable and efficient deployment of AI models like ViT

Key Takeaways

Deploying ViT on Kubernetes with TF Serving for scalable model serving

Full Article

Published Time: 2022-08-11T00:00:00.114Z

# Deploying 🤗 ViT on Kubernetes with TF Serving

[![Image 1: Hugging Face's logo](https://huggingface.co/front/assets/huggingface_logo-noborder.svg)Hugging Face](https://huggingface.co/)

* [Models](https://huggingface.co/models)
* [Datasets](https://huggingface.co/datasets)
* [Spaces](https://huggingface.co/spaces)
* [Buckets new](https://huggingface.co/storage)
* [Docs](https://huggingface.co/docs)
* [Enterprise](https://huggingface.co/enterprise)
* [Pricing](https://huggingface.co/pricing)
*
*
* * *

* [Log In](https://huggingface.co/login)
* [Sign Up](https://huggingface.co/join)

[Back to Articles](https://huggingface.co/blog)

# [](https://huggingface.co/blog/deploy-tfserving-kubernetes#deploying-%F0%9F%A4%97-vit-on-kubernetes-with-tf-serving) Deploying 🤗 ViT on Kubernetes with TF Serving

Published August 11, 2022

[Update on GitHub](https://github.com/huggingface/blog/blob/main/deploy-tfserving-kubernetes.md)

[- [x] Upvote 1](https://huggingface.co/login?next=%2Fblog%2Fdeploy-tfserving-kubernetes)
* [![Image 2](https://huggingface.co/avatars/129d1e86bbaf764b507501f4feb177db.svg)](https://huggingface.co/Aanuoluwapo65 "Aanuoluwapo65")

[![Image 3: chansung park's avatar](https://cdn-avatars.huggingface.co/v1/production/uploads/1659971187637-60d3b57ad7b174177faabd6e.jpeg)](https://huggingface.co/chansung)

[chansung park chansung Follow](https://huggingface.co/chansung)

guest

[![Image 4: Sayak Paul's avatar](https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg)](https://huggingface.co/sayakpaul)

[Sayak Paul sayakpaul Follow](https://huggingface.co/sayakpaul)

guest

* [Why go with Docker and Kubernetes?](https://huggingface.co/blog/deploy-tfserving-kubernetes#why-go-with-docker-and-kubernetes "Why go with Docker and Kubernetes?")

* [Containerization with Docker](https://huggingface.co/blog/deploy-tfserving-kubernetes#containerization-with-docker "Containerization with Docker")
* [Preparing the Docker image](https://huggingface.co/blog/deploy-tfserving-kubernetes#preparing-the-docker-image "Preparing the Docker image")

* [Running the Docker image locally](https://huggingface.co/blog/deploy-tfserving-kubernetes#running-the-docker-image-locally "Running the Docker image locally")

* [Pushing the Docker image](https://huggingface.co/blog/deploy-tfserving-kubernetes#pushing-the-docker-image "Pushing the Docker image")

* [Deploying on a Kubernetes cluster](https://huggingface.co/blog/deploy-tfserving-kubernetes#deploying-on-a-kubernetes-cluster "Deploying on a Kubernetes cluster")
* [Provisioning a Kubernetes cluster on GKE](https://huggingface.co/blog/deploy-tfserving-kubernetes#provisioning-a-kubernetes-cluster-on-gke "Provisioning a Kubernetes cluster on GKE")

* [Writing Kubernetes manifests](https://huggingface.co/blog/deploy-tfserving-kubernetes#writing-kubernetes-manifests "Writing Kubernetes manifests")

* [Performing the deployment](https://huggingface.co/blog/deploy-tfserving-kubernetes#performing-the-deployment "Performing the deployment")

* [Testing the endpoint](https://huggingface.co/blog/deploy-tfserving-kubernetes#testing-the-endpoint "Testing the endpoint")

* [Notes on different TF Serving configurations](https://huggingface.co/blog/deploy-tfserving-kubernetes#notes-on-different-tf-serving-configurations "Notes on different TF Serving configurations")

* [Conclusion](https://huggingface.co/blog/deploy-tfserving-kubernetes#conclusion "Conclusion")

* [Acknowledgement](https://huggingface.co/blog/deploy-tfserving-kubernetes#acknowledgement "Acknowledgement")

In the [previous post](https://huggingface.co/blog/tf-serving-vision), we showed how to deploy a [Vision Transformer (ViT)](https://huggingface.co/docs/transformers/main/en/model_doc/vit) model from 🤗 Transformers locally with TensorFlow Serving. We covered topics like embedding pre

Read full article → ← Back to Reads