Accelerated Inference with Optimum and Transformers Pipelines
📰 Hugging Face Blog
Accelerate inference with Optimum and Transformers pipelines for improved performance and speed
Action Steps
- Install Optimum for Onnxruntime
- Convert a Hugging Face Transformers model to ONNX for inference
- Use the ORTOptimizer to optimize the model
- Use the ORTQuantizer to apply dynamic quantization
- Run accelerated inference using Transformers pipelines
- Evaluate the performance and speed
Who Needs to Know This
Data scientists and machine learning engineers can benefit from this tutorial to optimize their models for faster inference, while software engineers can apply these techniques to improve the performance of their applications
Key Insight
💡 Optimum provides a range of tools to optimize and accelerate inference for Transformers models, including quantization and optimization techniques
Share This
🚀 Accelerate inference with Optimum and Transformers pipelines! 🤖
Key Takeaways
Accelerate inference with Optimum and Transformers pipelines for improved performance and speed
Full Article
Published Time: 2022-05-10T00:00:00.077Z
# Accelerated Inference with Optimum and Transformers Pipelines
[Hugging Face](https://huggingface.co/)
* [Models](https://huggingface.co/models)
* [Datasets](https://huggingface.co/datasets)
* [Spaces](https://huggingface.co/spaces)
* [Buckets new](https://huggingface.co/storage)
* [Docs](https://huggingface.co/docs)
* [Enterprise](https://huggingface.co/enterprise)
* [Pricing](https://huggingface.co/pricing)
*
*
* * *
* [Log In](https://huggingface.co/login)
* [Sign Up](https://huggingface.co/join)
[Back to Articles](https://huggingface.co/blog)
# [](https://huggingface.co/blog/optimum-inference#accelerated-inference-with-optimum-and-transformers-pipelines) Accelerated Inference with Optimum and Transformers Pipelines
Published May 10, 2022
[Update on GitHub](https://github.com/huggingface/blog/blob/main/optimum-inference.md)
[- [x] Upvote 3](https://huggingface.co/login?next=%2Fblog%2Foptimum-inference)
* [](https://huggingface.co/chuangzhidian "chuangzhidian")
* [](https://huggingface.co/sparsh2002 "sparsh2002")
* [](https://huggingface.co/Marially "Marially")
[](https://huggingface.co/philschmid)
[Philipp Schmid philschmid Follow](https://huggingface.co/philschmid)
> * [1. What is Optimum? An ELI5](https://huggingface.co/blog/optimum-inference#1-what-is-optimum-an-eli5 "1. What is Optimum? An ELI5")
>
> * [2. New Optimum inference and pipeline features](https://huggingface.co/blog/optimum-inference#2-new-optimum-inference-and-pipeline-features "2. New Optimum inference and pipeline features")
>
> * [3. End-to-End tutorial on accelerating RoBERTa for Question-Answering including quantization and optimization](https://huggingface.co/blog/optimum-inference#3-end-to-end-tutorial-on-accelerating-roberta-for-question-answering-including-quantization-and-optimization "3. End-to-End tutorial on accelerating RoBERTa for Question-Answering including quantization and optimization")
> * [3.1 Install `Optimum` for Onnxruntime](https://huggingface.co/blog/optimum-inference#31-install-optimum-for-onnxruntime "3.1 Install <code>Optimum</code> for Onnxruntime")
>
> * [3.2 Convert a Hugging Face `Transformers` model to ONNX for inference**](https://huggingface.co/blog/optimum-inference#32-convert-a-hugging-face-transformers-model-to-onnx-for-inference "3.2 Convert a Hugging Face <code>Transformers</code> model to ONNX for inference**")
>
> * [3.3 Use the `ORTOptimizer` to optimize the model](https://huggingface.co/blog/optimum-inference#33-use-the-ortoptimizer-to-optimize-the-model "3.3 Use the <code>ORTOptimizer</code> to optimize the model")
>
> * [3.4 Use the `ORTQuantizer` to apply dynamic quantization](https://huggingface.co/blog/optimum-inference#34-use-the-ortquantizer-to-apply-dynamic-quantization "3.4 Use the <code>ORTQuantizer</code> to apply dynamic quantization")
>
> * [3.5 Run accelerated inference using Transformers pipelines](https://huggingface.co/blog/optimum-inference#35-run-accelerated-inference-using-transformers-pipelines "3.5 Run accelerated inference using Transformers pipelines")
>
> * [3.6 Evaluate the performance and speed](https://huggingface.co/blog/optimum-inference#36-evaluate-the-performance-and-speed "3.6 Evaluate the performance and speed")
>
>
> * [4. Current Limitations](https://huggingface.co/blog/optimum-inference#4-current-limitations "4. Current Limitations")
>
> * [5. Optimum Inference FAQ](https://hu
# Accelerated Inference with Optimum and Transformers Pipelines
[Hugging Face](https://huggingface.co/)
* [Models](https://huggingface.co/models)
* [Datasets](https://huggingface.co/datasets)
* [Spaces](https://huggingface.co/spaces)
* [Buckets new](https://huggingface.co/storage)
* [Docs](https://huggingface.co/docs)
* [Enterprise](https://huggingface.co/enterprise)
* [Pricing](https://huggingface.co/pricing)
*
*
* * *
* [Log In](https://huggingface.co/login)
* [Sign Up](https://huggingface.co/join)
[Back to Articles](https://huggingface.co/blog)
# [](https://huggingface.co/blog/optimum-inference#accelerated-inference-with-optimum-and-transformers-pipelines) Accelerated Inference with Optimum and Transformers Pipelines
Published May 10, 2022
[Update on GitHub](https://github.com/huggingface/blog/blob/main/optimum-inference.md)
[- [x] Upvote 3](https://huggingface.co/login?next=%2Fblog%2Foptimum-inference)
* [](https://huggingface.co/chuangzhidian "chuangzhidian")
* [](https://huggingface.co/sparsh2002 "sparsh2002")
* [](https://huggingface.co/Marially "Marially")
[](https://huggingface.co/philschmid)
[Philipp Schmid philschmid Follow](https://huggingface.co/philschmid)
> * [1. What is Optimum? An ELI5](https://huggingface.co/blog/optimum-inference#1-what-is-optimum-an-eli5 "1. What is Optimum? An ELI5")
>
> * [2. New Optimum inference and pipeline features](https://huggingface.co/blog/optimum-inference#2-new-optimum-inference-and-pipeline-features "2. New Optimum inference and pipeline features")
>
> * [3. End-to-End tutorial on accelerating RoBERTa for Question-Answering including quantization and optimization](https://huggingface.co/blog/optimum-inference#3-end-to-end-tutorial-on-accelerating-roberta-for-question-answering-including-quantization-and-optimization "3. End-to-End tutorial on accelerating RoBERTa for Question-Answering including quantization and optimization")
> * [3.1 Install `Optimum` for Onnxruntime](https://huggingface.co/blog/optimum-inference#31-install-optimum-for-onnxruntime "3.1 Install <code>Optimum</code> for Onnxruntime")
>
> * [3.2 Convert a Hugging Face `Transformers` model to ONNX for inference**](https://huggingface.co/blog/optimum-inference#32-convert-a-hugging-face-transformers-model-to-onnx-for-inference "3.2 Convert a Hugging Face <code>Transformers</code> model to ONNX for inference**")
>
> * [3.3 Use the `ORTOptimizer` to optimize the model](https://huggingface.co/blog/optimum-inference#33-use-the-ortoptimizer-to-optimize-the-model "3.3 Use the <code>ORTOptimizer</code> to optimize the model")
>
> * [3.4 Use the `ORTQuantizer` to apply dynamic quantization](https://huggingface.co/blog/optimum-inference#34-use-the-ortquantizer-to-apply-dynamic-quantization "3.4 Use the <code>ORTQuantizer</code> to apply dynamic quantization")
>
> * [3.5 Run accelerated inference using Transformers pipelines](https://huggingface.co/blog/optimum-inference#35-run-accelerated-inference-using-transformers-pipelines "3.5 Run accelerated inference using Transformers pipelines")
>
> * [3.6 Evaluate the performance and speed](https://huggingface.co/blog/optimum-inference#36-evaluate-the-performance-and-speed "3.6 Evaluate the performance and speed")
>
>
> * [4. Current Limitations](https://huggingface.co/blog/optimum-inference#4-current-limitations "4. Current Limitations")
>
> * [5. Optimum Inference FAQ](https://hu
DeepCamp AI