Retrieval Augmented Generation with Huggingface Transformers and Ray
📰 Hugging Face Blog
Hugging Face integrates Ray into Retrieval Augmented Generation (RAG) for faster and more scalable contextual document retrieval
Action Steps
- Integrate Ray into the RAG contextual document retrieval mechanism
- Use the Ray library to speed up retrieval calls and improve scalability
- Fine-tune RAG models using the integrated Ray library
Who Needs to Know This
NLP engineers and researchers can benefit from this integration to improve the performance of their RAG models, while software engineers can appreciate the scalability benefits of using Ray
Key Insight
💡 Integrating Ray into RAG can improve retrieval call speed by 2x and enhance scalability
Share This
🚀 Speed up your RAG models with Ray! 🤖
Key Takeaways
Hugging Face integrates Ray into Retrieval Augmented Generation (RAG) for faster and more scalable contextual document retrieval
Full Article
Published Time: 2021-02-10T00:00:00.012Z
# Retrieval Augmented Generation with Huggingface Transformers and Ray
[Hugging Face](https://huggingface.co/)
* [Models](https://huggingface.co/models)
* [Datasets](https://huggingface.co/datasets)
* [Spaces](https://huggingface.co/spaces)
* [Buckets new](https://huggingface.co/storage)
* [Docs](https://huggingface.co/docs)
* [Enterprise](https://huggingface.co/enterprise)
* [Pricing](https://huggingface.co/pricing)
*
*
* * *
* [Log In](https://huggingface.co/login)
* [Sign Up](https://huggingface.co/join)
[Back to Articles](https://huggingface.co/blog)
# [](https://huggingface.co/blog/ray-rag#retrieval-augmented-generation-with-huggingface-transformers-and-ray) Retrieval Augmented Generation with Huggingface Transformers and Ray
Published February 10, 2021
[Update on GitHub](https://github.com/huggingface/blog/blob/main/ray-rag.md)
[- [x] Upvote 6](https://huggingface.co/login?next=%2Fblog%2Fray-rag)
* [](https://huggingface.co/mggarifulla "mggarifulla")
* [](https://huggingface.co/mltrials "mltrials")
* [](https://huggingface.co/aslessor "aslessor")
* [](https://huggingface.co/ram1813 "ram1813")
* [](https://huggingface.co/miladai "miladai")
* [](https://huggingface.co/anadimisra "anadimisra")
[](https://huggingface.co/system)
[system system Follow](https://huggingface.co/system)
##### [](https://huggingface.co/blog/ray-rag#a-guest-blog-post-by-amog-kamsetty-from-the-anyscale-team) A guest blog post by [Amog Kamsetty](https://huggingface.co/amogkam) from the Anyscale team
* [What’s next?](https://huggingface.co/blog/ray-rag#whats-next "What’s next?")
[Huggingface Transformers](https://huggingface.co/) recently added the [Retrieval Augmented Generation (RAG)](https://twitter.com/huggingface/status/1310597560906780680) model, a new NLP architecture that leverages external documents (like Wikipedia) to augment its knowledge and achieve state of the art results on knowledge-intensive tasks. In this blog post, we introduce the integration of [Ray](https://docs.ray.io/en/master/), a library for building scalable applications, into the RAG contextual document retrieval mechanism. This speeds up retrieval calls by 2x and improves the scalability of RAG distributed [fine-tuning](https://github.com/huggingface/transformers/tree/master/examples/research_projects/rag).
### [](https://huggingface.co/blog/ray-rag#what-is-retrieval-augmented-generation-rag) What is Retrieval Augmented Generation (RAG)?
[](https://huggingface.co/blog/assets/12_ray_rag/rag_gif.gif)
_An overview of RAG. The model retrieves contextual documents from an external dataset as part of its execution. These contextual documents are used in conjunction with the original input to produce an output. The GIF is taken from [Facebook's original blog post](https://ai.facebook.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models)._
Recently, [Huggingface](https://huggingface.co/) partnered with [Facebook AI](https://ai.facebook.com/) to introduce the [RAG](https://twitter.com/huggingface/status/1310597560906780680) model as part o
# Retrieval Augmented Generation with Huggingface Transformers and Ray
[Hugging Face](https://huggingface.co/)
* [Models](https://huggingface.co/models)
* [Datasets](https://huggingface.co/datasets)
* [Spaces](https://huggingface.co/spaces)
* [Buckets new](https://huggingface.co/storage)
* [Docs](https://huggingface.co/docs)
* [Enterprise](https://huggingface.co/enterprise)
* [Pricing](https://huggingface.co/pricing)
*
*
* * *
* [Log In](https://huggingface.co/login)
* [Sign Up](https://huggingface.co/join)
[Back to Articles](https://huggingface.co/blog)
# [](https://huggingface.co/blog/ray-rag#retrieval-augmented-generation-with-huggingface-transformers-and-ray) Retrieval Augmented Generation with Huggingface Transformers and Ray
Published February 10, 2021
[Update on GitHub](https://github.com/huggingface/blog/blob/main/ray-rag.md)
[- [x] Upvote 6](https://huggingface.co/login?next=%2Fblog%2Fray-rag)
* [](https://huggingface.co/mggarifulla "mggarifulla")
* [](https://huggingface.co/mltrials "mltrials")
* [](https://huggingface.co/aslessor "aslessor")
* [](https://huggingface.co/ram1813 "ram1813")
* [](https://huggingface.co/miladai "miladai")
* [](https://huggingface.co/anadimisra "anadimisra")
[](https://huggingface.co/system)
[system system Follow](https://huggingface.co/system)
##### [](https://huggingface.co/blog/ray-rag#a-guest-blog-post-by-amog-kamsetty-from-the-anyscale-team) A guest blog post by [Amog Kamsetty](https://huggingface.co/amogkam) from the Anyscale team
* [What’s next?](https://huggingface.co/blog/ray-rag#whats-next "What’s next?")
[Huggingface Transformers](https://huggingface.co/) recently added the [Retrieval Augmented Generation (RAG)](https://twitter.com/huggingface/status/1310597560906780680) model, a new NLP architecture that leverages external documents (like Wikipedia) to augment its knowledge and achieve state of the art results on knowledge-intensive tasks. In this blog post, we introduce the integration of [Ray](https://docs.ray.io/en/master/), a library for building scalable applications, into the RAG contextual document retrieval mechanism. This speeds up retrieval calls by 2x and improves the scalability of RAG distributed [fine-tuning](https://github.com/huggingface/transformers/tree/master/examples/research_projects/rag).
### [](https://huggingface.co/blog/ray-rag#what-is-retrieval-augmented-generation-rag) What is Retrieval Augmented Generation (RAG)?
[](https://huggingface.co/blog/assets/12_ray_rag/rag_gif.gif)
_An overview of RAG. The model retrieves contextual documents from an external dataset as part of its execution. These contextual documents are used in conjunction with the original input to produce an output. The GIF is taken from [Facebook's original blog post](https://ai.facebook.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models)._
Recently, [Huggingface](https://huggingface.co/) partnered with [Facebook AI](https://ai.facebook.com/) to introduce the [RAG](https://twitter.com/huggingface/status/1310597560906780680) model as part o
DeepCamp AI