How to use LangChain for RAG over audio files

AssemblyAI · Beginner ·🔍 RAG & Vector Search ·2y ago

Skills: RAG Basics61%LLM Engineering53%

Retrieval Augmented Generation (RAG) is a technique to increase the relevance and transparency of LLM responses. RAG works by supplying the LLM with documents that are relevant to a query. In this tutorial, we'll learn how to perform RAG on audio data using LangChain and Python. APIs: - AssemblyAI API key: https://www.assemblyai.com/dashboard/signup - OpenAI API key: https://openai.com/blog/openai-api Resources: - Code: https://github.com/AssemblyAI-Examples/rag-langchain-audio-data - Blog: https://www.assemblyai.com/blog/retrieval-augmented-generation-audio-langchain/ - LangChain webinar series: https://www.youtube.com/@LangChain Tools used: - AssemblyAI: https://www.assemblyai.com/ - LangChain: https://www.langchain.com/ - Hugging Face: https://huggingface.co/ - Chroma: https://www.trychroma.com/ - OpenAI: https://openai.com/ ▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬ 🖥️ Website: https://www.assemblyai.com/?utm_source=youtube&utm_medium=referral&utm_campaign=yt_ry_1 📈 Newsletter: https://www.assemblyai.com/subscribe 🐦 Twitter: https://twitter.com/AssemblyAI 🦾 Discord: https://discord.gg/Cd8MyVJAXd ▶️ Subscribe: https://www.youtube.com/c/AssemblyAI?sub_confirmation=1 🔥 We're hiring! Check our open roles: https://www.assemblyai.com/careers ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ #MachineLearning #deeplearning 0:00 Introduction 0:34 Repo and Blog 0:46 What is Retrieval Augmented Generation (RAG)? 0:59 Implementation overview 1:42 Environment setup 2:54 Imports 4:10 Specifying audio files 4:35 Audio document loader 5:12 Text splitting 6:04 Setting metadata 6:29 Text embedding 6:52 Building the Chroma database 7:10 Building the QA LangChain 7:50 Make application loop 8:50 Running the application 9:28 Asking a question and RAG response

Watch on YouTube ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from AssemblyAI · AssemblyAI · 0 of 60

← Previous Next →

Python Speech Recognition in 5 Minutes

Python Speech Recognition in 5 Minutes

Python Click Part 1 of 4

Python Click Part 1 of 4

Python Click Part 2 of 4

Python Click Part 2 of 4

Python Click Part 3 of 4

Python Click Part 3 of 4

Python Click Part 4 of 4

Python Click Part 4 of 4

Deep learning in 5 minutes | What is deep learning?

Deep learning in 5 minutes | What is deep learning?

How to make a web app that transcribes YouTube videos with Streamlit | Part 1

How to make a web app that transcribes YouTube videos with Streamlit | Part 1

How to make a web app that transcribes YouTube videos with Streamlit | Part 2

How to make a web app that transcribes YouTube videos with Streamlit | Part 2

Batch normalization | What it is and how to implement it

Batch normalization | What it is and how to implement it

Real-time Speech Recognition in 15 minutes with AssemblyAI

Real-time Speech Recognition in 15 minutes with AssemblyAI

Regularization in a Neural Network | Dealing with overfitting

Regularization in a Neural Network | Dealing with overfitting

Add speech recognition to your Streamlit apps in 5 minutes

Add speech recognition to your Streamlit apps in 5 minutes

Transformers for beginners | What are they and how do they work

Transformers for beginners | What are they and how do they work

Automatic Chapter Detection With AssemblyAI | Python Tutorial

Automatic Chapter Detection With AssemblyAI | Python Tutorial

Deep Learning Series Part 1 - What is Deep Learning?

Deep Learning Series Part 1 - What is Deep Learning?

Deep Learning Series part 2 - Why is it called “Deep Learning”?

Deep Learning Series part 2 - Why is it called “Deep Learning”?

Activation Functions In Neural Networks Explained | Deep Learning Tutorial

Activation Functions In Neural Networks Explained | Deep Learning Tutorial

Deep Learning Series part 3 - Deep Learning vs. Machine Learning

Deep Learning Series part 3 - Deep Learning vs. Machine Learning

Deep Learning Series part 4 - Why is Deep Learning better for NLP?

Deep Learning Series part 4 - Why is Deep Learning better for NLP?

Intro to Batch Normalization Part 1

Intro to Batch Normalization Part 1

Intro to Batch Normalization Part 2

Intro to Batch Normalization Part 2

Intro to Batch Normalization Part 3 - What is Normalization?

Intro to Batch Normalization Part 3 - What is Normalization?

Intro to Batch Normalization Part 4

Intro to Batch Normalization Part 4

Intro to Batch Normalization Part 5

Intro to Batch Normalization Part 5

Sentiment Analysis for Earnings Calls with AssemblyAI

Sentiment Analysis for Earnings Calls with AssemblyAI

Summarizing my favorite podcasts with Python

Summarizing my favorite podcasts with Python

Introduction to Regularization

Introduction to Regularization

How/Why Regularization in Neural Networks?

How/Why Regularization in Neural Networks?

Getting Started With Torchaudio | PyTorch Tutorial

Getting Started With Torchaudio | PyTorch Tutorial

Types of Regularization

Types of Regularization

Tuning Alpha in L1 and L2 Regularization

Tuning Alpha in L1 and L2 Regularization

Dropout Regularization

Dropout Regularization

What is GPT-3 and how does it work? | A Quick Review

What is GPT-3 and how does it work? | A Quick Review

Backpropagation For Neural Networks Explained | Deep Learning Tutorial

Backpropagation For Neural Networks Explained | Deep Learning Tutorial

Jupyter Notebooks Tutorial | How to use them & tips and tricks!

Jupyter Notebooks Tutorial | How to use them & tips and tricks!

Best Free Speech-To-Text APIs and Open Source Libraries

Best Free Speech-To-Text APIs and Open Source Libraries

Regularization - Early stopping

Regularization - Early stopping

Regularization - Data Augmentation

Regularization - Data Augmentation

Bias and Variance for Machine Learning | Deep Learning

Bias and Variance for Machine Learning | Deep Learning

Recurrent Neural Networks (RNNs) Explained - Deep Learning

Recurrent Neural Networks (RNNs) Explained - Deep Learning

What is BERT and how does it work? | A Quick Review

What is BERT and how does it work? | A Quick Review

Introduction to Transformers

Introduction to Transformers

Transformers | What is attention?

Transformers | What is attention?

Transformers | how attention relates to Transformers

Transformers | how attention relates to Transformers

Transformers | Basics of Transformers

Transformers | Basics of Transformers

Supervised Machine Learning Explained For Beginners

Supervised Machine Learning Explained For Beginners

Transformers | Basics of Transformers Encoders

Transformers | Basics of Transformers Encoders

Transformers | Basics of Transformers I/O

Transformers | Basics of Transformers I/O

How to evaluate ML models | Evaluation metrics for machine learning

How to evaluate ML models | Evaluation metrics for machine learning

Unsupervised Machine Learning Explained For Beginners

Unsupervised Machine Learning Explained For Beginners

Weight Initialization for Deep Feedforward Neural Networks

Weight Initialization for Deep Feedforward Neural Networks

Q-Learning Explained - Reinforcement Learning Tutorial

Q-Learning Explained - Reinforcement Learning Tutorial

Should You Use PyTorch or TensorFlow in 2022?

Should You Use PyTorch or TensorFlow in 2022?

What is Layer Normalization? | Deep Learning Fundamentals

What is Layer Normalization? | Deep Learning Fundamentals

I created a Python App to study FASTER

I created a Python App to study FASTER

How to create your FIRST NEURAL NETWORK with TensorFlow!

How to create your FIRST NEURAL NETWORK with TensorFlow!

Neural Networks Summary: All hyperparameters

Neural Networks Summary: All hyperparameters

Getting Started with OpenAI API and GPT-3 | Beginner Python Tutorial

Getting Started with OpenAI API and GPT-3 | Beginner Python Tutorial

Convert Speech-To-Text In Python in 60 seconds!

Convert Speech-To-Text In Python in 60 seconds!

Gradient Clipping for Neural Networks | Deep Learning Fundamentals

Gradient Clipping for Neural Networks | Deep Learning Fundamentals

More on: RAG Basics

View skill →

High Performance (Realtime) RAG Chains: From Basic to Advanced

High Performance (Realtime) RAG Chains: From Basic to Advanced

Coding the Ultimate RAG Engine from Zero

Coding the Ultimate RAG Engine from Zero

Build an LLM and RAG-based Chat Application using AlloyDB and LangChain

RAG Demo for Beginners: Full Hands-On Tutorial in Tamil | Build Your Own RAG AI | Karthik's Show

RAG Demo for Beginners: Full Hands-On Tutorial in Tamil | Build Your Own RAG AI | Karthik's Show

RAG with LangChain on Google Cloud

RAG with LangChain on Google Cloud

Google Cloud Tech

Build an End-to-End RAG API with AWS Bedrock & Azure OpenAI

Build an End-to-End RAG API with AWS Bedrock & Azure OpenAI

Related AI Lessons

RAG Evaluation with RAGAS: Measuring Faithfulness, Context Precision, and Recall in Production

Learn to evaluate RAG models using RAGAS, measuring faithfulness, context precision, and recall in production environments

Dev.to · Anna Danilec

Chunking for RAG: stop tuning the wrong knob

Learn how to optimize RAG performance with a practical chunking playbook, avoiding common pitfalls and improving evaluation metrics

Dev.to · saurabh naik

Your RAG Pipeline Isn’t Broken. Your Chunks Are.

Learn how to optimize your RAG pipeline by fixing common issues with document chunks

Your RAG Pipeline Isn’t Broken. Your Chunks Are.

Learn how to optimize your RAG pipeline by fixing chunking issues, a crucial step in improving retrieval performance

Chapters (16)

Introduction

0:34 Repo and Blog

0:46 What is Retrieval Augmented Generation (RAG)?

0:59 Implementation overview

1:42 Environment setup

2:54 Imports

4:10 Specifying audio files

4:35 Audio document loader

5:12 Text splitting

6:04 Setting metadata

6:29 Text embedding

6:52 Building the Chroma database

7:10 Building the QA LangChain

7:50 Make application loop

8:50 Running the application

9:28 Asking a question and RAG response

Watch this before applying for jobs as a developer.