How to Use OpenAI Whisper to Fix YouTube Search

OpenAI's Whisper is a new open-source, state-of-the-art speech-to-text. This project will use Open AI Whisper to make YouTube search amazing. Search on YouTube is good but has its limitations. With trillions of hours of content, there should be an answer to almost every question. Yet, if we have a specific question like "what is OpenAI's CLIP?" — we're served dozens of 20+ minute videos. What if all we want is a short 20-second explanation? The current YouTube search has no solution for this. Maybe there's a good reason to encourage users to watch as much of a video as possible (more ads, anyone?). Either way, the solution to our problem may have come from OpenAI's *Whisper*. This video will explore the idea behind a better search and how to build it with Whisper, transformers, and vector search. The search stack includes: - pytube for video/mp3 download - OpenAI Whisper for audio transcription / automatic speech recognition (ASR) / speech-to-text - Sentence transformers like SBERT (via Hugging Face) for building sentence embeddings - Pinecone vector database for scalable and fast vector search - Streamlit for app development - Hugging Face spaces for app hosting 🌲 Pinecone article: https://pinecone.io/learn/openai-whisper/ 🤖 70% Discount on the NLP With Transformers in Python course: https://bit.ly/3DFvvY5 🎉 Subscribe for Article and Video Updates! https://jamescalam.medium.com/subscribe https://medium.com/@jamescalam/membership 👾 Discord: https://discord.gg/c5QtDB9RAP 00:00 OpenAI's Whisper 01:48 Idea Behind Better Search 06:56 Downloading Audio for Whisper 08:22 Download YouTube Videos with Python 16:52 Speech-to-Text with OpenAI Whisper 20:54 Hugging Face Datasets and Preprocessing 26:48 Using a Sentence Transformer 27:45 Initializing a Vector Database 28:45 Build Embeddings and Vector Index 31:35 Asking Questions 34:08 Hugging Face Ask YouTube App #machinelearning #python #openai