Using GPT4 Vision & LangChain to Query Memories from a Video

In this video lets experiment with using GPT-4 Vision to craft video descriptions and then use a simple retrieval augmented generation approach leveraging LangChain to query those descriptions! Discover how we navigate through the process of creating a descriptive log from video clips – turning captured memories across the city into searchable text data. Important disclaimer: I am using the word "memories" here loosely, it just means objects and entities seen in a video. 📚 Chapters 00:01 - Introduction and project idea explanation 01:09 - Creating a new notebook and outlining the approach 01:35 - Discussing the need for a descriptor from video clips using GP4 Vision 02:23 - Testing live with a frame from a sample video 04:38 - Integrating descriptions with a querying system 05:46 - Fetching code snippets from OpenAI's cookbook for the project setup 07:10 - Setting up a conda environment and installing packages 12:18 - Loading and processing a sample video for description 14:15 - Writing a function to generate descriptions from videos 15:20 - Writing output descriptions to a text file and logging information 17:08 - Setting up text file generation with video file date extraction 19:28 - Beginning retrieval setup using Link Chain for querying descriptions 24:30 - Implementing the text loading and embedding process for retrieval 27:40 - Setting up the retrieval Q&A for querying document descriptions 32:01 - Running a test query on the system to find a specific video detail If you enjoyed this video don't forget to like, subscribe, and stay tuned for more coding sessions like this one! Cheers :)! 🔗 Links: - Subscribe!: https://www.youtube.com/channel/UCu8WF59Scx9f3H1N_FgZUwQ - Tiktok: https://www.tiktok.com/@enkrateialucca?lang=en - Twitter: https://twitter.com/LucasEnkrateia - LinkedIn: https://www.linkedin.com/in/lucas-soares-969044167/ Support the Channel! - Buy me a cup of coffee: https://tr.ee/7tYsD-tUu2 - Paypal: https://paypal.me