Using GPT4 Vision & LangChain to Query Memories from a Video
In this video lets experiment with using GPT-4 Vision to craft video descriptions and then use a simple retrieval augmented generation approach leveraging LangChain to query those descriptions! Discover how we navigate through the process of creating a descriptive log from video clips – turning captured memories across the city into searchable text data.
Important disclaimer: I am using the word "memories" here loosely, it just means objects and entities seen in a video.
📚 Chapters
00:01 - Introduction and project idea explanation
01:09 - Creating a new notebook and outlining the approac…
Watch on YouTube ↗
(saves to browser)
Chapters (15)
0:01
Introduction and project idea explanation
1:09
Creating a new notebook and outlining the approach
1:35
Discussing the need for a descriptor from video clips using GP4 Vision
2:23
Testing live with a frame from a sample video
4:38
Integrating descriptions with a querying system
5:46
Fetching code snippets from OpenAI's cookbook for the project setup
7:10
Setting up a conda environment and installing packages
12:18
Loading and processing a sample video for description
14:15
Writing a function to generate descriptions from videos
15:20
Writing output descriptions to a text file and logging information
17:08
Setting up text file generation with video file date extraction
19:28
Beginning retrieval setup using Link Chain for querying descriptions
24:30
Implementing the text loading and embedding process for retrieval
27:40
Setting up the retrieval Q&A for querying document descriptions
32:01
Running a test query on the system to find a specific video detail
Playlist
Uploads from Automata Learning Lab · Automata Learning Lab · 0 of 60
← Previous
Next →
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
A Quick Tutorial on NLP Basics
Automata Learning Lab
Automating your Digital Morning Routine with Python
Automata Learning Lab
Exploring Problem Solving with Python and Jupyter Notebook #1
Automata Learning Lab
Summarize Papers with Python and GPT-3
Automata Learning Lab
Automating Anki Flashcards with Python
Automata Learning Lab
An Experiment Tracking Tutorial with Mlflow and Keras
Automata Learning Lab
Automating Google Forms Submissions with Python
Automata Learning Lab
Productivity Tracking With Python and the Notion API
Automata Learning Lab
When your Machine Learning Model Fails Do This ;p
Automata Learning Lab
Machine Learning Tip#1 Practical Deep Learning Course
Automata Learning Lab
A Practical Summary of Matplotlib
Automata Learning Lab
Machine Learning Tips: Deep Learning Monitor
Automata Learning Lab
Machine Learning Tips#5 MLOPs specialization in Coursera #machinelearning
Automata Learning Lab
Automatically Changing Desktop Wallpaper with Python and the Nasa Image API
Automata Learning Lab
Building an Image Classifier to Filter Out Unused Images From Your Photo Album with Machine Learning
Automata Learning Lab
Automating VS Code Snippets with Python
Automata Learning Lab
How to Set Up a Machine Learning Environment with Conda and Pip-Tools
Automata Learning Lab
9 Google Search Tips for Machine Learning
Automata Learning Lab
Thinking Tools
Automata Learning Lab
Goal Planning for the IPad Pro
Automata Learning Lab
Automating Car Search with Python and Data Science
Automata Learning Lab
Generating Images from Text with Stable Diffusion and Hugging Face
Automata Learning Lab
A Practical Introduction to Data Science using the Spaceship Titanic Dataset from Kaggle
Automata Learning Lab
Quick Introduction to Python Syntax
Automata Learning Lab
Jiu Jitsu App with Python and Streamlit
Automata Learning Lab
Arbitrary Arguments in Python
Automata Learning Lab
2 Apps for Coding In The Ipad Pro
Automata Learning Lab
From Tensorflow to Pytorch?
Automata Learning Lab
Automate Lecture Notes with Python
Automata Learning Lab
Building an Audio Transcription App with OpenAI Whisper and Streamlit
Automata Learning Lab
Productivity Tracking with Python Short Summary
Automata Learning Lab
Summarize Papers with GPT3
Automata Learning Lab
Automating Expense Reports with Python
Automata Learning Lab
ChatGPT, Angry Pandas and AI Code
Automata Learning Lab
7 Strategies To Learn Anything Using ChatGPT
Automata Learning Lab
Building a Thought Summarization App with Whisper and GPT3
Automata Learning Lab
Automating your Morning Routine
Automata Learning Lab
Visualize a Neural Net Learning Polynomial Functions
Automata Learning Lab
Automating Notion with Python
Automata Learning Lab
Pose Tracking for Jiu Jitsu #automation #machinelearning #jiujitsu
Automata Learning Lab
Pose Tracking for Jiu Jitsu - Update #jiujitsu #machinelearning
Automata Learning Lab
Update to my Pose Tracking for Jiu Jitsu Project #machinelearning #jiujitsu #ai #deeplearning
Automata Learning Lab
ChatGPT API Released by OpenAI
Automata Learning Lab
ChatGPT API Response Format #machinelearning #ai #datascience
Automata Learning Lab
Beyond Stable Diffusion with Composer | Automata Learning Lab Paper Series #1
Automata Learning Lab
Beyond Diffusion Models with Composer #machinelearning #ai
Automata Learning Lab
Machine Learning for Jiu Jitsu
Automata Learning Lab
Finished my Project of Applying Pose Tracking to Jiu Jitsu #machinelearning #jiujitsu #ai
Automata Learning Lab
Prompt Engineering Basics #machinelearning #gpt4 #chatgpt
Automata Learning Lab
Visual ChatGPT: Integrating Images with ChatGPT Paper Series#2
Automata Learning Lab
Visual ChatGPT #machinelearning #ai #artificialintelligence
Automata Learning Lab
ViperGPT: Visual Reasoning with Code #machinelearning #ai
Automata Learning Lab
LERF - Language Embeddings + NERF for Querying 3D Spaces #machinelearning #ai
Automata Learning Lab
Summarize Papers with Python and ChatGPT
Automata Learning Lab
Large Language Models can use Tools Now! #artificialintelligence #machinelearning #ai
Automata Learning Lab
Sparks of AGI in GPT4? #machinelearning #ai #agi #artificialintelligence
Automata Learning Lab
Toolformer: LLMs can use Tools! #chatgpt #llms #gpt4 #gpt3 #artificialintelligence
Automata Learning Lab
Talking to Your Notes with LangChain #artificialintelligence #llms #gpt4 #chatgpt
Automata Learning Lab
MusicLM - From Text to Music #machinelearning #artificialintelligence
Automata Learning Lab
How to Talk to a PDF using LangChain and ChatGPT
Automata Learning Lab
DeepCamp AI