LangChain explained: external documents & SQL in GPT-4

Key Takeaways

hello hello hello Community Lang chain explained a lot of my subscribers asked here for this video so what is the environment I suppose we're gonna work with openai with gpt4 and then I will show you Lang chain it's an open source initiative by the community and of course all the connectivities to external service provider with a commercial API so this is the environment I'm gonna talk today now let's start we have here our open UI our gpt4 and there are a lot of other providers of llm models of large language models but we focus today on open AI but you can go with hugging face you can go with cohere whatever model you like from whoever so and the task is a very simple task we have question and answer so let's have here our documents let's say we have 10 000 documents and we have a question about a specific content in one of those 10 000 documents or about 100 documents that have gross references so how do we start that we use here our llm to analyze use the artificial intelligence to come up with an answer given that this is our data stack so beautiful I'm gonna do it in a way that I tell you have a look at this video by the YouTube channel by Sophia Young she is the one who's presenting the complete code whatever I present today I build upon her video she is only coding and I explain exactly what is going on and I show you I paint the overall picture so I have a very beautiful job today if you go to her Channel maybe you like it after 17 YouTube channels I decided to go with her she is also a Jupiter notebook in the description of the video so you can download this and you can follow so here we go now first thing is of course you take all your 10 000 documents with millions and tens of millions of token and you use here long chain with its very simple command two lines of code and you say okay for the LL app I take here my open EI and I have a chain type and this is just stuff it all together make one huge string of whatever and this crashes immediately because you know in the current version we have 4K prompt a 32k prompt may be coming up in the next month but for the moment we have everything that is has a length more than 4K tokens as an input to the llm will crash immediately but it has a beautiful solution when you choose here the second chain type map reduce you remember map reduce hundreds of years ago when we were programming in data bricks before Pi spark was invented cheap my goodness what it does is easy it splits the document in a particular batch size and feed each batch into the llm so beautiful you have done millions of patches and each batch is fed into the llama and this is it now if you want to be a little bit more clever then you have the chain type refine refine is very easy it takes the answer that it gets back from gpt4 after it analyzed the batch number one and either there's zero there's nothing relevant to the question in this particular text document or it comes back with a summary or a short answer and then this answer of patch number one is concatenated to the complete batch 2 so you see the longer you go hundred thousand ten thousand batches the longer you run wherever in whatever document there might be even the slightest reference to your question this information is extracted and you have a beautiful cross reference over ten hundred thousand documents and if you want to have a threshold that you get an quality indicator and you say okay for each batch I get Now 50 percent 70 a 95 quality so you say hey I want to have a threshold everything below 90 ignore everything above 91 take it and go with it so this is a very easy way you take care your millions and tens of millions of token and if you do this you better do not look at the bill that you will receive because remember that here uh opening my charges you based on the amount of tokens and you have tens of millions of tokens you are very fast into thousands of US dollars so although this is a methodology and if you say money doesn't care whatever the cost it costs beautiful you found your solution now for the rest of us yeah and then you got an official answer of course now for the rest of us we would like to spend not so much money so how we do this now I told you Lang chain here an initiative by the community open source they found a very nice way and it's called retrieval question and answer chain of course it uses the first method and they do it so what we do we pre-select our documents this is all there is to it before we feed all the documents to our gpt4 or any other llm we say hey in ninety percent of those documents there's no information to answer my question to execute my task so therefore we pre-select the documents so how we do this we need a reader and long chain gives us a reader CSV files data frame loader emails PDF PowerPoint Word documents I think more than 100 readers are available so whatever form your document is you'll find it here in the reader beautiful second if your PDF as I just showed you has I don't know 100 Pages you have to split it in chunks because 100 Pages we do not get 100 pages in one Rush here into our gpd4 model so you split up the documents in small chunks in small pieces what you know nltk tax splitter but you choose any tag splitter that you like now now it becomes interesting because now that way I have small text chunks and you know what we want to do we want to compare a semantic similarity to my question so how we calculate semantic similarity in a computer yes exactly like we have done for the last 10 years with Transformers with bird models with sentence Transformer with any Vector representation of data in a vector space that we can construct out of the vocabulary of the data set so we use embeddings and just to show you here about one year ago I made a video about the ipcc report 3600 pages and used exactly here as 3D visualization here of a sentence burden padding so let me show you how does it look like here for example if I perform the embedding of 3600 Pages this is example this is now a 3D visualization where you see each dot each cluster here each dot is a sentence and the sentence is cluster antimatter cluster and you see here urban climate climate change food security global warming decision making decision uncertainty so here you have now a 3D visualization of sentences that became here in a vector space a vector and you have all sentences of this report you can then dive into if you want to have a correlation between two specific topics here in a 3D visualization these are all the different sentences and a vector combination what else can I show you you can have different correlation you can have different topological analysis you can have a cluster analysis so whatever you want to do here you can do amazing things you can have I don't know you can Define how many thematic clusters you want to have you see here in 3600 pages I had close to 100 schematic clusters and you can zoom in you can have a detailed analysis of everything so this is something we're familiar with two years ago one year ago I had all detailed videos if you want to code this for yourself and in bearings you have a huge Choice Rico since we are working here we want to work with open AI we go with the open AI embeddings so this means each word or each token is now transformed into a vector a vector is a very simple representation in a vector space if we have more complexity we will go for a matrix and then a tensor representation and then we will have a tensor space so you see we convert the text with some mathematics to simple mathematical objects and we can calculate with simple mathematical objects and we can use for example the cosine similarity function between vectors to find out if they are semantically correlated so and here you have now a vector database and you say database yeah it's a data it's a simple old-fashioned database remember here 10 20 years ago so and you have a lot of providers and a lot of providers are professional providers are commercially providers they have here beautiful high performance API there may be cloud-based whatever you choose here from this list you take your credit card you inscribe for one month for one year whatever you are intention is and cloud-based or not you get an API key so you can use now here the vector database where all your transcription from the human readable text is now in a vector now of course in the old days we just kept it on our local PC on our local compute cluster so you can also do this here locally but if you have 10 000 clients I would recommend going with a professional solution so this is this and then of course you want here to find the relevant parts of your document I told you we pre-select here our document stack you have your question now you take the wording of the question and you transform now each word or age whatever token answer you have it's specific token into a vector so then you have a vector representation of your query let's say I don't know 768 dimensional Vector where this is your query and then you compare here to all the other vectors that are based on your document because you want to find semantic similar sentences or semantic similar paragraphs of text and we go here over the mathematical construct of vector similarity in a vector space and this is exactly what the famous retriever does it just says okay I take my very vector Omega cosine similarity operation and the 10 most similar vectors I take and either I transform Back To Human readable text and then you can have a look of it but suddenly all here your stack of tens of thousands of documents suddenly you just have 10 documents so you see what we achieved we have now reduced here outside of openai outside of llm with here if you want some Cloud professional service provider with some yeah of course you have to pay for weekly monthly or yearly rate whatever you choose we have now reduced here this pile of documents let's say to 900 words and a lot of people ask me what is an index this here those types the reader the splitter the vector database or the vector store or the vector hyper parameter however you like to call it this is the function they perform and what they tell you either they give you exactly the sentences or if you want imagine this is a newspaper new document is a newspaper what is the index what does it mean index is just a page number you look in the New York Times for example for I don't know financial news and you find out now that on page 12 of the financial times you have exactly the contents that you're looking for and the system tells you a on page 45 there is a addendum or there is some correction or some other important information this is an index there's nothing complicated to an index it just extracts the corresponding relevant wording and gives you here hey page 12 of The New York Times on page 45. this is the information you get back you do not have to read in this horrible stack here but you have only two pages from The New York Times so you understand what a long chain index is for a given query you get the relevant text back from your document stack your documents documents you downloaded from the internet PDF CSV file what have a hundred different forms of document can be calculated so beautiful and now that we have this so now we do exactly like before we have not a relevant text let's say there's a I don't know 900 words and each word has four tokens so we are below 4000 token then we have the question we tokenize our question and we are below 4K who are prompt you know that again we have different ways of changing the information together and then we just feed this in and this is now all there is to it if you have some external documents whatever the form is with your blank chain index anything else that I forgot no I don't think so yeah I know of course you got back from open UI from gpt4 your question was hey is this specific molecule in the biopharma environment toxicity the answer is yeah the toxicity of this singular molecule is for example I so this is all there is to it and the if you want the intelligent is not happening now exclusive in your llm because it's too expensive so you with Lang chain you are able to have a reader a splitter external databases either professional or you you can do those with yourself you do not need a professional service provider but if you will have let's say 100 000 apis per hour you should go with some professional but you see this is all there is to it now of course if you have now let's say some specific task and you do not know how to handle this task so what we do I for example I connect here to chat GPT I have here my prompt I say hey Jets GPT I have this problem how would you structure here my action point to come to a solution so that I can handle this problem and this is exactly what in Lang chain an agent is doing this is the job of an agent so an agent ask the intelligence we say we go with openai gpt4 what to do give them a specific topic problem whatever it tells them hey create a list of action item four breakfast and then what we get is nine response from gpt4 tells us as an answer this is your first action you can do second action you should do in the third action you should do beautiful so now we know what are our action points and we have as an agent in Lang chain we have here tools and tool boxes and whatever you call it we have tools what are those tools those tools are data pipelines to commercial service provider what you know you know for example serp API if you say I want here I need here to connect to Google search I have a specific question to Google search and come back with the answer from Google Search which of course you just could go to Google type in your question and string copy your answer but we want to optimize it you have here a service provider serp API and if you go for a semi professional version it's 50 bucks per month for if you go for the professional version It's 250 dollar per month and you have so mined many service provider and the tools that an agent can use now is I don't know I think there are more than 300 service provider available on the internet some are free some are on a freemium based model and some simply say hey provide us your credit card details and we are happy to provide our professional service to you so your Wikipedia you have World from alpha you have Weather Channels you have chat GPT plugins you have Bing search if you do not like a Google search so you choose and the last one you see in the list is sapier and there you have access to more than five I think it's more than five thousand apps right now so it is an unbelievable amount of professional service provider that you can connect to and if you have an account there if you already pay for some of their services I would say check out if they have a compatible tool for Lang train your agents if not you have to pay for whatever service you need so beautiful now the first action as I told you was go and perform a Google search on automated Google search use the apis to ask the question and come back with the question beautiful so we have now here text that is the answer to the first action item the second action is okay and now you have to connect to a very specific Finance database I don't know where you have ID and authorization and of course you have to pay for API access but if you want some specific Market data financial data whatever you want this is now the second action connect ask your question get the answer and come back with the answer and let's say the third action was exactly what we did here before we had here a question for this documents those are my documents my internal documents and now I have here my internal documents so I have if you want a generic Google search I have a detailed Financial database search either Google with SQL or no SQL or whatever database there are so many service provider out there in the internet that will do and will provide the job when the third action is here my personal accounts for example so you see this is it and the agent now comes back from using all the different tools that he needs for the job and he whenever he encounters a problem he goes back to gpt4 and ask for help and super T4 tells the agent what he should do next so you see the agent is just a list where on the list is one two three when you go shopping we need butter we need milk we need bread this is it an agent is imagined is simply a list you do when you go shopping okay real intelligence is of course within gpt4 within llms within all your AI systems because those are just databases those are just whatever you need but the intelligence is here but remember every time you connect back to ask for advice you have to pay so just to make sure it is not as bad as here but you have to pay because you need the intelligence from gpd4 so beautiful now you have your agent so and now you know what's coming because now you can change this information together you have a highly dense information one two and three you remember your four ways to train your information together and if you have changed it together and you validated it the moment comes that it sends it off now this highly complex agent model here with our new prompt and we feed it into gpt4 and you're not gonna believe it after some time we get an answer back and the answer is for example the toxicity of this financial there were at X Y I don't know what a bond structure is high so whatever your original question was now you have an answer based on your financial accounts based on a specific Google search on the risk of this financial product and you also had here access to an expert database a finance database where you checked the I don't know performance of the last five years that were confidential so you have here a complexity of interaction that length chain is really helpful to perform for you so again Lang Chang is open source but whenever you need a commercial and a Prof for-profit service provider to provide some service for you over Lang chain remember you have to pay them first you get your API key and then the whole thing starts running so this is it and I say thank you and if you're interested in a practical example there is I got a lot of question can you live without length chain can I do if I just go for one or two times can you do it without language and the answer is of course yes let's have a look at some funny examples you know they are so boring examples so here we have Lang Chan here we have read the docs as I showed you we have six modules we have the llm model we have the prompt we have what I showed you is the index memory is just that somebody is constantly writing what you are doing and we just had a specific look at chains and now agents and agents was also in my last slide but you know if you want to know if you can do what an agent does without Lang chain imagine this here we have our beautiful Agent Zero nothing seven and of course as an agent he has his tools You Know M provides him with some I don't know some beautiful car or whatever an airplane or whatever so and then as an agent normally here within Lang train you ask an intelligent llm what to do if he is not if he doesn't know what to do system goes back here over length Gen 2 gpt4 now imagine this here now on a human level so our beautiful agent series here nothing ask now Moneypenny how to prepare for a specific task how to prepare breakfast now Moneypenny provides him with a list of some detailed instruction that he has to work on this is exactly what our Lang chain agent would do now we have here zero zero seven doing exactly the same weight and you remember tools are ways that an agent can use to interact with his environment with his ecosystem with other agents so beautiful and if you ask for the tool yes again this is the list of tools and do not forget I've seen a YouTube video where there was a competition how many micro service provider you can subscribe to and for Lang chain and there was one who subscribed to 16 professional service provider to credit card name security code and gets the API back and it was done in under one hour so if you want to have a benchmark if you sign up with your credit card for all the external provider can be done for 16 under one hour if you need this but if you now think about what would be an example here for length chain or without length and for our agent imagine the job to be done now if we are talking here about breakfast what would happen now the artificial intelligence system would connect via a link chain agent to the serp API to ask Google for some ideas for your breakfast choice and Google would return about 125 documents on different breakfast options now the other person without a system or whatever you're interacting with so let's say her length chain Avatar provides also her personal preferences so what happens you bring those data together in your vector store analysis and there you have a closing similarity operation we are from those 125 breakfast options based on your physical condition in the day your detailed preferences her preferences a set of 10 breakfast choices is now calculated via a cosine similarity and the specific question that you ask for this is the job so if you have done this set of 10 breakfast choices 10 is a small enough number that we can enter this as an input to our open EI API for gpt4 and gpd4 performs on highly intelligent ranking given that those 10 breakfast set has to be analyzed given this situational appropriateness those two agents are currently interacting with and of course there's a winner and gpt4 returns here one breakfast option say hey given for YouTube this is the best breakfast for the particular job so gpt4 comes back with an answer instructs now a different length chain agent for example to buy the ingredients online or buy the products online for coffee beans a specific bread champagne whatever you like for breakfast and have of course instant home delivery and now it happens that suddenly one Lang Chan agent says hey wait a minute there was something so there was an unforeseen activation of an agent and asked that TPT or gpt4 here for some some corresponding music for this particular type of breakfast so again you use the intelligence of llm to find here the perfect music for this particular breakfast for this particular situation of our two agents and then another agent activates connects to the API of the music stores paste there with your credit card or does it online downloads it immediately and instructs Siri to play the music for the breakfast so you see this would be here now a non-standard example that you see exactly what length chain is doing interacting with an llm and interacting with external professional service provider and so if this was would be the job this could be here exactly what is happening under the hood and now it is up to you if you want to decide if you are in this situation and you have given here as an agent a particular job if you want to apply length chain or you enjoy your professional life without length chain I say thank you and I see you in my next video