Unlock Enterprise Knowledge: Vectorizing Documents for AI /ChatGPT

Key Takeaways

hello community welcome today to a special edition user feedback now in one of my last videos I was talking about Beyond Vector databases and I got quite a lot of response from you and there's a particular comment I would like to focus on so the question or the comment was what if gpt4 does not have the specific domain knowledge that is represented by all of those documents that I mentioned in the video when I was talking about my particle physics papers hundreds and thousands of paper and if you ask you would need to upload all of them into gpt4 which would be expensive and hope that its representation match your own systems tokenized representation now I think it is a highly interesting question so let's focus to find an answer and try to explain what went wrong here with this chain of sword now at first gpd4 for me personally gpd4 is a black box so you might ask why because I do not understand how an answer is generated that I have for example only a One-Shot prompt and given that with a One-Shot prompt we can't alter the weights of the system so we have no fine tune ability at all but we know that the system can learn to a certain extent when we provide within our gpd4 prompt or chat GPT prompt examples one shot prompt few shot prompting the system can learn if you ask me tell me the mathematical Theory exactly how this happens within gbd4 I can't because for me it is a black box I can't also explain how gpd4 learns from in-context examples if you have few short prompts if you give multiple examples three four examples how can the system learn just without altering any weights there are some preprints that indicate yeah it is a system of the activations that form a model within the model but I haven't seen any clear plausible mathematical deduction a theory I can't explain this to you and this is why I think it is so important to understand what we can understand so what I know about gbd4 is gbd4 and chatubity is based purely on probabilities this means I have no reproducible results whenever I ask the system a question I will get a different answer why because it is based on probabilities there is no ground truth probabilities mean it can be altered and I don't even know when it will be altered so try it out yourself ask the same question different system different time of dates different session you will get different results so this is what we know and then a lot of my us ask questions about Vector database Vector store postgres cognitive search if you work on azure how does this connect to this black box and this is exactly the background to the question of the viewer so let's have a look at this what if gpd4 does not have the specific domain knowledge of my specific prompt query and I think yeah absolutely let's assume this gpd4 has not been pre-trained and fine-tuned on a particular data set like my Enterprise document or my pre-print published yesterday on a prep archive preprint server for particle physics there's no chance in the world that gpd4 has been pre-trained or fine-tuned on this data from yesterday so how do I provide new data to gpd4 now the second part of this question is his or her I don't know question is you would need to upload all of the particle physics paper into gpt4 now we cannot I cannot fine tune gpt4 no way at all I have no access to fine-tune GPT form so upload all of them into gbd4 and hope its representation so I think you are talking about a vector representation of all of the sentence in all of the documents match your own systems tokenized representation so there's a lot of problems going on so let's try to understand and try to find a solution so at first to make it simple let us assume there's only one paper on particle physics one document so instead of hundreds paper of particle physics we just go one easiest part now let us create a dummy document for this experiment because I wanted this is really a synthetic example there's no pre-trained knowledge Within gpd4 so you have here an insurance plan absolute synthetic fixtures company was just invented by GPT 3.5 Turbo this is your document beautiful it's just a dummy document it just has 578 words and more than 4 000 characters this is all we need to know and now comes the part where I say okay and now in this document I put one specific sentence I indicate this in red that is an answer to a question I'm gonna use chat GPT or tpt4 on this document and then we will see in to what extent we have to encode this document so let's do this very easy wait make four experiments we do it live you can watch it in real time so I will ask jet GPT a specific question it will always be the same specific question then second experiment I provide the document planized this is exactly here referred to as plan eyes this document completely this one I will provide in the prompt as a One-Shot prompt to Chai GPT and ask a specific question it is the same question as before that is not included in the pre-trained and fine-tuned tpt4 data set and not in planize then I include here this red sentence and now this is this version with the red sentence with the answer to my specific question and the document and ask to chat GPT or gpd4 the same specific question and then finally I provide that GPT in a new session we delete cache everything in my one shot prompt with only the answer sentence to my query prompt that I inputted in GPT form and we ask the same specific question so we will see if it is necessary to encode here or if any encoding happens do I have to provide the whole document or not so here I go I say hey chat TPT 3.5 only that's the insurance plan of the company Flying Eyes include hearing aids so Flying Eyes I invented there's no insurance plan so what will be the answer I don't have real-time information I'm not aware of any specific insurance plan of the company Flying Eyes this is great so now comes the interesting part so and here we go now I say it does the insurance plan of the company flying ice include hearing aids given that I provide you with new information about its specific insurance plan and then comes the complete insurance plan but you see it is not here life insurance uh vision insurance life insurance disability it is without hearing aids so what is the answer based on information provided about the flying irons and chewing plan it does not explicitly mention covering for hearing aids the plan details where an aspect of HealthCare coverage yes yes yes the specific inclusion or exclusion of hearing aids is not mentioned beautiful so we have now the complete document provided in if you want here a One-Shot prompt we give them here the complete map the complete insurance plan of our fixtures company flying eyes and we get the answer it is not included so what do we do now I just copy this is a new information okay so now I've given it the information that it is included where is it hearing ads here we offer financial support given our employee is 55 years older so this is the only piece of information in the whole document and chat GPD comes back thank you for providing the updated information according to the new information you provided the plan does offer financial support for hearing aids but only for employees who are 55 years or older so you see now it is included isn't that great you know what I do now I still copy this one I say hey let's start a new chat delete cache everything restart GPT 3.5 I say does the insurance plan of the company flying a hearing it given that a priority with now I can say again new information about and now you're not gonna believe this now I only include the relevant sentence so not the old document only one sentence less than I don't know 50 tokens and now that Tubidy comes back based on the new information provided the insurance plan of the company flying ice does offer financial support for hearing aids for employees who are 55 years older so you see I did not have to put in the whole document just the relevant part that provide an answer to my query prompt why why do I do this one piece of information that is important to understand the sole purpose of a normal common Vector database system will have access to store now I know there are open source and they are very elaborate very expensive databases I'm talking here about a standard normal not with some extra extension the normal way back to a database system the normal Vector store is sole purpose is to provide new and relevant information to chat GPT or gpt4 given a specific user query in the prompt so we have a query we asked chatgpt something that is not integrated and it's pre-trained and fine-tuned training data set and the question is how does it happen do I have a vector representation like the viewer asked in his comment to my video so again what we learned we know it's probability based and whatever answer we get back it is not reproducible answer it is not a law of nature where you can take an apple and let it fall down to the Earth and you know there is gravitation sometimes with cheap adhes systems you will find out there is no gravitation and sometimes with gbt systems you will go and then answer back there's an anti-gravitation working it is not reproducible it is a probability based system it is not anything at all that it is reproducible that it has a ground truth that it is an experiment you can repeat like in physics 100 000 times on the planet Earth and you will get the same result this is not gonna happen so some you are saying hey what is the statement of chat GPT about this so here's the statement of chat GPT it is important to note that my responses are generated based on probabilities and statistical patterns and not on real-time information or having personal experiences of course not this is a probability machine and this is also the problem if you use jet GPT now the medical sector or in the medical application it works with probabilities but more about this in a later video so beautiful so this is now the question of my viewer this is what we set out to answer let's make it a little bit clearer so here is now the second part the main part of this question and I would like to indicate what is maybe the misunderstanding because we can learn a lot of this so he says here she said we would need to upload all of the papers into gpd4 now we know no in the augmented prompt to type GPT after we have done everything with the vector database everything with the vector store everything with another solution in the augmented prompt to chat GPT is only the relevant data to answer the query of the user that asks GPT for an answer and this relevant data can be just one sentence maybe less than 100 tokens maybe a paragraph with 500 tokens but never exceeding 32k tokens so next and hope that it's representation so the representation of all of my documents now we learned there is no representation going into the augmented prompt to chat GPT there's no Vector representation the communication between the subsystem is in plain English it is called a language model so whatever the systems feeds back to the augmented prompt is in English sentences and then last part these representations that are not existent match your own systems tokenized representation now any output of my system if I use now a sentence Transformer and I focus here on a sequence of words that is in sentences is in English sentences so any system that you have does not output vectorized data it outputs English sentences because this output will be the input to a language model and a language model understands English language sentences and English words that is the whole point so you see this is now the clarification regarding the question of this URL foreign example here is a beautiful video by Microsoft mechanics the official Microsoft video and it is about can chat GPT work with your Enterprise data a new Enterprise data here are assumed is not uploaded to gpt4 but you want as a company for all your 300 employees that they can use jet GPT to get some nice chatty answers back if they put in a query a question to chat TPT regarding the Enterprise policy the Enterprise healthcare plan whatever so how do you do this if you have your let's say private Enterprise data and hundreds of document but you want to give your employees jet GPT and chat CPT should answer in the correct way and form about your policy that you have in your Enterprise let's have a look at this he answered everyone click on this light bulb I can see the process it went through to provide a response here we can see that GPT first takes the child history and the last question to produce a good search query you can also see the rest of the prompt okay let me stop here you see this is now the inside of chat triputi this is the prompt the augmented prompt that comes back after the cognitive search by Azure has been done so the prom says Hey chat Tubidy you are an assistant to help the company employees with their healthcare plan question be brief answer only with the facts listed in the list of sources below and now you have your sources your sentences that are the solution that answer your query and you see here there's a document and it is oops not one sentence but a paragraph beautiful and in this paragraph and in only this paragraph you'll find a solution to the query you had in your prompt so this is really now from Microsoft that hosts gpt4 and they host here this vacto database this cognitive search this postgres ql they have it and you see here in their documentation what the augmented prompt after the cognitive searches you'll say hey answer this but only use the sources below and now you have here one paragraph of one or two documents this is it not 100 documents already nonsense and it is in plain English there's no representation no Vector embedding nothing it is a language model beautiful okay so beautiful let's summarize where did the conceptual misunderstanding happen I think it is focused on the semantic search agenda so to perform a semantic search and not just the TF IDF index search like I showed you in my last video on my Enterprise documents assistant uses the mathematical function of a cosine similarity on Vector objects in a constructed specific Vector space where my Vector embeddings have been calculated to respect similar semantic content of my sentences now it is the function of the vector database to provide now corresponding Vector representation for any text that you input and normally the plugin performs the cosine similarity function like this happens in open AIS retrieval plugin to extract now the relevant information from all the Enterprise documents and come back with the necessary information to respect to answer the user query to the activity let me stress this again in all of jet GPT prompt design on prompt engineering we use English sentences it's a language model that we are interacting this is the pure Soul beauty of jet GPT we do not have to code anything so don't let you be surprised with maybe complexity of vector representation and embeddings you can buy and different as embeddings you can buy just focus on the simple understanding and I hope this answered your question and yes we are done this is the last slide I hope it was informative I hope this is an answer that provides some insight to my viewer so I say thank you and I hope to see you in my next video