Tutorial | Chat with any Website using Python and Langchain (LATEST VERSION)

Key Takeaways

good morning everyone how's it going today welcome back to the channel and welcome to this new video tutorial in which I'm going to show you exactly how to build this application that you see right here now this is a chatbot built completely using Python and using the latest version of Lang chain and it allows you to chat with pretty much any website so let's say for example we're going to talk about langra which is the latest product released by Lang chain I'm going to paste the URL right here here and now I'm going to be able to ask my language model about the information inside of this website for example what is Lang graph now this is interesting because langra since it's such a new product um no language model is going to know anything about this however here we see that we have um our chat B is answering questions about it so we're using the information inside of this website as a context okay uh um I understand that this may look a little daunting at the beginning but believe me it is simpler than it looks and I promise that by the end of the video you will have a product working that you will be able to Showcase to your clients to your colleagues to your co-workers to your employers Etc and maybe even deploy it for your team to use okay uh if you like videos about Ai and software don't forget to subscribe and yeah let's get right into the video [Music] okay so the first thing that we're going to want to do is we're going to want to create a virtual environment for our application okay here as you can see I am already in visual in vs code and I already created a repository a git repository for this web for for this project in this case I now I'm going to create a virtual environment remember that a virtual environment is important for you to be able to install your python dependencies only for this project and so that they don't overlap with other projects that you might be working on the side okay uh in order to do this I am going to use cond if you want to use cond don't forget to install anaconda or miniconda so I'm going to be doing cond create D- name to name my environment I'm going to call it chat with website and I'm going to also specify the python version that I want I want to work with python 3.10 I'm going to hit enter okay now if I hit enter there you go it's going to start creating my virtual environment feel free to use whichever virtual environment technique you want you can also use VN for example but lately I've been using cond a little bit more than vmf and here you have it that if you want to activate it all you have to do is say cond activate and the name of your environment and there you go now everything that I install inside of here be it using cond or pip it's going to be installed only in my virtual environment okay so now we're ready to start the development all right so the first thing that we're going to do right here is we're going to create a Source directory and inside of here I'm going to create my app.py file which is going to contain my application and also right here I want to start installing the packages that I'm going to be using for my application to work okay um remember that I told you that we're going to be building the front end using python um in order to do this we're going to be using a library called streamlit which allows you to create graphical users user interfaces in just a few lines I'm going to show you how it works in just a moment so you're going to do pip install streamlit and also for our application to work we're going to be using Lang chain and Lang chain open AI which is the third party package for open AI okay you hit enter for you it's probably going to take a little bit longer than that because I already had them installed before recording but there you go now we're ready to start coding wait I just realized that just a moment ago my fa my head was on the way so you couldn't see what I was typing uh so I'm just going to paste this like this right here and it's going to be available anyways in the GitHub repository that is going to be linked in the description okay so we installed stream lead Lang chain and Lang chain open AI now what we're going to do do is we're going to import streamlit as St and we are going to initialize the applicate the front end of the application doing St set page config and this method takes a few par few arguments the first one is the page title and this is the what is going to show in the tab okay of your of your of your browser in this case I'm going to call it chat with websites and the second one is going to be the page HIC and I'm just going to leave it as a robote head like that just going to close this right here now let's just add a title for example to do that you just say h St title like this if I'm not mistaken St title yeah St title not page title just title like that and we're going to say chat with websites and if you want to add a a sidebar like we did like we had in the in the app that I showed you just a moment ago all you have to do is s sidebar and actually you have to say with st sidebar and anything that is inside of here is going to be put inside of your sidebar I'm going to say the I'm going to add a header there there's going to be heading settings and right here let's add the text input that is going to take your website URL okay so let's add that st. text input and here we're going to say website URL there you go and we're going to store whichever value the user adds here in a variable called website URL like that let's see how that's looking so far in order to run your streamlet application remember that you cannot do Python and then run your application from like this python SRC app.py this is going to return an error because stream lid is supposed to be um oh we got out of we got out of my environment so I'm going to do cond activate chat with website there you go remember that if I do python SRC app.py that's going to give me an error because streamlit applications have to be run using streamlit run so I'm going to do stream L run and then the path to my application and and here I have it let me show you it appeared here on my other screen and here I have my sidebar with the settings the website URL and my title right here let me show you now how to add the chat components for the front end so in previous videos what we did is we had we were creating our own element to display messages right and that is a that is a good method and it allows you to to bend the possibili of what streamlit can to right out out of the box but since then actually streamlit has introduced a new component to their component list and now you can actually create chatbots that like chat GPT natively inside of stream lit so let me show you how to do that in order to create a an input bar for your dialogues you you can you do St dot chat input okay and here you just write like the prompt type your message here for example if I save this let me just open the application right here I'm going to say always rerun and here you see that we have our text input right here for our chat putut right now what we want to do as well is we want to add the messages and the nice thing right here is that you can add the messages after the chat input right here and they are still going to show on top so let's do that right now in order to add a message you have to say with st. chat message and between parenthesis you're going to have to say who is sending this message uh in this case I'm going to say that this one is the the human and you have to add semicolons right here and go um st. fright and then you just say whichever your message is so hello um let's say actually the first one is the AI and then we're going to add another message with st message chat message and this one is going to be human and we're also going to add St ride and let's say I want to know about um about Lang Ching for example there you go now if I save this you can see that here are the messages that are being sent here I mean my sidebar is still there and you can see that here is my my input right here and all of the messages that I add right here with using with st chat message are going to show like this so here I can for example add a third one one I'm just going to copy this one right here for AI nah so there you go here hello how can I help you I want to know no all right so that's not very helpful but now you now you you can see how to how to start creating these chat BS okay so for now we have only Built the front end what we're going to be doing right now is we're going to actually start giving it the capabilities and the possibilities to chat with your data from the website all right so let's do that now okay so the first thing we're going to want to do is we're going to add some functionality right here let's add some interactivity okay so what we're going to do is so that when the user types something right here we're going to reply something automatically okay so first things the first thing to do is to store the value of whatever the user writes here in a variable so let's do that let's say user query is going to equal this and then if user query is not known and also if it's not equal to an empty string what we're going to do is we're going to write our messages right here so if it's not none we're going to write a human message with the user query like that and we're then going to add a response from the AI and the response is going to be an automatic I don't know like this there you go and we can just erase this one right here now if we save this let's see what happens hello and it replies I don't know how are you I don't know okay so so far there is some interactivity right here but we don't have a history yet so let's add a history but before actually creating our history object what we're going to have to do is we're going to emulate our response from our language model OKAY in order to do that we're going to create a function called get resp response that is going to emulate that we it's going to simulate that we already have the logic behind our chatut okay so let's just create that function right here it's going to be get response it's going to take the user input and it's going to return just a very generic I don't know now let me just add some comments right here this one is going to be the app config right here is going to be the sidebar this right here is going to be the user input there you go now right here inside of the logic that is going to handle any change in the user input we're going to say that the response is going to be equal to just get response and we're going to pass in the user query from our user which is the text that the user is going to add right here okay and now instead of hardcoding the answer right here we can very well just uh return the response like this so now we are emulating our response from our chatbot which every time is going to uh respond I don't know but in a real I mean in an application that is already working instead of this part right here we're going to add the logic of our chat putut inside here so so let's save this and let's see how this works hello it seems to be working still how are you there you go so it seems to be working now what we're going to do is instead of logging one message at a time because as you can see right here we still don't have the the history instead of logging one message at a time we're going to log I mean we're going to show the entire message history and now that we have this function right here that's going to make it way easier so how this works is we're going to have to create a variable that will contain all of our message history okay I am going to initialize it right here on their app config I'm going to say chat history and I could initialize it as an MTR array but I'm going to initialize it with a test a sample message a welcome message from the AI okay something like hello I am a chatbot how can I help you okay so I'm going to initialize it as um an array and here I will add this message however when working with Lang chain what you want to do is you want to use the schemas that Lang chains comes with okay so the schemas are part of Lang chains core package so we're going to come from lank chain core remember this is a new syntax for the new version of Lan chain so from Lan chain core we're going to tap into the messages module and we're going to import the AI message schema and also the human message okay and here for the welcome message is going to be the AI that is going to be um adding this message uh sending this message so I'm going to say that this is a AI message and this one takes a content param and here you're just going to write whatever the content of the message is in this case it's going to be hello I am a bot how can I help you okay and this is it's important to use these schemas because these are the building blocks of any Lang chain application okay if you're using any other framework of course this is not what you're going to be using but since we're using Lang chain these are the schemas that you're going to be working with when you send when you work with with the messages for for the language model OKAY especially if it's a chat model um so now that we have this chat history variable initialized what we're going to do is we're going to append whatever the user sends as a query so whatever the user writes right here we're going to append it to the chat history so let's do that let's say the chat history append and we're going to append whatever the user wrote which is the chat query but we're going to append it as a human message like this and remember that it has to be inside of the content and there you go it's user qu it's the user query right there and now we're going to append as well the the response which we got right here and in order to do that we're also going to do his chat history append AI message content response there you go now I can delete this right here and just for debugging I'm going to show you what the chat history variable looks like in real time so I am going to log it in the sidebar so I'm just going to say that with my St sidebar I want to do St WR my chat history like that okay so there you go this is my application that is running and you can see that my chat history for now is only an AI message that contains hello I am a boat how can I help you now if I send a message saying hello you can see that I get a respon I mean that my message is appended right here as a second element as a third element we have the AI message with the response saying I don't know which is the one that we got from this me this function right here okay however there a problem with this and it is that if we for some reason want to ask another question let's say how are you our chat history is reinitialized okay which means that here my second element is no longer hello but is the new message that I sent and this is the new response um and this is a thing with Lang with uh with streamlit and let's just deal with that right now so the problem with this is that is in the way streamlit works okay so every time something happens in a streamlit application what it actually is doing is it's reloading your entire code okay so anytime anytime an event happens an event can be something like you type something here and hit enter or you type something here and hit enter anytime an event happens what streamlet is doing is it's reading your entire code again okay and that wouldn't be a problem if we were not initializing our variable like this right here because as you can see if whenever we type something right here it reads the entire code and while it's reading it it is reinitializing the chat history variable again with just that welcome message so what we want to do is we take we want to take this variable right here outside of the of the of this streamlet cycle and we want to make it persistent in the state of the application okay and in order to do that it's uh streamlet actually makes it very simple we're going to be working with an object called session State and the session state is an object that does not change every time you read the you reread the application okay so what we're going to do is we're going to say that if our chat history um property is not inside of the object session State then we are going to initialize it we're going to set our chat history to equal and we're going to initialize it just like we did before to this object right here like that okay and now when it's reading the application again it's it's going to arrive here and it's going to say so chat history has not been defined yet so we're going to initialize it with a single message and then the second time it comes here it's going to arrive here and it's going to say oh but this time chat history is already initialized of my session state so it's not going to re to redefine it and right now what we're going to have to do is instead of using chat history itself is we're going to have to tap into the session stage at history object instead now if we save this and we come right here you can say that if I say hello hello is appended here as a second message and then I have we my I don't know but now if I say how are you I hit enter you can see that it's actually appended to the rest of the conversation because now my chat history variable is not being reinitialized every time I something happens in my application okay so there you go and now we can actually show our conversation as messages and not just as a debug thing right here okay so for I mean we want this messages to be right here in order to do that instead of logging in logging them in the sidebar we're going to add here the conversation and in the conversation we going to say that for well here it is for our message inside of the session State chat history because we're tapping inside the chat history that is stored in the session state that we just defined we're going to say that if the message is an instance of an AI message we are going to add a chat message saying um remember that in order to add chat message you do with st chat message um and if I'm not mistaken we have to add here that it is a AI here we're going to write the message content okay and then else if it's a human message we're going to do exactly the same thing with st chat message and this one is a human and here we're going to say that St WR we're going to write the message content okay so let's see if this works there you go so just to make it clear and to be sure that we are all understanding what's going on here I am looping through all of the elements inside of my chat history which is stored in a session State variable and if it is an instance of my AI message object which are the schemas that we downloaded that we imported from Lang chain we're going to show it as a chat message and the chat message is going to come um it's going to be formatted as an AI with a little robot right here and if is it's an instance of a human message that means that we're going to Showcase it we're going to display it as a human message like this and we're just going to write s write message content like that and there you go okay so now let me just reload this hello if you see that if I reload this application the only message that I have is the first initialized message and then I can say hello how are you it's going to say I don't know um okay and then it's going to reply I don't know again so now our front end is pretty much finished now we can actually start coding the rag tools however something important to keep in mind is that at least for the purposes of our small application we remember that I showed you in the introduction that let's show it right here remember that I showed you in the introduction that if the user didn't write anything right here we were going to disable this part okay and that is very important at least for the purposes of this small application because otherwise we would have to handle what happens when the user sends a query and there is no website URL sent right here I mean that's also very possible to handle uh but for the purpose of this tutorial we're going to keep it as simple as possible so we're just going to disable all of this part if there is no website URL okay so let's do that right now um in order to do that I'm going to come right here to the side part part and underneath the website URL I'm going to say that if website URL is none or if it's an empty string I'm going to show an info box saying please enter a website URL okay and then else which means that if there is a website URL and it is different to an empty string I am going to actually show the application that we started to code so now if I save this and I come back here I reload this you can see that I need to enter about it enter a website URL I can I mean for now I can just enter whatever I want and now I have access to the chat but interface okay there you go what we're going to see now is what is actually happening behind the scenes in a rag applic okay um what is rag rag means retrieval augmented generation uh in other words it means that we're going to augment the knowledge of our language model with context that we're going to retrieve from a custom knowledge base okay um I have a diagram that I had made for a previous video if want to check that out this one comes from the video of chat with your PDFs but the idea is pretty much the same thing for chatting with your website okay what you're going to want to do well I mean what we're going to do in our application is we're going to take our HTML documents which are going to be fetched by adding the website URL right here and from them we're going to use Lang Ching to extract the text okay however the text might be huge I mean in our case it was a it was a short blog post but it can also be like a huge Wikipedia post or something like that and the problem is that you cannot feed a huge Wikipedia post to most language models um most of them have a context window that only allows you to send like uh limited number of tokens and if you send like the entire context the like all of your knowledge base you're probably going to go beyond that context window so what you want to do is you're going to take all of the text from your website and you're going to split it into different chunks of text so in our case for example um the blog post for Lang Lang graph um the first chunk is going to be probably the introduction the second chunk is going to be the first paragraph and Etc until the conclusion okay and this chunks of text then we're going to pass pass them through a an embeddings model OKAY in our case we're going to use an embedding models an embeddings model that also comes from open AI but there are other embeddings models that you can use um for example you can use them the ones that are available in hog in Phase okay now what does the embeddings model do what the embedding models uh do is they take your text in our case we're taking each chunk of text and they're going to vectorize it or in other words they're going to create a numerical representation of your chunk of text now why do you want your text to be in a numerical representation the answer for that is that with all of those numerical representations we're going to be creating a vector store and the use of this Vector store is that we're going to be able to find the chunks of text that are relevant to what the user asked in their query so for example if they're asking about Lang graph or what is a neural network or whatever we're going to embed their question using the same model that we used to to embed our chunks of text and we're going to perform a semantic search and the computer can only find uh can only perform a semantic search with vectorized data it cannot do that with simple words because computers do not understand words they understand numbers so in our case we're going to um have our vectorized chunks of text we're going to vectorize our user question and that is going to allow us to find the chunks of text that are relevant or similar to whatever the user asked so if the user asks a question about um for example what iside neural network the vector store is going to find which chunks of text are relevant to answer to that question it's going to return some chunks of text and from some and we're going to take those chunks of text to send them to our llm as context so the final message to our language model is going to look something like based on the following context and then we're going to paste whatever we found here answer the following question and we're going to send in the question from the user and in our case we're also going to pass in the history so our full prompt to our language model is going to look something like based on the following context and then we're going to add the retrieved chunk of text that we found from our HTML page answer complete the following conversation and then we're going to send the entire chat history of our conversation and then the user query so that it knows knows the history of the conversation and it also knows the context to answer our our question with okay so that's basically how rag uh chat BS work and uh I'm going to add this diagram as well in the GitHub repository that you can use it I'm just going to have to change this to HTML but yeah this is pretty much what is going to happen and the vector store that we are going to use for this tutorial I am going to use if I'm not mistaken but all there are many Alternatives there is pine cone there is quadrant uh there is f um and all of them have their strengths and weaknesses but yeah I hope that it's clear what is actually going on behind the scenes in this in this project so the first thing that we're going to do is we're going to create our Vector store okay and in order to do that we're going to take the contents of our HTML chunk them into split them into different chunks and then create the vectorized representation of them to store in to store them in our Vector store okay um now first before we do anything else we have to install a new package okay this package is called Beautiful soup because the tool that we're going to be using to fetch all of the text from our website uses a uses a library called Beautiful soup in case you haven't heard of beautiful soup is a library for scraping websites which basically just allows you to get information from any website um and in this case it's like very simple we're just going to be fetching all of the information in between P tags like paragraph tags right uh so in order to install it you can do pip install beautiful subp 4 or in my my case since I'm using cond I'm going to do cond install um because I'm using a cond environment but I mean honestly you can also do peep install uh beautiful sub 4 and that would be good um so now that it's installed what you can do right here is I'm going to say that from L chain Community because remember that uh in the new version of Lang chain all of the third party applic are stored inside of langin community and we're going to say that we're going to tap into the document loaders uh module we're going to import webbased loader okay and this is the one that requires um that requires beautiful soup I'm just going to add it right here beautiful soup 4 so you know there it is and now what we're going to do is we're going to add a new function down here that is going to be called um get Vector store from URL and then it's going to take our URL we're going to first initialize our loader using the webbased loader and this one takes in the URL and it's not like this and then from that webbased loader we're going to take the documents and the documents uh we're take them from loader. load and we're just going to return the documents for now I mean um the objective of this function is to return the entire Vector store but just for debugging purposes I'm going to start returning just the documents so get the text in document form okay so let's see how that works if I save this this and now down here if there is a website URL I'm going to say that the documents are going to equal get Vector store and then I'm just going to pass in the website URL that I got from the sidebar and I am just going to debug this adding a adding it to the sidebar with sidebar we're going to St write the documents okay so let's see how that works now I'm going to take my langra blog post and I'm going to come back here just rerun this going to paste this and just let's see how it goes now here it is now as you can see we have an array with a document inside of it and here we have the page content and all of this is the text from our blog post okay now as you can see we only have one one element right here because what is remember that what we're doing right now is we are right here we are just extracting all of the text from our HTML page okay now we have to chunk it into different text so let's let's just do that right now so in order to split all of our text into different chunks of text um we're going to be using a tool that comes from Lang chain uh that is called the recursive character text splitter okay um however langin actually allows you to do this in several ways if you want you can take a look at the documentation they have um a huge uh variety of text Splitters that you can use in our case we're going to use a very simple one which comes inside of Lang chain the Lang chain package P itself and from here we're going to tap into the text spitter modules and we're going to import uh recursive character text splitter like this okay and from here we're going to use this one to split the document actually let's put this in singular because uh remember that this one returned only one document so what we're going to do is first we're going to initialize our text splitter like this and we're going to initialize it using the recursive character text glitter that we just imported and we're going to say that the documents this time in plural or document chunks to make it easier we're going to say that we're going to use our text splitter to use split documents so that we know that the resulting elements are going to be documents themselves as well and we're going to pass in the document that we fetched using webbased loader let me just add a comment right here split the document into chunks like that okay now let's see how this looks like going to instead of now instead of returning the full document I'm going to see how it looks like if I return just the chunks and here we're going to say that we're returning the document chunks I mean this doesn't change anything just to make it clear for you um so now we can come back here I'm just going to reload this just going to paste this page again and if I hit enter you can see that we now have several documents right here and each of them contains different information from the website okay and they are shorter and what we were are right here here so far um also something important about this is that each document not only contains the content of the document itself but it also contains metadata all right and this includes the source of the document and the title and the language as well so this is important if you have like several several um data sources in this case we're only we're only talking to one to one um data source which is this blog post but if we had another if we had like if we were talking to several websites or something like that at the same time this would be useful to know from where our our chatbot is retrieving the information okay so there you go that's how we split the text Doc the text into different documents so we're going to create create our Vector store from these chunks that we have gotten from our HTML page okay um to do that we're going to need an embeddings model in our case we're going to be using open AIS and we are also going to need a vector store for the sake of this tutorial we're going to be using chroma which is a very simple very minimalist and very easy to use um Vector store that you can use in your local machine but feel free to use whichever you want I mean there is quadrant there is even pine cone if you want a hosted one um f so yeah just for the sake of this tutorial we're going to be using chroma and we're going to complete this right here so we're going to take those document chunks and we're going to import chroma remember that chroma is a third party integration so as of the new version of Lang chain this one is going to be included in the L chain Community uh package because all of the third party Integrations come in the um Lin Community package from now on and we're going to tap into the vector stores module and from here we're going to import chroma just like that and right here we're going to create a vector store from the chunks um so let's create it let's call it Vector store and we are going to initialize it from chroma and in line chain uh whenever you import a vector store from the third party Integrations you can also do you can also initialize your vector store from documents like this and this method takes two parameters the first one is the documents that you want in this case are the document chunks which themselves each each one of them is itself a document and the second parameter is the embeddings model that we're going to use remember that I told you that we were going to use open AI embeddings for this one and as of this new version of Lang chain we are going to be importing anything related to openai from its own from its own package so from Lang chain open AI we're going to import open Ai embeddings and we're going to initialize it right here like this open a embeddings don't forget to initialize it with the parenthesis otherwise it's not going to work and then we're just going to return this Vector store right here okay so now our get Vector store from URL is finished and we can just remove this debug right here now now next step is to actually use that Vector store so the next thing to do is we're going to have to remember that I told you that we were going to need an open AI embeddings model in order to vectorize our chunks of text okay uh in order to use the open AI model we're going to need the API key from openai so we're going to go to platform. openai come we're going to come to open AI keys and we're going to create a new key here I'm going to call it YouTube tutorial this is you can name it whatever you want and I am going to copy it and this is going to allow you to use the models remember that these models are paid for but they're pretty cheap so I mean you shouldn't you shouldn't really be paying more than a few sense for for an article like this for example um and now what we're going to do is we're going to come back to our project and we're going to create a file called EnV and this file is going to contain all of our secrets which means that it's going to contain our open AI API key now whatever you store here is going to be available to your application as a an environment variable and this thing right here we're going to add it as well to our sorry to our git ignore file and here we're just going to add EnV this is so that ourv file is not tracked on git and we do not accidentally push it to GitHub for example where other people can see it and they would be able to see our API key and they would be able to use it at your expense which probably you don't want that and which is also the reason why I'm going to be deleting this API key from my uh dashboard before uploading this video now that we have created all of these things in order to actually make this oh in order to actually make this environment V variable available to my application I'm going to need to install another package um something important remember that at least if you're using Lang chain you're going to have to name your open AI API key variable environment variable exactly like this because this is the name that Lang the environment variable name that Lang chain looks for by default otherwise you would have to specify it manually and here we're going to install python. EnV okay this is just for reference let me just install it I'm going to come right here I'm going to say pip install python. EnV make sure I am running inside of my virtual environment there it is I mean to you it might take a little a little bit longer because I already had it installed and now what you are going to do is you're going to say from EnV we're going to import load. EnV and now at the beginning of your application ideally you're going to run this function right here this function is going to make all of the variables right here a available for your application okay if you don't do this if you don't do this um this information is not going to be available for your application so now that we have our API keys we're now able to actually create our vect store with the open AI embeddings so so far it's fair to say that we have finished this part right here which means that we have extracted the text from our HTML files we have divided it have split it into different chunks of text um which are in the form of a document we have used open AI embeddings in order to vectorize them and create a vector store um not with pine con but with uh chroma okay now the next step to do is we are going to create a retriever chain the retriever chain is basically a chain that is going to take the user query and actually it's going to be a little bit different to what we see in this diagram because in this diagram we are only embedding um the latest user query and finding chunks that are similar to that user question but in this example that we're going to be coding in a moment we are going to be getting the the text the chunks of text that are relevant not only to the latest question from the user but also to the entire conversation so we're going to be embedding the entire conversation history and we're going to be using it to get the chunks of text that are relevant to the entire conversation so in order to do that we're going to come right here and I'm going to organize this in sort of a pipeline way so we're going to uh create a function that gets the retriever chain um I'm going to call it get context retriever chain because it's going to it's a contextual retrial chain and this one is going to take in the vector store I mean feel free to organize your code As You Wish um I think this is just makes it easier for educational purposes it doesn't mean that is the best way to do this in production of course um but here you can see that we're going to be using the vector store that we got here in order to to get our retrieved chunks of text or our retrieved documents Okay so so the first thing that we're going to do is we're going to initialize a language model okay um language models in Lang chain come either in the community in the third party Community package or since we're going to be using open AI models as of the new version they come inside their own package inside Lang chain open AI so we're going to import the chat model which is chat open Ai and we're just going to initialize it like this right here okay uh the second thing we're going to do is we're going to uh create a retriever now in Lang chain pretty much every Vector store that you create has a method that Returns the same Vector store as a retriever that is um that allows you to use it inside of U that allows you to use it to retrieve uh relevant text related to a to a query okay and then finally we're going to initialize a prompt now I don't think we have imported the chat prompt template so we're going to do that right now um the chat prompt template you're going to import it from L chain core um yeah I'm going to do this right here from lank chain core because remember as of the new version the core components of L chain come inside of the package L chain core and from here we're going to we're going to tap into the prompts module and we're going to import chat prompt template and also inside from here we're going to import messages placeholder so the prompt that we're going to initialize we're going to initialize it as a chat prompt template which basically is just a prompt but in the form of a list of messages and this one some this one takes a an array of messages sorry I forgot to mention that here we need to initialize it from messages in order to actually in order for it to actually take an array of messages and right here the first element that we're going to pass in is a messages placeholder with a variable name of chat history like that okay now what is this messages placeholder right this uh doing so as I told you before this right here is a prompt that is going to be um populated with the variables that we're going to pass in and the messages placeholder just replaces all of this with the value of the chat history variable if that variable exist otherwise it's going to leave it empty okay this is important because if at the beginning of the conversation we do not have a chat history this is going to remain empty and as the conversation continues this variable is going to change so uh that's how you use messages placeholder um the second element right here is going to be um also something interesting right here is that right here we're going to pass in a human message we could I think that technically we could use human message uh schema however Lang chain also allows you to send in messages as a topple the first element of the topple is going to be the kind of um the type of message that this is in this case it is the user and remember that in prompts at least in lunching they work pretty much the same as F strings in Python which means that you're going to add here your variables and then when you populate the prompt this variable right here is going to be populated with whatever uh value you set to this variable right here I'm going to show you exactly how how this works in a moment but so that you see it in action but just um keep in mind that this fireable right here and this varable right here are going to be populated with whatever I pass in in the chain and then we're going to add a last um uh message to this prompt and it's also going to come from the user and it's going to instruct the language model I have it right here let me copy it cuz it's a bit long it's going to it's going to prompt the language model to given the above conversation because remember that we have the chat history and the latest input from our user given the above conversation generate a search query to look up in order to get information relevant to the conversation which basically is going to create um um a search query that is is going to find the the chunks of text that are relevant to the entire conversation and not only to the latest prompt okay um and then finally we are going to use a method that also comes from Lang chain itself called create history aware Retriever and this one comes from Lang chain from Lang chain from the Lang chain package itself so from here uh you're going to tap into chains and we're going to get the create history aware retriever okay and uh this was this one's actually going to be the the complete retriever the retriever uh chain so I'm just going to initialize it like this retriever chain and I'm going to say create history aware Retriever and this one right here takes as first as first argument it takes a language model which is the one that we just initialized right here the second element is a retriever which is the retriever that we um got from the vector store that we had previously created so let me just add it right here and the last part is the prompt which is the one that we initialized right here and then this is the one that we're going to be returning uh from this function right here uh in order to show you how this works let me in order to test this function that we just created um I'm going to call it from inside of from inside of here so after our user sents a query I am going to call this chain the chain that has been returned from this function um that we have right here to see which documents it has retrieved right because remember that what we're doing here is we are retrieving the documents that are relevant to the entire conversation and that's what the chain is doing so let's just come down here to after the user query and let's call our retriever chain that we just created um in order to do that um actually something important that I forgot to mention before is that since we're using chroma we're going to have to install the package for Chroma so I want you to pip install chroma DB um now I have already installed it off camera but uh it's probably going to take a little bit longer for you and I'm just going to add it here chroma DP and um this is the now once you have installed it you're going to be able to run this code right here um so we're going to take the user query and we're going to pass it to the retriev chain so let's test the retrieved documents and we're going to use of course the retriever chain that we just created and as of the latest version of line chain in order to call a chain you need to use the method invoke now invoke takes uh an object with key value Pairs and in the key you're going to have to write the name of the variable that you're going to replace in the prompts that are inside of your chain right remember that here we have two variables chat history and input so these are the ones that we're going to be replacing uh so we're going to pass chat history and we're going to pass in the chat history that we saved in the session State and the second uh variable that we're going to pass in is the input which we're going to send as the user query uh which is the one that we send right here okay and these retrieve documents how about we just log them s WR retrieve documents okay let's see how that works um so let's come back here I'm going to take my web my blog post again I'm going to run it and here I'm going to say what is Lang graph and here if I'm not mistaken we're going to get the documents that are relevant to this conver to the entire conversation because remember that we're passing in the entire chat history okay and there you go now you have the documents that are relevant it seems to be working correctly I'm going to remove this test right here because um that's not how what the application is supposed to do but now we have tested that the retriever chain is working correctly so we have successfully tested that this part of the application works correctly now what we want to do is we're going to create a last function and what this function is going to do is it's going to return us the final chain that we're going to need to actually respond to our prompts okay um this final chain let me show you in the diagram so we finished this chain that would give us the ranked results I mean the documents that are relevant to not only the user question but to the conversation now what we want is a chain that will take this documents and we'll include them into another prompt saying something like based on the following uh context and then it will pass the documents answer the following question and then we will pass the question of the user alongside the chat history and then we will get an answer so let's create this function first um this function is going to be called it's a a stuff documents uh chain let me just let me just um import it first uh in order to use this you're going to have to import this directly from Lang chain and it's going to come as well in the chains sorry in in the chains module so from L chain the chains and in this case we're going to we're going to go into the subm module combined documents and we're going to import the function create stuff documents chain now this function right here what we're going to do um let's call it um get let's call [Music] it get conversational rack chain and this one since we are doing this like a pipeline so this one takes the respon The Returned element from this one this is going to take the returned element from this one which is say retriever chain right and here we're going to be using this function right here okay stuff documents chain like that and this function as you can see right here it takes a language model and a prompt it can take an output passer as well we'll just add a language model and a prompt so let's declare them up here this one is going to be our chat open AI that we had previously imported and in order to create a prompt we are going to to create it from the chat prom template and just as be we did before we're going to create it from the messages right now we're going to start the prompt with a system message telling the AI what it is supposed to do so I have this text right here it's a very very simple prompt it goes like this answer the user's question based on the following on the below context and then you just add the context and this is a function that we're going to have to to this is a variable sorry that we're going to have to replace during the chain okay when we call it uh I'll show you how that works in a moment now the second part is actually the messages placeholder just like we did before um this one is going to take the variable of chat history and it will be populated if we have a chat history hisory in this chain if it doesn't have a chat history it's going to leave this blank and then lastly we're going to append to our chat history our user input like that okay and then let's just close our prompt so as we were saying the stuff documents chain the first element that it takes is a language model and the second one is the prompt that we have just initialized right here there you go however this chain as you can see it all it's doing is it's just taking the the context that we're given it and it's passing it in it's passing it inside of here and right so when this in other words when we call this function what is actually going to be happening behind the scenes is there is going to be um stuff documents chain invoke and then as as a parameter we're going to pass in the context which are going to be the documents that are going to be retrieved from our retriever function and then of course the chat history and the input um which is the users query however right here we don't have the retrieve documents yet what we're going to do is link these two these two chains together so we have the retriever chain which gives us the documents that are relevant to our conversation and the the stuff documents chain which uses those documents to return an answer and given the chat history of course so we're going to basically just plug them in together using this other function called create retrieval chain okay create retrieval chain actually I haven't imported it let me just import it in a moment it also comes from chains I'm going to call and it it's called create retrieval chain like that so it goes from it comes from Lang chain. chains so you're going to create you're going to create re one second create retrieval chain like this and the first element that it takes is a retriever it can be either a vector store as retriever as we did for the retriever chain or it can be a runnable and a runnable is in other words in this case it can be a chain and we're going to pass in the runnable of the the runnable that was returned from our function up here which is the one that we're going to be passing as an argument right here so let's pass that one first um the retri a chain and then secondly it takes a combined documents chain which is basically the one that we had just created up here so let's copy it and paste it right here and actually this is the this is the last uh chain that we're going to be returning so this function right here all that it's supposed to do is it just I mean just to recap this function right here it is creating a documents chain that will take context and answer whatever question we have based on the documents that we pass as context and it's plugging that chain together with the retriever chain that we did before to create a final to create our final chain that will take the will'll take our user query our chat history and and it's going to return to us the entire uh it's going to return to us a an answer based on the entire conversation and on the context that the chain has found okay so let's test this chain right now and see how it does in order to test this function right here we're going to call it inside of our application okay and a good place to call it is right here once the URL of the website URL has been entered by the user right in order to do this actually what I want to do first is refactor this a little bit and actually I just want to initialize the state of the application I mean the the initial the chat history in the session state if there has been a website URL added otherwise it we don't really need it so let me just add here session State and then right here create conversation chain and actually something that I wanted to mention is that we probably need to make the vector store persistent as well because remember that in order to create the vector store we used the embeddings from open aai and that is even if it's not an expensive model it it is resource I mean it is resources that we would be spending every single time that the application runs we don't want to be re-embedding the same content from the website every single time that we make a query so let's just add the vector store to the session state right here if the vector store is not in session State what we're going to do is we're going to set the session State Vector store to get our Vector store from URL and now instead of instead of calling it from here we're going to call it from the session State like that okay there you go so now we have our Vector store in the session State we are creating our retriever chain from the vector store that is in the session State and now we can actually call our conversational rack chain from here so let's do that what we do let's just add here conversational let's call it conversation rag chain and this one is going to be equal to get conversational rack chain and then we're just going to pass in the retriever chain like that so there we go and this is the one that we're going to be able to use to actually answer the questions from our user so here let's do instead of calling it from the get response part let's let's call the conver ation rack chain like this and remember that since we're using a chain we can use the L cell language the Lang chain expression language to only pass right here the variables that we're going to need to replace in our in our prompt okay and that is actually very simple the first variable that we're going to have to to to to fill is the chat history and we're basically just going to pass in the chat history from the session State and secondly of course we're going to pass in the input from the user all right and let's just call this the response and let's see what the response is before before actually logging um like filling the messages Etc let's just let's just write the response like this and see what it contains okay so let's come right here let's just rerun this plug this in right here now it is embedding all of the contents here vectorizing them and here we have this now let's say what is Lang graph and there you have it here you have the response and apparently the response contains the context it contains the answer saying that langra is a module built on top of Lang chain Etc and it also return to us the chat history which is very convenient okay so now we see that we have have to use the answer right here um and this is the part that we're going to to to return um let's add this part all all of this part to the get response function because that's what we I mean that was the objective from the beginning right to to update this with the actual logic that is going to be happening behind the scenes so let's do that now you know that to refactor all of this into the get response function what we're going to do is we're going to take all of this and all of this out of here and Abstract it out to only leave the user interface with the function get response and that that should we we will deal with all of the logic behind the scenes so in order to do that I'm going to move this get response function and I'm going to move it right here underneath the other functions and it is here actually that I'm going to add these two functions right here there you go now I know this is not the most optimal way of doing this recreating the entire retrieval chain and the conversation chain every time the user makes a question but the thing here is that we're mixing the back end and the front end to make this easier to understand okay so in a real world scenario or in a more or a closer to production application what would happen is that all of this would be in the back end right and this would be initialized and we would just expose the the method from the last fun from the last um chain the method invoke that will be the only thing that we would expose but here since we are doing this all of it in the back end and the front end at the same time we are a bit forced to mix these things together so let's just return the response right here okay so every time the user asks a response we're going to build a retriever chain we're going to build the conversational retrieval chain um and we're also going to invoke it with this with the chat history and with the input okay and this function is of course never going to be called before our chat history is initialized because the only the only place where we call it is underneath here okay so now we can oops now we can uncomment this these two things and remember that the response actually included an answer element and this is the one that we're going to be returning because this one is the one that actually return that actually contains the the the answer from the chat but because there was the the response contains also the context and the chat history but that's not what we're going to be needing for now so this I suppose that should work let's just test it and see we're coming back here let's just reload this and again I'm going to try with langra again and let's see let's say um do you know what is langra let's see what it returns if it's going to tell us there you go and now let's just to test the conversational part of it can you explain that in simple in simpler terms let's see if it remembers what we were talking about before there you go of course langra is a tool that helps you there you go so now we have a conversational retrieval chain that works correctly with the data that we're passing in from a website okay let's test it now with Wikipedia for example so to test this again let's run it through the 2024 US presidential election and just to show you that it actually does not work without the augmented data I'm going to use just chpt 3.5 which is the language model that we're calling in our application and let's say who are the candidates for the 2024 US presidential election and it's probably going to return that it doesn't know because it's knowledge ends in January 20 2022 um so what we're going to do is we're going to take this URL we're going to paste it right here now this is indexing all of the data in that website I mean in in that web page and let's say let's ask the same question again who are the candidates for the 2024 US presidential election and it is going to say that as of now the candidates are have not been officially determined the incumbent President Joe Biden is expected to run Donald Trump Etc okay so it seems to be working all right who were in the primaries for example let's see if it also continues the conversation knowing that I'm talking about the 2024 presidential election and it does so everything seems to be working correctly um there you have it you have an application that is capable of responding questions responding to qu to your questions about any website um that has text in it and I hope this was an easy thing uh that it wasn't too hard but that it was challenging I hope that you learned a lot especially about the new features of the latest version of lank chain and how some of the packages have been restructured I hope that you like this and remember that if you like videos like this don't forget to subscribe and to uh leave a comment to tell me what kind of other videos you want to see all right thank you for watching and I'll see you next [Music] time [Music] [Music]