Building Web Chatbots with LLM: Chapter 19

Key Takeaways

[Music] we should be ready now to build our simple web application which will allow weights andb users to ask questions from our bot let's get started in the config P we Define our configuration including the weights and biases project the job type if you work in a team then we set the team which becomes the weights and bies entity and we also Define where do we pull our Vector store from and where do we pull our prom template from and in this case we are using weight Andis artifacts that we created in the previous video we can also set the model that we want to use uh to run our application we will be also using several functions defined in the chain. pi file specifically we want to load our Vector store and again we will pull the vector store from weight Andes artifacts where we uploaded it after um ingesting documents we also want to load our chain and here specifically we will use the conversational QA chain from longchain uh we will be using the chat open AI model and we can choose between 3.5 turbo and GPT 4 uh in this case we Define in the settings GP uh chat GPT uh which is a 3.5 turbo model and we Define the parameters which again we pull from the config file and then we return this conversational retrieval chain which is a utility defined in L chain and we will pass it on to our application so that uh so that we can respond to user questions by calling this chain and finally we have the get answer function which uses our chain along with the user question to provide back a response so let's go back to our application in this case we are using a simple gradio user interface uh which we will expose as a web application we Define this chat class which will become our interface H that persists our Vector store and our chain in between user calls here we need the configuration and we need the weight run which uh is used to pull our artifacts including the vector store and the chain what we call this uh chat interface uh what happens in the background is in case um the vector store or the chain are not loaded we load them uh we save them in that uh chat object and then we use the get answer function to create a response and append it to the history then we use gradio blocks to Define our user interface specifically we have a text box for the user question we have another text box to specify open AI API key if we want to our if we want to expose it to our users and we don't want to share our open AI API key in this case I have set this as an environment variable so I don't need to provide it but in case you want to expose uh your application on the web and you don't necessarily want to sponsor um the open AI API calls then you can request the users to provide their own key uh we're also storing the state uh we have uh our uh chatbot object and then whenever a user submits a question we call our chat object um along with the default configuration and we pass the question along with a state and receive back the answer that we can uh expose in our user interface so let's run this application now and see how it looks um in our browser again we're using weights and biases to pull our Vector store and our documents and the application is running on the local uh URL so let's uh click this link and we can see our Q&A B uh exposed um via our browser uh we can deploy this uh we can deploy deploy our application for example or on hugging face spaces or on replit there are multiple ways we will not go into the details of deployment in this course but I encourage you to check out the additional links we provide uh below this video to see the different deployment options that you may use let's add try to ask our uh chatbot query just to verify that it's working properly and now we can see the answer uh we can see that to share a report with team members there's a list of steps we can follow including saving the report clicking the share button and so on uh it could be correct but this looks in fact a bit different from what we've seen in the previous um videos when we uh when we explored a similar query and we implemented our retrieval chain and maybe it would be interesting to see what happened behind the scenes and how this uh chain was actually run what was fed uh what was retrieved from our Vector store which documents were fed into the prompt and to do that we can use a weights and biases Tracer so let's take a look there all right so when we go into weights and biases uh remember we set our environment variable L chain W be tracing to true and that means that all of the queries and responses are streamed into weights and bies and we can analyze it there and see what went well or what might have potentially gone wrong in this case we can see the question that we asked in our web interface how do I share a report with my team members and there is also the answer that we also saw in the user interface and we can also see the trace timeline the sequence of um of function calls that uh ultimately resulted in our generation we can see the conversational retrieval chain that we Define in L chain which in turn calls the stuff document chains with a bunch of input documents uh from L chain and then if we go drill down into the llm chain and ultimately the open AI uh we can see uh the prompt here which includes all of the documents that were stuffed into this um into the template and if we scroll through this list of documents we can see um first like the system part of the template uh we uh should be able to see uh different documents which were retrieved from the vector store um and if we scroll through uh through this we can uh hopefully find a document that talks about about collaboration and sharing qu and bies reports and it tells us that once you have saved the report you can select the share button to collaborate draft copy of the report is created when you select the edit button draft reports out to save and we can see like this is actually what influenced um the answer uh of our llm chain so by using weight invis Tracer you can debug you can understand what happened behind the scenes what is the sequence of API calls uh that was done to produce the answer and in this case we found in the prompt the document that was used to generate this output uh this way you can uh do error analysis if some of the responses are potentially flued as unhelpful by your users you can see and drill down what happened maybe uh the correct document was not retrieved for a given query and we need to improve something on the document search and retrieval site maybe the model didn't perform so well and didn't find the right answer within the prompt and maybe we can steer that for example by better prompt engineering or changing to a more powerful model maybe we should switch from GPT 3.5 to GPT 4 or anthropy clo or a coher command model there are many options available and by doing this experimentation we can improve our application and this will be the topic of the next module where we will uh develop a deliberate process of experimenting improving our application and evaluating our results see you then