Claude-2 meets LangChain!

Key Takeaways

okay in this video I'm gonna be going through anthropics Claude 2 and how to use it with Lang chain and we'll do a couple different examples of using it with Lang chain for a conversational bot where you've got a Search tool and also for some Rags some retrieval augmented generation so basically using a vector store to get information out and then have Claude to take in that information and don't forget one of the advantages with claw 2 is it's got a 100K context we're not going to use anywhere close to that but rather than just having say three documents coming in we can actually pass in a lot more for this okay so let's get started so you will need an anthropic API key these are actually pretty hard to come by at the moment but you will just put this in if you've got this my guess is that they will open up over time and once you've got this in there you'll then be able to use it just like you would with open air Ai and in the last video I looked at the pricing compared to open Ai and it's actually very competitive to a lot of the offerings that openai is offering okay so the first thing you need to know is that we basically use the Claude 2 model as a chat model so in link chain we have just completion models and chat models so they're using this new not so new now but they're using this chat sort of message format so first off we need to basically bring in the system message the AI message the human message for this so This is actually the same as what you would do with uh GPT 3.5 Turbo with gbt4 Etc we can then basically bring in the Claude model so here we're just bringing in the chat anthropic chord model and then we can just Define a human message and send it up now now in this case I'm not defining any system message or any of that sort of thing I'm basically just bringing it in and putting in a human mess message and you can see sure enough if I ask it how do you compare to gbt4 it will give us an answer back now it is very interesting the answers that you get back from this certainly Claude 2 is not perfect by any means but it it does seem that it seems to be more honest when it doesn't know things so there are definitely hallucinations I'm not saying there aren't but I I do find that you'll see especially when we look at the information retrieval stuff later on in the video that it actually brings out some really nice uses of the model where it tends to basically see if it's not sure or give more sort of answers that are built just upon the context of what what it the information that it's given okay so this is basically a just a standard generation out if we want to build a chat bot and incorporate some memory into the chat bot and search into the chat bot this is how we would do it so we're going to still use the chat anthropic model and we can use Claude to set temperature to be zero in this case we're going to have a conversational buffer going on here and for a tool for search I'm just using DuckDuckGo in here just basically it's free you don't need an API key for this you can just use it straight out so you can see that because this is a tool we're actually going to end up using a react chain here we need to give the tool a name and we need to give it a description so I've taken those from the sort of Lang chain standards that they use for the Google search engine they'll work fine for this you could also play around with this yourself if you wanted to have a specific use case for it all right setting up the actual agent so we're going to pass in the tools in this case it's just one tool we're going to pass in the llm I'm going to tell the agent type is it's a conversational chat right so it's using the chat format and it's a react description agent in this case so meaning that it's using the react template I've covered this before in the past so my guess is if you've watched a few of my videos you've probably seen this okay so I've set the most to be true here just so we can see what's going on and if we start out just asking a simple thing hi I'm Sam you can see it actually has the personality built in right hey Sam nice to meet you I'm Claude an assistant created by anthropic to be helpful harmless and honest so you'll see these three words used a lot both in this but also in the data sets and stuff so okay I've picked the topic of the day to be Elon Musk announcing this week about his x dot AI so I asked it what is Elon musk's x dot Ai and you can see sure enough the react kicks in we're not having any problems with the sort of reasoning on the react chain it's returning stuff back which Leng Ching can then pass quite easily and it basically decides that it needs to do a search it's going to do the search is going to be Elon musk's x dot e i it basically does that gets the information back and then summarize the key details and gives us this answer back so we've got Elon Musk recently announced the creation of a new company focused on artificial intelligence called x dot a I we can see the company was formed after musk file paper work in Nevada so there's standard information that we can see there so if I ask it okay who is on the team interestingly the search doesn't do a great job here it doesn't seem to get the names of the people and some of them are quite famous AI researchers so that's why I was interested to see okay would it be able to get that it doesn't seem that it has and that seems to be more to do with the search not returning it just returns about the first 12 employees from what we can see there and we don't actually get the names back scrolling through yeah we don't actually get the names back of the people in the context from the search so in that case we wouldn't expect this to give us the actual details but it does use that context to form an answer pretty well so in your most recently announced x to the eye additional team consists of 12 members including a musk himself other members from organizations like openai and deepmind that specialize in AI research and development details on specific missions and focus of x dot AI are still emerging all right so when I ask it what do they plan to do so here you can see it's executing the search Elon Musk team plans gives us a response back it does actually get this gives us a very similar response to before except now we can see that it's advancing AI capabilities to help understand the nature of the universe which is their big goal so that's using it just in a conversational chat bot with memory and with search so I should point out that like here I basically didn't have to mention x dot AI again I just said what do they plan to do and he knew that I was talking about Elon musk's x dot AI That's because we've got the memory in there and it's able to use that so that's the kind of standard chain a lot of people will use with maybe multiple tools or something like that so you could just take that and use that straight away doing something for information retrieval what I did was just wrote a quick script and went and scraped some articles about x dot Ai and the x dot AI site so there's not a lot of these I think it was about seven in total I can't share the zip file of those I think just for copyright reasons I shouldn't be sharing that but they were from places like TechCrunch x dot AI itself and it's not a huge amount of text you can see when I bring this in it's only seven documents that we brought in here so in this one we're going to be using Claude 2 for information retrieval we're going to be looking at just loading this up anthropic as far as I know doesn't have a publicly available embedding API now it could be wrong but I decided to go for the instructor embeddings here so that's why this notebook actually using a GPU but I'm actually using the lowest end GPU T4 for this so you don't need a big GPU to run this and and he can basically say the instructor betting's still probably my favorite embedding that's open source you can just use as much as you want I keep meaning to make a video just about instruct embeddings because the way they do it is very cool I just haven't gotten around to it anyway so we're bringing that in we're loading that model in and we're going to use that with chroma DB and I've done videos about this before we're going to put persist our little Vector database on the disk here and we're going to use those instructor embeddings now when we actually make the retriever here normally you would have seen me use like a retriever of k equals three or something here I can go right out to a much bigger one so I've gone for k equals seven to make sure that we're getting the right answers in there and don't forget that's we could go probably even out to close to k equals 70 even more with this because we've got such a huge context window of a hundred thousand tokens we probably don't want to go out that far there's not a lot to be gained from that but it is nice to be able to go out to to seven I kind of feel is a nice spot why don't we want to go out to 100K the reason why we don't want to go out to 100K is probably just we're going to be wasting tokens we're wasting money at that point so the 100K token limit is fantastic but it's not big enough that you would fit an entire company's data in there right that would be millions and millions of tokens and it's also it just doesn't make sense for Speed and for cost to try and always stuff everything into the context window you could play around with this yourself and scrape some things and you could actually probably stuff all seven of those articles or all seven of those scrapes into the context window and do it that way but if you're going to be asking lots of questions across that you're probably wasting money by doing that so it's not something that I would recommend you do so in this case I'm going for a context we're going to basically put in seven contexts in there so after that you can see that next up I'm just loading up the chat anthropic model we've basically got the temperature set to zero here and we're defining it our chain here so this is just going to be a retrieval QA chain where stuffing everything in because we've got such a long context window there's no need to do anything but stuff in this there's no point in doing multiple passes or something are we pass in our retriever in this case I'm doing return Source documents equals true don't have to do that I'm not really using them for anything here and then we can basically run it and see our outputs so when I ask it okay what did Elon Musk announce it comes back with Elon Musk announced the launch of a new artificial intelligence company called x.ai a very nice succinct answer and clearly answering what we asked there okay this time now when I try and ask it who is on the team this is the kind of answer that I think is really interesting to get from them so you can see here that we've got this I apologize but I do not actually know the full list of people on the xai team based on the information provided the context mentions some notable members like Elon Musk and then it goes on to actually list the people out that are mentioned or at least some of them that are mentioned on the site it also mentions Dan Hendricks advises the team however a full list of team members is not provided so that's certainly given us better than what we got from just doing the search before when we ask it okay what is he going to call his GPT model if you remember Elon was talking about the truth GPT and sure enough here it basically says based on the context provided it seems Elon Musk intends to call his GPT model truth GPT in an interview from April 2023 he said and then it basically gives some quotes and the quotes seem to be accurate from when I checked them there's one coming up about China and I checked that and actually that's a real quote from him who's going to be CEO okay based on the context provided it appears that musk will be CEO of x dot AI the announcement States our team is led by Elon Musk CEO of Tesla and SpaceX where is the company going to be based now this is a kind of question I ask because it deliberately doesn't say that as far as I know so here we get I apologize but I do not have enough information in the provided context to definitively say where x dot AI will be based the context Mex mentions the extra days recruiting engineers and researchers in the Bay Area and most aai development has been concentrated on Silicon Valley so I think this bit it's adding itself now the Bay Area that stuff like that that makes sense but then it also I put deliberately had some articles in there about musk and China and you can see that so it brings that up as a location so it talks about that he's traveled to China but it says you know however there's no explicit statement about where xai's headquarters or main office will be located this is a really good step forward in getting out information from the context of where it's prepared to tell you that hey I don't know the exact answer for that here's some things that might be the answer but they're not there's no sort of 100 Proof in the context that you've given me of that answer so I think this is this thing really shines final one what did musk say about China and AI and we can see based on the context provided Elon Musk said that China is interested in working cooperatively on AI regulations specifically he said China is definitely interested in working in a Cooperative International framework for AI regulation now I searched this out and this quote does exist from him in some of the Articles so it's good to know that it's doing that beyond that I don't know exactly what else muss said about you know China and AI the context talks about must reach and trip to China and meetings with government officials there as well as China's recent interim measures of AI but it does not provide direct quotes from musk on anything else related to China and AI again this is really useful if you are looking to get something that is prepared to tell you hey I don't know for example with chat GPT or the 3.5 turbo model it will often just make up things in this now I'm not saying that chord 2 doesn't make up things but on the whole it seems like the the responses that we're getting here are more consistent with this kind of thing anyway have a play with with the notebook if you've got access to the API I have a play with the notebook you can try it out I might make one more video just showing you some of the reasoning of what they're doing so as always if you're interested in these kind of topics I'm going to go back to doing some of these with open source models as well please click like And subscribe if you've got any questions or anything please put them in the comments I will talk to you in the next video bye for now