PaLM 2 Meets LangChain

Key Takeaways

okay so in the previous video we looked at using Palm 2 invertex AI with both using it as an interface in the cloud and also using it with code to create a chatbot from scratch Etc in this video we're going to look at basically combining Palm 2 with Lang chain how you could use it inside Lang chain and we're going to do the same kind of thing that I've done for some of the other models where we basically get doing a conversational bot doing a a Rec retriever bot and then also in this one I'm going to look at doing a pal chain to assess the reasoning of the model as well okay so you're going to have to install these packages here to basically get this going and it's very important that once you've installed them you have to restart the notebook if you're using collab if you don't do this you'll basically run into errors later on once you've restarted The Notebook you basically go through the authentication dance with Google Cloud just to basically authenticate as a user so that the notbook can then contact Google cloud and contact vertex AI specifically and then you bring in vertex AI put in your project ID and your location here to initialize the project you can see the version of L chain I'm using and then we're going to start off just by using the the chat format model so you can see here that from Lang chain we're bringing in the two different kind of models we can bring in the chat vertex AI model and the vertex AI model so there are two ways that you can actually use Palm 2 there's one using vertex Ai and there's another one using makeer Suite I'll go through a separate video at some point for makeer Suite in here but this is using the Google Cloud platform vertex AI version of the Palm T models once you brought in the chat model you'll see that it's just like using the turbo OPI for open AI or Claude 2 where you basically need to bring in the different kinds of chat prompt templates so we've got a system prompt template we've got the AI message we've got human message in here and then to basically load the models themselves you'll see here I've basically got this one is just the standard completion model so this is text bison and then we've got this one being chat bison here for the chat vertex model both of them are set to same temperature same top P some top k for there all right so to do a simple message I just made make a human message and pass something in how do you compare to gbd4 and then just pass that into the llm here so you see here I'm getting back I am powered by Palm 2 which stands for Pathways language model 2 large language model from Google AI so that's just a standard thing of setting up you know setting up an llm and then pinging it now we could have used the llm the non- chat one and then just passed in a string into that and then that will return back so if we want to do a basic chat box with some memory and search this is the same as I've done for a number of the different models and you'll see that it's very similar only thing that we're really changing here is that we're bringing in the chat vertex AI chat model and we're going to basically just initialize that just like we did above here so we've got everything else is going to be the same from when we've done it for open AI when we've done it for for Claude so we're just swapping out the actual model itself here so we've got our search set up yeah and then you can see that because we're using a chat model when we're going to initialize the agent we're going to basically initialize the tools passing the chat llm that we've just set up here and we're going to have a conversational react description agent is the type of the agent that we're doing in here so remember that basically just means that this name and the description is being passed in and the model can then just make a decision to basically use these kind of things so for example when we run it hi I am Sam final answer is it just decides to respond there's no use of search there's nothing like that so we just get this hello Sam how can I help now I'm using the same questions that I used in the clawed model so this allows us to compare some of the strengths and weaknesses of of both models the thing that has changed is that day when I did the CLA model there wasn't a lot of information about Elon Musk x. on the internet now I think there's probably a lot more in search so it's actually returning different data was okay what is Elon musk's x. you can see here it basically decides to do a search it passes in Elon musk's x. we're getting an observation back which is actually was the search from Duck Dogo in this case and then you can see we're getting the finished chain or the final answer back Elon musk's x.a is a new artificial intelligence company founded by Elon Musk the company's goal is to understand the true nature of the universe so it's done a pretty good job at finding that information as we go through it when we ask it who is on the team we can see that okay it probably hasn't done as good a job as before more because the search results seem to have changed rather than the actual the actual llm changing anything so we get back to the team at Elon x. AI is comprised of Engineers from Google open Ai and other top AI companies what do they plan to do again the search now is delivering different results than what we had last time so we can there some talk about what they're doing and what they announced in this case the model's final answer is that Elon Musk XA plans to build a rival to chat GPT I don't think that's exactly what they outlined they were more interesting and doing some other things but it's certainly an interesting answer so next up we're going to look at using the Palm 2 for information retrieval here now I've basically deleted the files that I've downloaded here so basically what I did here if you didn't see the previous video about Claude is that I scraped a number of different sites and got a number of different articles about the launch of x. and I also basically scraped X a itself to get some information for that there's not a huge amount of documents there's only seven text files for doing this but obviously just due to copyright reasons I probably shouldn't share those here you should just scrape anything that you want and put in here or put in other text documents Etc you could even change this quite easily to be PDF files so you can see here that we're going to be bringing in at the chroma database looks like just today chroma has basically changed their database so you need to pin it to the previous version which I've done up above otherwise you will run into errors so if you're running into errors in some of the older notebooks that probably why we're also going to basically bring in pantic which it's going make use of and we're going to just load up those those text documents and you can see there's seven of them in total I'm then just basically splitting them another thing to point out here is that we're also going to be using the vertex AI in embeddings so this is from the Palm to gecko model and you can see we're just bringing in Lang chain. embeddings and this is going to let us use their embeddings for doing the vector store lookup Etc as we're going through this so we we bring in the the documents there's only seven of them we're going to split them up into nice chunks of 100 nice chunks of a th000 characters each with a 200 overlap in here now this this is some sort of boilerplate code taken from Google that just rate limits getting the embeddings so that you're not going to run into errors or something like that and then also this is where we're using the pantic base model for bringing this in so we basically can embed documents into this and it's going to return back a number of documents and we can actually set the batch size for the embeddings that we want to do here so you can see here the embeddings are going to be our custom vertex AI embedding things these are going to work out to be 768 long and we can basically look at how many of these do we want to do per batch there just testing it out we can have a look at one of them can see sure enough that we're going to get an embedding like that it's going to be 768 long that's what we're going to be using for the embeddings and if we look at the embeddings themselves we can see that this is going to be the text embedding gecko that we're using in here and it's basically got the settings that we set up before all right next up we're going to basically make the chroma database so here we're just passing in these embeddings into the chroma from documents so remember I'm now using the older version of choma I think this is going to change in the future but it seems to me that it's just come out and the past 24 hours and Lang chain itself hasn't updated to the new one so for now we're pinning the version of chroma to the previous version I'm passing in the vector retriever put k equals 7 for this as well so that we're passing in seven examples for each one makes it a fair comparison to what we were doing before with Claude model and you can see now we've basically just got a set up our llm so again we're using the chat llm for this and we're just going to stuff everything in and we've got our retriever that we've just basically defined up there and now we've got our QA chain to basically ask the questions so if we ask what did Elon announce Elon announced the formation of X AI a new intelligence company the goal of X is to understand the true nature of the universe so that's a very good answer and remember this is no longer using search on the internet to do it it's just getting the answer from those seven docks that I I scraped and put in there so we're doing a vector store lookup we're bringing the seven top results back and then we're looking at those in here when we ask who is on the team remember we asked this on the internet it didn't get good results here we're getting very good results team is led by Elon Musk CEO of Tes SpaceX team also includes and we've got names of many of the people on it so one that it didn't seem to get right whereas Claude did was that what is he going to call the his GPT model so it was eluded that he was going to call it the truth GPT model that's not in here basically just says he's not announced here which is technically true because he just alluded to it in the previous one who's going to be CEO and this sort of shines an interesting light on how the Palm 2 models and the open AI models are quite different than the claw model so I don't think it's specifically says that Elon Musk is going to be the CEO but it's kind of hint to that a lot so Palm 2 basically just makes it a straight up fat whereas if you go back to the clawed results for both this one and this one it tends to to hedge a little bit or it tends to tell you that okay while there was no direct answer for this this is what it was alluding to or this is what I think the answer probably is and that's the same for this question here is where is the company going to be based in here the final one asking about the China and AI it gives us the information back it doesn't give us the quote where Claude model was prepared to actually give us a direct quote which I checked it was exactly in there it seems that perhaps the Palm model is less likely to give us quotes I don't know this is something you'd want to test out more going through all right the last thing that I wanted to look at with palm 2 is the whole idea of using pal chains so remember pal stands for program AED language models and so the idea here is that we're going to use the vertex AI model we're using just the the standard completion model here so text bison 001 we're setting up the the panel chain from a math prompt so we can just pass in a question like this the cafeteria had 23 apples and this is one of the GSM 8K questions that you've seen me use many times over and you can see when we give it it's up to this model to generate code out now we could have actually used the code completion model here too this is something that you could try out yourself of testing out the code completion versus just the text completion here but this text completion model seems to do this quite well so you can see it generates a function which then Lang chain basically uses the output passer to pass this function and run this in the python read evaluate print Loop and we can see that we get the answer out which is nine in this case the correct answer this is one of the things that the open source models are just not good at being able to do is to be able to generate something out like a function that works like this also has the reasoning correct here now that said it's not like Palm gets these all right but it does get quite a lot of them so let's look at the uh next one up is able to get this one right so this one interesting one of basically comparing prices it then works out blue ties versus red that he buys two times as many red ties as blue ties which we can see is mentioned up here and then finally is able to generate the the correct answer out the next one though it doesn't get the correct answer out here you can see Maggie spent a quarter of her money while RZA spent onethird of her money they each had $60 so one should have spent $15 one should have spent $20 how much do they have left so 15 uh and 20 is going to be 35 from 120 we should have 855 here and yet it's doing 25 so the mistake that it's made is if you look here is that it hasn't worked out that they each had $60 it's just thinking that okay they had $60 to share and this was the total spend so sure enough this would leave $25 this is an interesting case and it just shows you how language model kind of think as they're going through when they're doing this sort of when they're doing token by token we want things to be in a good order so you'll see here that by putting that they each had $60 back here it's definitely confused it here it hasn't gotten the right answer if we take the exact same problem and we write it out Maggie and Risa each had $60 Maggie spent a quarter of her money while re a third of her money how much do the two of them left so I've just changed that one sentence to be like this and now straight away it can get the right answer it is something to think about getting your inputs in a sort of logical format that a large language model can deal with really has a big impact on this now you'll fine with uh gbg4 it probably doesn't need to have that as much I I do think the sort of gp4 is a grade above the the Bison models at least we don't have access to the Unicorn models so we can't evaluate those but looking at the Bison model we can certainly see that next one up is in November toy was $1,000 in December the price increased by 80% in January the price decreased by 50% what was the price of the toy after the discount so you can see here it's done a nice job of toy pric 1,000 times 1.8 for the 80% increase then the 50% of the December is going to be the January price price return that and it's going to equal $900 that's it's done a good job there repeat words now the words ones it doesn't seem to do as well for this so repeat cheese seven times every third say whiz so this one I guess depending on how you interpret it it does cheese cheese cheese whiz cheese cheese cheese whiz cheese and then none for that you could say that's correct or you could have expected maybe the third one to be whiz instead of cheese there the next one though you can see that it's on track even with these sorts of things so this is where the reasoning and sort of logic abilities of pal to are actually quite strong for this and we look at the last one so say the letters of the alphabet in capital letters only the odd ones so this should be 1 3 5 Etc you can see the result out is we got the opposite we basically got two four so we got b d f and those ones so it's just slightly off in its logic there does seem to be getting the right idea but it's certainly hasn't gotten the example for them anyway this is basically how you can use uh Palm 2 with Lang chain and vertex AI to be able to handle a variety of different tasks whether that's just plain chat Bots with memory whether that's using something with a vector store retrieval or something and thirdly whether you wanted to do logic and reasoning tasks with a power chain there the pump 2 does do quite good at these things definitely has a different feel than the open AI models has a different feel than the clae models as well so try it out and see what you think as always if you've got any questions please put them in the comments below if you found this kind of video useful I'm going to be doing more videos around these topics please click like And subscribe I will talk to you in the next video bye for now