Step BACK Prompt "Engineering"

Key Takeaways

we're going to learn a completely new prompt engineering technique that is called stepb back promting developed by Google Deep Mind we're going to quickly learn what is this technique and then we're going to use a cookbook that is been put out by Lang chain and we're going to see a live demo of how you can use step back prompting to improve your rag retrieval retrieval augmented generation let's get started the first thing is we have got a new paper called step back prompting or take a step back evoking reasoning why abstraction in large language models technique is a very simple approach the approach is you have a original question and whenever you have a original question the first step is abstraction and in abstraction you ask the llm to create a step back question something that is higher level so for example if you have got a question that says what happens to pressure B blah blah blah blah blah then you just simply create a step back question that says what are the physics principles behind this question this the first step once you have the step back question ready then you use the step back question as an abstraction guided answer so you use that and create a step back answer and that is what is going to be used as reasoning to create the final answer let's look at another example if you have got a original question that says Estella neopol went to which school between August 1954 and November 1954 then you create a step back question that says what was Estella Leopold's education history and that creates a step back answer and from that you do reasoning to create the final answer it is all the algorithms let's quickly go ahead and then see a very quick demo before we actually do the handson this is a cookbook that has been put out by the L chain team and if you see in this question the question that they are asking is was chat GPT around while Trump was President this is the question was CH GPT around while Trump was president and for this question if you ask this question normally like the Baseline it would say yes chb was around while Donald Trump was president in fact you can see from this paper even Chain of Thought could not answer these questions without step back prompting so if you go back to this demo that they have put together so the Baseline Baseline is it says yes chat gbd was ared while Donald Trump was President while that is not true CHP was launched on November 2022 When Donald Trump was not the US president so how do you manage to do it it actually creates a step back question this is the base question and then from that it creates a step back question that says when was chat GPT developed then you use the Dr go search API rapper to get the information from the internet for both the question and also the question gen which is basically the step back question and you use both the knowledge to create the step back chain that will finally give you the right answer that says no chat GPD was not around while Donald Trump was President chat GPD was launched on November 30222 which is after Donald Trump's presidency as you can see that this question has been rightly answered using rag the retrieval augmented generation from the Internet thanks to the step back prompting let's go look at the Google app notebook to do handshot step by step the first thing if you're doing it on local machine you might already have the open AI AP configured if you do not have you can use this piece of code to configure your open a AP key in your OS environment path once you do that then you need to install four different libraries open AI of course to use open AA model line chain for you to use this entire PL chain duck go search for us to do retrieval augmented generation using internet and L chain hub for us to get the step back prompting prompt chain from the Lang chain Hub or the lsmith as you can see here you can directly get this into your own Lang chain workflow once you have all these things installed ready the next thing that you have to do is you have to import the required particular libraries and the classes from line chain. chat models import chat open AI this is the main the chat model that you're going to use and then you import the prop temp plate and other aspects then you need to also create the few short prompts for it to understand how to create the step back prompts or the step back examples so you use a few short chat message template to do that and then couple of other you know utility functions the first thing that you need to do is you need to give the examples okay whenever you ask a question like this could the members of the police perform lawful arrest then you create a step back question how does a step back question look like for this input the step back question should be slightly higher level slightly an abstraction what can the members of the police do that's the abstraction the higher level question for another question for example input Jan syles or Yan syles was born in what country whenever you have a question like this you tell the llm to create a step back question that sounds like this what is Yan syndel or Jan syndel personal history so when you give these examples then you turn them into the few short example prompt so you use chat prompt template from messages human the input the input that we just saw and the output from Ai and you use this as a few short prompt that can go inside the prompt template in itself then you create the chat prom template from messages the system prom the Bas basic thing like you are an exported World blah blah blah blah and then you have the few short prompt and then the user and the question so this way the large language model knows that whenever question like this is being asked it has to generate an answer like this and then that goes inside this question gen so it the prompt is there and you specify you know the chat GPT or chat open AI with temperature zero and then the final output partiel so once you send that I'll create the question engine for example for a question that says was GPT 4 around while Trump was pred that is a question that we are sending it to it and then we use question gen. invoke to take this question and create this output and once you have this particular output then you use from Lang chain utilities import d. go search APA rapper to go to the Internet and get the answer so you first use the retriever on question then you use the retriever on question gen inw which is going to be what is the timeline of gbd4 existence which is the step back question and then finally you're going to import or pull the step back answer from the L chain Hub and then you create a chain you have got the normal context then you have got the step back context and then you have got the question in itself and you get the response wrong with chbt API and then finally print the output and once you invoke the chain with the question it says no gbd4 was not around while Donald Trump was President according to the provided context the gbd4 was launched on November 30 2022 which is after the time period when Donald Trump was the president of United States and the Baseline is it just says that it cannot provide any information so let me quickly go over the court before I give you a quick demo so import all the required libraries like in this case Lang chain and different glass process create the few short examples of how it should generate the step back question once you have that then that is mainly what you're going to do for a given question what should be the step back question and then use whatever R technique that you have got to get the answer for both the question and also the step back question and then use that in a chain with the step back logic you have got the normal context you've got the step back context and then you have got the question so you're going to retrieve for the normal context retrieve for the step back context and then you're going to get the actual question you're going to send it and once you invoke it you're going to get the right answer so let's go ahead and then ask a different question now instead of this so I going to ask was something like this was um let me ask new question P at laa model laa multimodel model was lava multimodel model around while Donald Trump was was a president we all know that laa was recently launched and would ask the same question again and let's see what it says okay it created a step back question that says what is the timeline of laa multimodel models existence use duck du go search APA raer retrieve the answer for the question we going to ask it to create for the step back question once it is done then you can go here pull the step back answer from the long chain Hub or lsmith and then create the normal context chain the the chain that we discussed about and you ask the question so in some cases I got the answer only in the second attempt that's what I've noted here so once we ask the question we are going to get an answer let's see what is the answer going to be it says based on the given context there is no direct information or relevance uh the information privately provides about lava model by UC Davis and Microsoft research so it didn't give a wrong answer and also it didn't give the right answer so let's go ahead and ask one more question was Joe Biden the president while Trump was the US president and know it's a very weird question let's see let's ask the question again send it to Lang chain retriever so this is the who was the president of United States during a specific time period get the answer get the answer let's see what it does what it does what it does what it is okay here you go you got the answer saying no Joe Biden was not the president while Donald was the president Donald Trump served as the 45th US President Joe Biden on the other hand became the president as the 46th president so succeeding Donald Trump so we have got the right answer the factuality has improved hallucination has reduced it's a bitter rag retrieval augmented generation system all thanks to the paper from Deep mind about step back prompting and also thanks to L chain for putting together this wonderful cookbook that we can use right away for you to start all the required links in the YouTube description you can directly click the link and check it out see you in another video Happy prompting