Build SELF-DISCOVER from Scratch with LlamaIndex

Key Takeaways

hello everyone this is Raa developer Advocate engineer at Lama index in this video we'll look into implementing a recent paper which is self-discover uh large language model self composed reasoning structures as a Lama pack so let's start with understanding the approach and what this paper is about so basically uh self-discover is a general framework for llms or you can say prom flow strategy to self-discover the reasoning structures to solve a complex problem that are in general difficult with usual proing mechanism so basically it uses llms to select multiple reasoning modules from a large pool of reasoning modules which you can check towards the end of the paper to compose a reasoning structure for the llm to solve the task or the problem that uh is given to it so they showcased that the approach improves the gp4 and pal to performance on challenging Ben benchmarks by margin of 32% compared to CO2 methods so this is designed from a thought that we as humans if given a new problem we think U about what skills we already have from prayer experience and how can use those skills to solve the particular task or problem so the approach is basically divided into two stages um stage one and stage two basically the broader input is a task which we need to solve and the reasoning module um basically I said as I said uh there are uh some reasoning modules from which you select to solve the task right so these are the two inputs U for the whole process and then yeah let's start with the stage one process which has three modules to say which are select adapt and Implement so select chooses the relevant reasoning modules from a large pool of reasoning modules the paper use some 39 reasoning modules and then once you get a selected modules from the first part uh you sent it to adapt and based on this uh module certain modules reasoning modules refres those descriptions to be task specific and then finally um those rephrased ones are sended to the third part which is Implement which creates an execution plan from the adapted module descript ions uh which is something like a reasoning structure similar to a Json format and then in the stage two we basically give the task as well as this reasoning structure for the llm to solve the task and given an answer for the given task or the problem so now uh let's see how you can implement this in the Lama index as a Lama pack we will actually use Query pipeline to implement it um is a probably a simple sign of proms flow so let's uh see how you can do that essentially we have a task and reasoning modules as inputs and give them to a select prompt so basically in the select prompt we ask the llm to select some of these reasoning modules to solve this task and you basically get selected modules uh as an output and then we have an adapt prom uh for the given task and uh for these uh selected modules you send uh them to the llm and then refres the descriptions of these modules for the specific task and then you have an Implement prompt for which there is a task and send it to llm and get a reasoning structure and finally along with the reasoning structure for the specific task you basically generate an answer so these are the different parts of the the whole process and which is part of stage one and stage two so we'll see so the uh we'll implement this whole flow using query P plane um concept which is available in the Lama index take the input send it to the prompt or get a prompt template and send it to llm get the modules and then have an adap prompt uh send uh send the previous step input output as an input to the next one and continue with that process so you can find the uh the Lama pack file here and then uh so it basically contains a self discore pack and uh so that will have certainly some three more uh three functions get modules run and an IT function where you can initialize the llm and then uh the default reasoning modules and the veros so this the Run function has has basically you need to configure the pipeline as I said there are uh two stages and the first stage has three different parts which is Select adapt and Implement and the second stage is uh use the reasoning structure and for the task to generate an answer so the [Music] pipeline has a pipe uh all these parts to it so for the configuring the pipeline you basically set up the templates whatever we have discussed and then add these modules to it and set up the links and uh have the pipeline ready and once the pipeline is ready you basically give the task and the reasoning modules as a input and basically uh get the answer for the task so here if you can see um these are the templates you have uh select uh prompt template which has which takes task as well as reasoning modules as input and then adapt prom template which T takes uh the um task as well as selected modules from the previous step and the third one is the Implement PR template which takes uh um the adapted modules from the previous step and the task um and finally this is given to the final one the stage two one takes the reasoning structure and then task to give a final answer for the given task or the problem right so and then yeah the core the core has the all these modules here so there is an input component which takes the task and the reasoning modules as a high level input and you have llms for each of these parts like select llm adapt LM Implement LM and reasoning llm and all these uh prompt templates right right uh so we have all these modules available now so we just need to set up uh the links between all these modules that I have shown in earlier in the figure right so you start adding the links like what is the input for the promt template and which is Task and then the reasoning module as the input and you send the promt template to the llm okay and once you get whatever the output you get you basically send the selected modules to the adapt right adapt from template so um that's what you do here and then you even have you even send the task as well as the selected modules to the adapt R template and pass the adap prom template to the llm again and similar case with the Implement prom template you have the adapted uh modules and task to the Implement prom template andm get a reasoning structure and send the reasoning structure to the uh the final reasoning prom template to get an answer so that's how you configure the entire Pipeline and then you run it um by taking the task as well as uh reasoning modules so we have a Lama pack ready and then we'll use this llama pack to solve some of the task so we'll go through this notebook on how you can use the Lama pack to solve a task so we need llm for the for it uh we'll use the default one which is gp4 llm for it and then you can actually use the the Lama pack in two ways uh One downloading the Llama pack and then directly use the cell discard pack from the available llama packs so I'll go with the second option for now you can even test the first one so let's get the Lama pack and pass the LM um we can send the gp4 llm or any other llm you want to test it out but if you don't send any llm as an input um it directly uses the default GB 3.5 Taro as an llm and then uh we have some task defined here so one of the task is like U Michael has 15 oranges and he gives four oranges to his brother and trades three oranges for six apple with his neighbor later he disc got two of them and then goes to Market to buy 12 more oranges and five apples and decides to give two apples to his friend Fin and we want to know how many oranges and apples does Michael have at the end during this process so this is the task question and then we'll uh go through it so you can see how each of these things are running you have the input which is Task and reasoning modules and then send it to select prompt about the task and reasoning module to get a selected uh modules and then use that prompt template which takes the task as and selected modules as input and send it to the llm to get the uh rephrasing versions and then the Imp Implement prompt template as well uh and finally get an answer so so basically it has it the answer says like Michael initially has 15 oranges and zero apples and then after giving 4 oranges to his brother it has he has 11 oranges left and then he basically trade some apples and get uh for oranges and so each step um you basically do some operation and get a final answer that he has 18 oranges and N apples so that's how um it break down the whole problem into different steps and uh did some reasoning and got a final answer in the same way there is another task here where Tom buy some ingredients for dinner and um you want the estimate of uh the cost basically the cost for purchasing these items and so basically he pays with a $50 bill and what is the chain one needs to receive purchasing these items so in the similar way um the task reasoning modules and all these stages it goes through all these stages and finally gets an answer um the actually the time taken for Sol to solve the task depends on how complex the problem is because um the number of selected modules might be more based on the task so that's how you can see now this task is taking more time than the previous one um the previous one hardly took 27 seconds and this took uh more time right and so basically yeah it identified the problem and gathered the information broke down on the problem and did some thinking and solved the problem and uh reviewed that solution and got the changes like $31.5 so that's how you can use uh self-discover to solve a complex problem uh please explore this Lama pack and um see how you can use in your workflow uh see you in the next video with a new llama pack or any other new technique available thank you