AgentWrite with LangGraph

Key Takeaways

okay so recently I looked at the model called long writer which was all about creating an llm that can generate articles 10,000 out now one of the things I mentioned in there was the whole interesting thing about this thing called Agent write which is what they use to actually make the data set in there so in this video I'm going to show you making a version of agent right where we don't need to use their model we can just use other models to make long outputs and really this idea you could abstract to a whole bunch of different things and I'll talk about that as we go through the Lang graph code and have a look at this so just quickly to go through the concept here is that what they ended up training their model on was this data set of really long articles and the Articles basically have a sort of standard prompt input and then they might be 5,000 words out I think it was for anything from 2 to 10,000 words is what they generated I'm going to show you in here that basically you can build an a agent with Lang graph that does the exact same thing if you wanted to make your own data set to get a really specific data set in this case the promp ask for an article that's 5,000 words long you can see that it's come out to be over that as we go through this if we look at the paper the cool thing is they've basically outlined what their agent does they've also given us the prompts and stuff like that so we can reproduce this and in their GitHub they've even given the code that they used to do this and they're not using any particular agent framework in there they've just got everything hardcoded in there so what I wanted to do is basically convert that across to Lang graph and even look at some ideas of how we could improve this system so the system here is that basically you give it an instruction so here you've given it an instruction to write an article on the Roman Empire he will then make a plan and it will decide like paragraph one should be about this and that should be 700 words long paragraph two should be about this that should be this amount long so you've got this plan and then what you do is you split these up and you send the original instruction the plan step whatever the model's written already in and you get it to write the next paragraph in there and you just keep appending these to it so we know that the models can handle really long input context so that's you know not a problem so each time it's like we're giving it the whole article and just asking it to write the next paragraph for this and telling you what the next paragraph should be about and how long it should be so the concept is simple now what you can do is take this even further and you could put in self-reflection steps you could put in tools you can put in a bunch of things that you could add to this using the Lang graph framework but what I wanted to do just here is just show you basically how you could get this together and then also talk about the whole idea of getting composability when when you're building your agents so that you can swap things in and out you know quite easily we'll look at doing this with a few different language models so that we can try see how it goes for various language models as we go through this so let's jump into the code and have a look at this okay so the first thing I guess to cover and just remind ourselves is that we when we're using Lang graph we've got a few key things we've got to build the graph and this time rather than just put it in a collab I'm actually breaking this out into a proper project a little bit rough because I've just put it together but you'll see how we can make everything composable in here so we can reuse these we can basically repurpose bits of the code quite easily as we go through this okay so the first thing in Lang graph is we're going to have our graph right and for the graph we need to basically Define our state where we're going to have a dictionary of things that we're going to pass around we are then going to basically Define the nodes and then we're going to define the edges of how everything is done here and this is a pretty simple project to see how things go and then we can expand on this to add in other things which I'll talk about as we go along first off I guess let's look at our main file so our main file we're basically just going to import our environment variables so this is going to have our keys and stuff like that we're then going to basically instantiate the graph right so this is what we're putting together here and then putting together the instruction that we've got here let me just make this easier to read so we can see that this instruction or this initial prompt as I often call it as I'm passing it around is Right a 5,000-word piece about the HBO TV show Westworld and its plot characters and themes make sure to cover the tropes that relate to AI robots Consciousness finally tackle where you think the show was going to go in the future Seasons had it not been cancelled all right so the key thing is I guess we're going for 5,000 words now as I showed you before we do end up hitting the 5,000 words right where getting over 5,000 words for doing this and that's not something that you would normally just get out of an llm right unless it's fine-tuned for doing that all right let's look at how this is put together so our graph basically we're going to just take this graph State and the graph State we're going to have the initial prompt it's then going to make a plan and we can see this sort of down here if we see okay we're going to have a planning node we're going to have a writing node we're going to have a node now we could put in some extra things like a a self-reflection node in there we could use some tools if we wanted it to do some research about this really you could take this a whole bunch of different directions quite easily from this but here we've got our initial prompt our plan the number of steps I like to keep that our final document that we're going to have out the word count and then the LM name that we're going to have in here now one of the key things is because I'm trying to keep everything composable I've got the nodes as separate things and I've turned out in in this case I've put a file for each node now I could have put all the nodes in one file right in here and I didn't even need to make it as a folder in here but I think this is a good way of being able to plan out the different things now if you're used to doing things with perhaps crew AI or something like that you would think of these as having all the separate agents in some ways you can think in Lang graph of each of these nodes as being agents that are going to do different things so in this particular one what is it doing the planning node right so this is obviously going to do the planning thing we've got some stuff here that we just print out to basically see that it's running we get the initial prompt we update the number of steps that we've passed around and we're going to make a plan now to make that plan we're going to use the planning chain right so let me just close this out if we come in here and look at the plan chain here we can see that it's going to bring in the prompt for that so if we look in here we've got our planning prompt and this is what I took from their paper right so they had prompts for this and we can just pass in the initial instructions the initial prompt to go in here but as we're doing it in the chain we can basically just load up that text file now the cool thing is at any point if we want to change our prompts all we have to do is come in here and change the text files they've got one for planning they've got one for writing I'd probably add one in for self-reflection stuff like that perhaps some for doing research or doing other things like that but you can see here we've got the planning chain we've got the writing chain so the writing chain and in fact actually in both of these I'll show you sort of like a trick that I do is that when I'm creating these chains because at the start you want to create your chains right and you want to make sure that they're going to work so the chains are just this Lang chain expression language and here it's basically is taking out prompt that we're going to feed something into it we're going to pass it into the llm we're going to pass it into the output passer now one of the things that I often do with these is that because you want to test these and see okay are these prompts good are they giving me what I want that kind of thing what I can do is I can just add in a little main part where now if I come in here and I just run this file it will basically just take a test instruction run it through through this chain print out the results so I can see okay is the plan chain working like I expect it to work let's have a look at that now okay so you can see if I want to just test this part of the code I can basically come in here run this it's getting an LM already itself and sure enough we can see that it's made a plan right that this was appr prompted about asking it to do intimate and fasting workbook this is taken one of the ones taken from their data set and we can see sure enough it's decided that it needs to have six paragraphs the first paragraph is going to be 300 words the second paragraph 400 words third one 400 Etc and each of them has got their own separate Topic in here so we know that the plan chain is working now there if we come and look at the right chain we can see also it's doing the same thing it basically takes in the right prompt it's going to take in the template for this and then we can pass things in now this particular promp passes in lots of different things we've got the instru instuctions we've got the plan we've got the current written text and then we've got the step The Next Step that we're going to basically do in there so you can see when we want to test that we can basically test it by giving a test instruction a test plan a test text and then pass in the step that would actually be in there now I've just got paragraph one in here to see that it's running going back to our nodes now you can see we've got the planning node there we've got the writing node which is a little bit more complicated I'll talk about that in a second and then I've also got a a saving node and the saving node I've added a few things in here so that I'll know I'll add the total word count I'll get it to basically do a count when it does the writing of the the number of words we'll save that at the end we're going to write these to markdown so I'm just bringing in the tool that I've made right markdown file in here now I'm not making extensive use of tools in this one we certainly could expand this to do some research about the the thing that we wanted to write about to do a variety of different things and you could imagine that for each plan step for each paragraph he could go off and do a search about specific keywords related to that paragraph and pull down the info and then combine it in that way as you're going through so even though their agent right idea in the initial paper is actually quite simple this could be expanded to be a much more sophisticated kind of perplexity researcher or doing something really really specific to what you want it to do anyway we can see finally we end up saving out both the final Dock and the plan in here all right if we look at the writing note is definitely the most complicated one so we've got a function up there just to help count the steps I could have made that its own node but it's a bit overkill for that here we're basically going to get in the initial prompt we're going to get in the plan that was made in the planning node and then we're going to split that plan so just like we could see before each plan had a step for each paragraph in there we're splitting those up and we're going to then basically say if it's got more than 50 steps it's too long but if it's less than 50 steps and this logic I took from their paper going through it then we're just going to go through this list of steps in here and we're going to basically pass in the initial instruction which is actually the initial prompt the plan the text that's been written so far and then we're going to pass in the steps so you can see this some debugging code we can see that once it generates out like the first paragraph it appends that to the responses and it also then adds that to the text that we've got in there so we're basically getting out each of these now why am I keeping them separate in the list as well is that I could have a reflection step in here that checks each one and says okay is there a nice transition between paragraph 2 and paragraph 3 or paragraph 6 and paragraph 5 we could have something where it looks at that so here we're just going through we're doing a loop going through this we finally join them together we do a word count on the final document we print that out so that we can see that and then we basically pass in the final Dock and the word count to the state things so that they can be used in the next node which is the saving node right so that we looked at before so we've got our nodes we've got our entry point is going to be this planning node we've got the edges in here this is a really simple little agent that's just stringing things together and in some ways my defin of agent is usually that you'll have some kind of decision point in there which this doesn't have right this could be done purely with chains but it does show you that this is how we could take this and expand it to be much more sophisticated and also I just wanted to show you a simple example of where you've basically separated out your chain CH your llms your nodes and really your tools is the next thing that you would separate out maybe into a folder there if you've got multiple Tools in there I'll show you some more examples with tools in a future video all right so I've got this now I can basically just run it and we'll see that it's going through planning the writing it's going to then basically write the document and then once the document is done it's going to save it out so this westw one that I did here is the first one like that we'll get a new one coming out and I've also set it up so that we can see what llm it actually was written with okay so we can see that it's gone through it's finished it's printed out it's written a doc that's going to be 5,661 if we come up here I've made it so that it's saves at the end what the actual llm that it used here so we can see that there's our Westworld Doc and sure enough at the end it's done a word count and we've got over 5,000 words there if you want to see the plan we've also saved this out so we can see that the plan that we got was paragraph 1 paragraph 2 paragraph 3 and you could imagine you could even put things like a human in the loop step where it shows you the plan first and you could go in and edit the plan so you could go in and say oh actually you know what I don't like paragraph 7 I want you to change that to something else or actually for paragraph 3 I only want it to be 300 words so it gives you a lot of control you could imagine doing sort of a human in the loop step for that okay so now I'm going to just show you like if I wanted to basically try this out with a different llm so I'm going to come in I'm going to turn off the GPT 40 model in here I'm going to put on llama 370b okay so I've updated this now so we're going to be using the Llama 3.1 70b model in here and I'm just going to come into Main and I'm going to just change the model name to be llama 3.1 so we know where that's coming from in fact what I'll do is I'll just call that Gro so that we know it's from there okay I've gone in there I've actually fixed up the name of the model as well and now if we run it sure enough it's doing the planning it's doing the writing of the dock in here it's going to I guess we should have probably put the hyphen there to match what they've got but anyway it's right the dock as we speak and then it's done all right now we can see that in this case it's basically the dock wasn't quite 5,000 words we got 4 and a half th000 words from this but you can see again we've still got an article about this if we look at the planning we can look at the Llama 3 planning we can see that actually in some ways maybe this was simpler so you could imagine that we could use one model for doing the planning if we could use one model for doing the planning perhaps more high-end model and another model for doing the actual writing of this if we wanted to use the Google model we could basically just bring in that and use say the latest flash model if you wanted to bring in olama we can bring in olama if you want to do something with a local model you've got a whole bunch of it's easy to swap in out models in here because everything is in this modular and composable way so this is something then that we can also deploy quite easily right just as we've got this main file up here we can actually deploy this to have maybe an API endpoint that this is getting sent in each time we could have put in an input thing for put someone to type in their prompt as we go through this but the idea here is we could basically now use the same kind of idea as their original agent write but we can swap it out the models we can swap out the prompts very easily we can add in nodes for self-reflection and other things as we go through this since I originally looked at the long writer it looks like they've also done another paper very similar kind of thing because once you've got one of these things working it's quite easy then to turn around and generate a bunch of different data sets for different kinds of things this one seems to be for making citations basically training a model to get really good at giving out citations in here and sure enough if we come in and look at this we can see that they've got again a similar kind of agent slpress for making this supervised fine-tuning instruction data sets in here so overall I thought this was a nice way to look at building an agent that's very reusable composable that you can swap things in and out from this and use it as in the paper they obviously made something that could generate a data set but you could also use this for generating reports that someone asks for you could imagine you could put in a rag system with this you could put in search really there's a whole bunch of different things that you can do just by adding to the nodes adding in conditional edges in here Etc that allow you to expand on this whole agent idea in here all right as always if you found the video useful please click like And subscribe if you've got any questions Etc please put them in the comments below I'd love to hear what people are doing with this if there's any particular thing that people would like me to cover as an addition to this let me know in the comments I can certainly show you how to ban this I've already done a video about putting sort of rag into this kind of thing so that you can look at this I also thought that moving away from collab for a little bit was a good way to just show people how to put something like this together in more of a real world project for something like this anyway thanks for watching and I will talk to you in the next video bye for now