Building agentic LLM application Workflows with LlamaIndex

Key Takeaways

hi everybody I'm lri VP of developer relations at llama index and Welcome to our YouTube channel today I'm going to be walking through workflows our brand new abstraction for building complex agentic applications I'm going to be walking you through our notebook which shows you all of the steps in building a workflow and all of the things that you can do with workflows first we have to install our dependencies we just have three llama index core which is the important one we're also installing llama index open AI because we're doing one thing with llms and we're installing llama index uil's workflow which is what contains our visualization tools that I'll be demonstrating later once we've installed our depths we bring them in as Imports having done that we bring in our open AI key from The notebook's Secret store because we need it to be able to instantiate the llm now let's look at the most basic possible workflow it just starts it does one thing and then it stops there's no reason to have a real workflow that's this simple uh this is just demonstrating the most basic possible workflow so you can see how they're defined so most workflows are going to be defined as a class the class can have any name so in this case it's called open AI generator and it inherits from the workflow class within your workflow you can have as many steps as you want want each is defined by a decorator called Step once you've uh thrown in your decorator you can define an arbitrary function name and you use type declarations to Define what types of events it accepts and what types of events it emits in this case it's accepting a start event which is a special kind of event and a stop event which is another special kind of event the start event is how the uh workflow orchestrator knows where your uh workflows should start the start you can have as many as you want but it all starting events will go to all the functions that have uh that accept start events so in our generate function we're just setting up our llm we're calling GPT 40 uh and we're using it to complete our query uh our query we're pulling out of our event as you're going to see uh you can attach arbitrary data uh to your events and uh in this case we've decided to pass a query once we've got a response from the llm we call stop event and we pass back our response as a string now that we've defined our workflow class we can execute it so first we instantiate our open AI generator and you have two parameters here one is your timeout which is how long it will run before halting it's important to give your give yourself enough time uh to actually execute whatever your workflow is going to do uh and verbos which we're going to have false at the moment but we'll see what that looks like later once you're once it's instantiated you call run on your workflow uh we're awaiting it here inside of a notebook that works if you were running this in vanilla python you would need to wrap this function in an async function in order for it to work we pass the query whats L index and we print the result and it gives us this response from G describing what llama index is one of the neat things about workflows is that you can visualize them we use p is to do this so we call draw all possible flows uh and we call it with the class name and what file name the output should go to and it will generate an interactive draggable diagram uh that shows all the possible flows within your workflow I've taken a screenshot of it here you can see that there's the start event which goes to the generate function which just goes to the stop event and then it's done there's not a lot to see here yet because we haven't made a complicated workflow so let's go to a more interesting example this one demonstrating our ability to to loop we're introducing a few new things here one of them is custom classes you can create Uh custom event classes to pass around your workflow you can create as many of them as you like you can give them arbitrary names and you can attach arbitrary metadata to each one in this case we've got a failed event with an error and a query event with a query in the loop example we've defined two steps one is the answer query step it can accept more than one kind of event this is the other new thing it can accept a start event or a query event and it can omit a failed event or a stop event because llms are so good these days and needed this thing to fail sometimes I'm using a random number generator to simulate failures in this case if my random number between uh is zero then I say emit a failed event and if my random number is one then I emit a stop event simulating a success our second step is improve query so if answer query emits a failed event then improve query will pick up that failed event it can either emit a query event or a stop event if the random number is zero then it will emit a query event simulating a an llm improving your query for instance if your query failed because it was too vague you could use an llm to improve your query and resubmit it uh or if it couldn't figure out how to improve the query the stop event gets emitted with that result let's draw all the possible workflows there you can see our start event goes to answer query which can either go to a stop event if it succeeds the first time or it can emit a failed event which goes to improve query improve query can also fail it can go to stop event or it can emit a query event which goes back to answer query this is that Loop that I was talking about and it can branch and you can see how you could in a complex workflow put together a whole bunch of these to manage your work manage your data moving uh around your flow let's execute this now this time when I instantiate it I'm going to set verbose equals true which means it's going to Output what steps are getting executed it's running an it's runs the step answer query which produces a failed event because it produces a failed event improved query gets run in this case improv query produces a query event which goes back to answer query answer query again proves a fail uh produces a failed event which goes back to improved query which produces another query event which goes back to answer query which this time produces a success and gives you the answer to your query so now we've done looping and branching let's talk about State uh as a convenience we have a global state within your workflow this is super useful for for instance loading data and then making it available to all of your functions in this case I'm simulating loading data by creating a a database variable with some values in it I'm doing something new here which is I'm calling the step decorator with pass context equals true this means that the context a variable of type context will get passed to the function you can attach arbitrary data to your uh context using the context. DAT object uh in this case I'm setting up this fake database and then returning a query event the query event will get picked up by our query function because it says that it accepts query events and in this case it's pulling right back out of the context data the value of some database which it can then output we run this with verbos equals true and you can see that it works exactly as we described it runs setup produces a query event it runs query produces a stop event which gives us our result of course this flow is still essentially a a linear flow a more realistic example would be if your start event could either be a query or a data population event and you needed to wait for your data to populate before you could start answering queries so let's set that up and see what it looks like here's our weight example flow class again we're passing our context and in this case if the incoming start event has a data attribute then we attach it to our uh fake database variable uh now our query accepts a start event if it sees a query attribute attached to the incoming start event and it goes and looks and sees if there's any data in the context data and if it does it returns a stop event with the data if it doesn't it simply returns none and does nothing let's run this the first time we run we're going to run with a query the result that we get is none so it says no you can't because we haven't loaded any data yet the second time we run we attach some data saying yes you can and you can see that the query step gets run but there was no event produced by it because there was no query attached to it what we attached to it was data but the setup step did uh run and set up our uh pulled our data into the context so now the third time when we run it and we ask it it we ask if we can kick it uh we get the data yes we can because query gets run and produces a stop event let's visualize how this flow Works our start event can go to either setup or query and both of them produce stop events under different circumstances waiting for incoming data like this is a very common pattern so our context object has a convenience function called collect events this will capture events and store them and it will return none until all of the events that it has been specified to to collect have been collected those events will then be attached to the output of collect events in the order that they were specified let's see what that looks like in practice in our collect example flow we have three types of of custom events an input event a setup event and a query event and we have three functions set up collect input and parse query Each of which accepts a start event the setup event uh runs only the first time and it uh sets up some data in the context and then it emits a setup event the collect input function will run uh if it sees an input attribute on the input on the incoming event if it does it emits an input event the parse query event simulates a query if it sees a query attribute then it outputs uh a query event which will then get uh then all three of these will get collected uh by run query runquery we've specified accepts an input event or a setup event or a query event and the first line calls Collective events and specifies that we want either that we want a query event an input event and a setup event if if uh all three events haven't happened yet then what ready will return is none so this function prints not enough events yet until it has seen all three events happen if it's seen all three events happen however it says now I have all the events and shows us the output of what ready is so let's see this running first we call run and we can see that the setup event runs but uh run query says not enough events yet then we run with some input and you can see the input function runs but still not enough events yet and then the third time we run with a query and it says I've got a query and now it runs it says I have all the events and it shows us what's in our the output already the important thing to notice here is that the output of ready has these events in the same in order that they were specified so in collect events I said give me a query event an input event and then a setup event and you can see here a query event an input event and then a setup event we can also visualize this so we can see a start event going to parse query collect input and setup in all three cases you can see them emitting their specific type of event and those events all going to run query which is what calls the stop event another thing you might want to do with collect events is to monitor failures in this case we're calling a failure example flow which tries to do a thing in this case generate a random number less than five uh sorry greater than five um if it's greater than five there was a success and it says and it calls a stop event saying that there was success if not it emits a failure event the monitor failure uh function accepts failure events and then it has a collect events call which is waiting for three separate failure events if there haven't been three failure events uh sorry if there have been three failure events then failed will be not none uh and it will print that the same simulation has failed and admit a failure uh result to the stop event so now let's call this six times in a row see what happens first there was a failure so it returns none then there was a success so it returns success then there was a success so it returns a success then there was a success so it returns a success then there was a second failure so it returns none then there's a third failure and now it says that the simulation has failed and you can see that our failure events are different and it has collected all three sep separately the visualization here is again pretty simple you can see it's calling try to do thing it can go to stop event if it succeeds or it can emit a failure event which goes to monitor failures which also goes to stop event the final thing I'm going to demonstrate is stepping through a workflow step by step this is really useful for debugging I've instantiated a new copy of failure example flow here with verbos equals true and instead of calling run I'm going to call run step you can see that it runs the step trying to do thing and that there was a failure and that it produced a failure event then it waits until I call run step again then it calls monitor failures because that failure event was waiting for me to run steps and then monitor failures produces a stop event we can visualize this using draw most recent execution in this case it calls the instance of the class rather than the class name and we give it a file name again it puts this file which says there was a start event which went to try to do thing which went to fail which generated a failure event which went to monitor failures you can see how this would be useful for debugging stepbystep a complicated workflow that concludes our tour of creating running and visualizing workflows feel free to check out the show notes with dots examples and the link to this notebook thanks for listening and I'll see you next time bye