Faster LLM Function Calling — Dynamic Routes

Key Takeaways

today we're going to take a look at Dynamic routes in the semantic router library now Dynamic routes expand what we can do with this Library by quite a lot unlike a static route a dynamic route is able to dynamically generate the parameters based on a particular input that can then be taken into whatever you want to do with those parameters so the main use case here is function calling now the good thing with Dynamic routes is that they can generate this output but they're still very fast just like our static routes so they are fundamentally the same object and I think what would be best is to just take a look at how they differ which is not by a huge amount okay so we're going to start in the docks of the semantic router Library I'm going to go over to Dynamic routes and I'm just going to open that notebook in collab it's now on version 0.050 this is actually no longer necessary so I need to remove that so I'm going to insall the library first then I'm going to come down to here and I'm going to initialize a static route now these are just basic static routes and the reason we're loading those first is because we want to see what the difference is between these and a dynamic route so yes we initialize those and then we're going to initialize our route layer now the initialization of a route layer whether you have dynamic or static routes or both is exactly the same it doesn't change and again we can use go here we can use open AI there's also now a new Fast embed encoder as well if you want to run the embedding part locally I'm going to use open AI because we will also want to use the open AI llm as well so API key enter this and there we go okay we do also support that go here llm as well and soon enough you will also have local llms but for now I'm just going to use open AI it's the it's the easiest okay so we can test that it's working and this is purely static routs let's see how we might create a dynamic route so here is how we would set up our Dynamic route you don't need to do it like this directly you can actually soorry this is the actual definition of our Dynamic route what I'm doing before here is creating this the function schema that is required for our Dynamic route so the function schema I can just show you what that looks like maybe quickly so if I run this it looks like this this is our function schema now what I'm doing here with the get schema function here is I am taking an existing function and I'm formatting it in a particular way so we're using the Sphinx do string format here we're adding a lot of description as to how exactly this function should actually be used so finds the current time and a specific time zone okay that's like the description okay what does this function actually do we need this for our Dynamic gr to understand you know what this does and how it should be used then we specify okay we have our time zone the type of our time zone is a string and the description for it is this okay so the time zone to find the current timing it should be a valid time zone from the I time zone database and then we give some examples and then we specify do not put the place name like Rome or New York you must provide this particular IA format so we do that that is then going to okay we provide this format and it's going to give us the time in that particular place now we run that function get time that we just created here we put that through our get schema function here we get our function schema output and then this is what defines the difference between a static route and a dynamic route we simply pass in this function schema to our to our route definition so if I remove that this is now a static route if I add that back in it's a dynamic route okay and that's all there is to it so we have our new Dynamic route I'm going to add it to our route layer and then I'm going to ask a Time related question okay and that should trigger the time or the get time Dynamic route and it should hopefully get the right inputs for that route okay and we see that it does so we have function call and we have these inputs for our function so then I can connect this up to the function that we created so say out equals this and I want to say get time and it's out and it's the function call and then and I see that's the this okay let's see what we get okay six 16 and basically you can expect this to work with any function C that you'd expect an LM to normally be able to handle because we're using an LM here so what we're really doing is we're setting up that kind of like agentic workflow where an agent agent will decide what to do and then generate the input for whatever it decides to do we're taking away the decision part on you know which route to take or which tool to use and we're using semantics to make that decision but we still rely on the llm to generate the function call itself which you know we've seen it it does and it does pretty quickly now that's it for this video I hope this has been useful and interesting so thank you very much for watching and I will see you again in the next one bye