LangChain v0.1.0 Launch: Observability

Key Takeaways

observability is a key part of Lang chain and we've put a lot of focus into making L chain have the best observability as possible as you're building complex applications with language models whether it be really long sequences of chains or whether it be more agentic Behavior being able to understand exactly what's going on inside your application becomes really really important and even if you're working with simpler chains often times the prompts you construct come from many different sources and so being able to see exactly what the inputs to the language model are exactly what the outputs are and all the steps along the way is really really important to being able to iterate and improve your application and we've designed Lang chain to have a bunch of different options to make it incredibly observable and have a high level of observability into exactly what is going on so walking through a few of these different options let's take a look at what it looks like if we're using an agent without loging we'll go over agents in other sections of this tutorial but here we can construct it rather quickly and then we can invoke it and we'll see that it takes a little bit to respond under the hood and and we'll see this later on it's making a few different LM calls it's calling out to a search engine and so there's quite a bit of of uh API calls that are being made and so we get back a final output but we don't exactly know what happen happened inside and so that makes it tough to iterate on the prompts there's you know we're using an agent prompt here um there's uh the llm that we're using there's the output parser that we have and all of those can be tuned according to what we want to accomplish but in order to do that we have to understand exactly what's happening so the first method that we have we added this really early on in Lang chain and it's been around ever since it's it's it was inspired by offer press's self- askk paper and it just prints out um the steps that happen um at at a highest level so we're not getting all the details but we're getting the high level of what's happening so if we set verose equals true here um and then recreate the agent and then we call it again we can see here that we have some nice printing of what exactly is going on so we're calling this tool with this search query we're getting back this response and then we're getting back this final answer and so again we're not seen all the information but we're seeing high level what steps are occurring so this is a good first pass we've added another method since then called debug equals true and this logs way more information and so it might be Overkill sometimes but other times it can be really helpful to see exactly what's going on so here we'll set verbose equals false we'll set debug equals true we'll recreate the agent and then we'll call it again and here we start getting lots of logs of everything that's going on and so this is really useful when you want to dig into what exactly is happening input output level of every step along the way so here we can see that we have so this is the exact prompt that's being used we've got a system message and a human message and again when you have really complex prompts in this case it's kind of simple but when you have really complex prompts it's really useful to be able to see what's going on we can see the formatted prompt we can see the exact Return of the chat gener um we can see the token counts there and then you know we we keep on getting this for the rest of the agent this is great if you're in a a repple environment like this uh uh Jupiter notebook but when you're maybe not in this or when you want to persist these logs more or if you just want to explore these logs in a more intuitive way right this is this is a little bit of a dump of everything um then you're probably going to want a more uh full-fledged tool and and that's why we built Lang Smith um Lang Smith isn't the focus of uh this video or the uh uh launch of Lang chain 0.1 in general um but it's absolutely worth highlighting because Lang chain is built to be the most observable framework but we've also built tools along the side to make that a reality so with Lang Smith what we can do is we can go into lsmith we can go to our project we can just set these three environment variables um and so we can do that here and I'm going to pause the video and put in my API key so you don't see it I've done that now we can now set debug equals false just to get rid of this noise and then we can run it and what's going to happen is in the background it's going to log everything to this Trace section here and then I can click into it and I can see the exact steps that occurred I can see so so here we're highlighting so as you notied with the Json there's a lot of events that are happening there's a lot of different formatting of prompts that are happening parsing that's happening um and so we have one view where you can see all of that you can click into the ones you can see uh uh what exactly the inputs are what exactly the outputs are um you can go into a playground experience and and edit it and play around with it there's a bunch of other functionality that's part of Lang Smith that I won't go into here but then we also have this most relevant run where we just show you the important events like API calls to llms retrieval calls tool calls things like that and so this is a really good way to see exactly what's going on inside your chains agents even just L calls if you're making if you're doing a lot of uh uh complex prompting there and and uh again this isn't the purpose of Lang chain 0.1 but I think this really shows how we've instrumented the whole Lang chain ecosystem around this really important idea of observability um which we've found to be incredibly crucial for building llm applications that actually work