Why Use LangChain? A Blunt Overview for Advanced Engineers

Key Takeaways

fellow Engineers welcome by the end of this video I want you to understand Lang chain to the point where you know if you should or shouldn't be using link chain to build your agentic software in this video I'm going to share key features of Lang chain we're going to talk about strengths weaknesses and I'll give my raw unfiltered opinion of Lang chain based on my decade of experience building software my name's Dan I go by Andy Dev Dan and I took this deep dive into Lang chain so you don't have to unless it's actually valuable for you let's dive in so I want to answer this question for you guys length chain why is this useful here's a general outline of everything we're going to go through we're going to blaze through these what I want to give you is the high level idea of what Lang chain can do for you skip ahead at your own risk these ideas do compound on each other at the very end I'm going to give a summary of why you should use length chain and why you should not use Lang chain why would I use Lang chain instead of just querying open AI or whatever LM provider that I'm using so this is actually pretty slick you can get set up with your llm and three lines of code no API requests no handling errors no anything like that it just works I've queried open AI you can do that with a single and a multiple response format and so as you can see we got the results returned as you can see here this is the llm model so this is the base model this is text eventually right this is not gpg turbo this is not a chat model another feature of Lang chain is that they have async support out of the box right when this runs it runs async and it fires off all of these LM queries at the same time another Super useful feature oftentimes you don't need to keep requiring you really want to reuse your results so you can set up this llm cache and reuse the results so as you can see here with the maxed out temperature we're going to get different results every time but since we're using the llm cache this string has already been cached pretty useful this is going to keep your costs lower when you need to re-query all right so you know another solid feature so far so good right potentially useful things you can serialize and of course you can track your token usage so let's talk about the chat model right so their chat interface definitely feels a little verbose but it also feels clean and simple so you know as you can see we're setting up the openai chat we're setting up our messages we have a system message which describes the top level functionality of the llm and then we have the human message which is you know just you querying the llm so pretty good and then we can dump the response down here so I'm asking if I should quit my job and pursue my business full-time to my AI chat that is a elite product manager and you know giving some pretty solid advice telling me that uh I should validate my product or service have a clear path to revenue then it might be worth considering the chat openai model interface does use GPT 3.5 turbo out of the box which is great reusable chat prompt template so if you've done any amount of work with llms you know that generating the right prompt and templating your prompt is a lot of where the magic happens let's be honest that's I mean llms are all about communicating via language so prompting is super important I found that the Lang chain API for prompting and templating uh is just a bit verbose like why do I need to build all these intermediate classes it's just string templating um and this is a theme that you'll see I think as we move forward one of the big critiques of Lang chain is that it's just string templating this is kind of true this is also not true it'll make sense as we proceed but as you can see you know helpful assistant is going to help us convert English to Spanish we're going to pass in the text and these variables get properly filled in to our system and human template so that's chat based templating there are simpler ways to do this which we'll show off in a second here other things that could be useful with the chat models um there is streaming support so if we hop over to link chain here you can stream back the entire response so you can get that you know real live interface type experience nice to have I know some people really want this and don't want to build it themselves so this could be useful for you and of course we get our token usage that's kind of expected let's move on to text embeddings all right so this is kind of the big one of the big interesting features text embeddings really are taking Center Stage right now in the AI world and I'm not sure that's the right path to go down it makes sense it helps us compare information and then generate responses but just focusing on Lang chain it's a really really really clean really simple implementation right so just three lines I've got my open AI embeddings I've got my text and I'm embedding my text right this is kind of the dream right you have your function and you have your content and you have your response right and in all perfect cases all perfect scenarios this is how software works everywhere as you can see our response is a big Vector with uh you know our text spread out against Dimension so that it can be compared against other types good stuff here really simple so let's talk about prompts some more a lot of what you're going to be doing with llms is creating templates and replacing the templates so link chain has really taken note of this and have built some pretty solid tooling around it so you can see here I'm actually saving and reloading this prompt into this naming consultant prompt.json so we can open that up naming consulted prompt.json as you can see we've you know created a serialized Json response right so you can imagine you you have a bunch of these prompts stored you know wherever you might store them and you can reuse them based on whatever condition needs to be met so you know we then reload the prompt from the file and then we do our prompt templating then we query we're building a AI product that generates software automatically so we get you know Auto gen Ai and if I go ahead and rerun this we're going to get Auto build AI or codecraft AI it's weird that this is just string templating or it's kind of silly that this is just string templating and there seem to be two different prop templates one for chat and one for llm again I wish llm providers would just smash these together I know there are reasons for it but this is kind of useful you can provide few shot examples as you know sometimes it's helpful to provide examples to your llm this helps the llm create more accurate responses so kind of useful depends on your use case let's talk about indexes right so this is like I said For Better or For Worse the center of a lot of AI work right now you have a large Corpus of data and you embed that data and then you query against that data using simulatory search link chain has realized this and created some pretty sophisticated tooling around it that solves the common problems surrounding embeddings right so a couple cool things right at the back we have the text loader we have documents we have the text splitter so this is definitely not a simple problem and it's great that they have an interface to automatically split text for you there are big problems here with losing context of the chunk that you're chunking when you have a large Corpus of data and they have a single class with a single function that splits this into documents for you that's fantastic I'm using chroma DB let's say local open source embeddings database just to keep things simple using open AI embeddings building up the database you pass in your row and you pass in your embedder and then you can see here I have skateboardingtricks.txt that's getting chunked and then I'm asking what tricks require a 360 rotation and then I'm doing the DB similarity search and getting all of the responses so as you can see here I did a decent job it's picking up you know 360 hardflip 360 Ollie heel flip uh 360 360 kickflip 360 pop shove it but then it's got some bad results in here right like it's these are chunk by documents so it's just returning the entire documents but uh you know air grab Bull and ramp this is more of a problem of me not chunking properly but the point is made you can do embeddings and simulator search very quickly I mean what do we have here maybe you know 10 11 lines of code super useful let's go ahead and move on so when it comes to indexing and running question answer on your embedded documents link chain provides a really clean really simple way to do that there's a bunch of magic happening here under the hood definitely check out the documentation if you want to learn more about that but in even fewer lines we have a index to quickly respond to our query and so now I'm saying lists all frontside 180 tricks and as you can see here the front side 180 tricks are front side 180 nollie front side 180 switch front side 180 so it looks like it got all of them and just to show off the skateboarding tricks it's just a list of all you know a ton of skateboarding tricks it's important to differentiate between querying on your data and running similarity search right similarity search is a lot more generic it's just going to give you back things that seem related to the text that you've written in your query whereas full on llm querying is going to allow you to ask questions about your data link chain provides a great way to accomplish that all right so this is one of the areas I think link chain really shines you can load data from basically anywhere in this example I am loading data from a CSV file and they also have a notion database table loader which is pretty fantastic I found that this was kind of slow not sure what's up with that but just to list a few more zero CSV data frame email figma Drive Hacker News we have you know notion HTML images you can pull in images uh YouTube um word docs they have a really really great library of document loaders and so why is this useful a lot of what's going to differentiate llms is what data is it being queried against right we all have that data set up to you know September 2021st whenever the cutoff period is that's not special what's special is what data set can you give your llm and langchain helps give you a bunch of out of the box loaders to quickly load your data from many many common sources there's S3 in here a lot of people are going to just go right for S3 of course right the documents were loaded there's definitely better ways to format this but you can see here here's the data set right so home price square footage bedroom and if we hop back you know we have home price square footage bedroom so that's all great all right so let's look at another example there's this idea of vector store retrievers which is what I was showing off in the first example where you can write away a query on your documents right so we have get relevant documents query and run similarity search on your your documents I'm printing out the page one by one and I'm loading using the hn document loader so Hacker News document loader and this is the post topic right so chat gbt outperforms Physicians and quality empathetic answers to patients good stuff I'm trimming some of the comments off because I don't need all that um no it's just a safe space uh from the uh to save space and uh expenses from the embeddings here right so just don't need to run that and then you can see here I'm saying what's the worst case scenario of AI outperforming positions so we get some of the you know ideas with negative sentiment on chai gbt outperforming Physicians here based on that query that's pretty powerful again I want to highlight the out of the box loader in just two lines we have data from this Source one of the big things I look at and I'm sure that you look at this too when you're looking at new tools is how fast will it allow me to go how much BS boilerplate do I need to set up and work through how many times am I going to have to code a wrapper around some endpoint some variety of endpoints beautiful solution to that and it's going to save you a time one more thing I want to highlight here with indexes there are different options for text splitting check that out if you're interested let's talk about chains you know up till now we have been talking about models prompts and indexes now chains are where things get interesting this is where we start putting things together chains chain together of links which can be Primitives or other chains Primitives include prompts models and arbitrary functions okay so that's really important the big question when I was using Lane chain is like what's a chain what are the components of a chain why is it called Lang chain after you watch this section it's going to make sense so what is a chain here is a chain so in this example here we have provided an llm and we have provided a prompt and we're saying verbose so we're saying you know be detailed tell me everything that's happening as before we have a prop template what's a good name for programming language that focuses on attribute I'm going to focus on the fast performance and then we're printing that out so this is using the openai llm should not have called this chat that is uh wrong so let's go ahead and fix that quick and down here we are running the exact same version using chat so the output for this looks like this right so we have the llm and we have the chat version we got Lightspeed code and we have rapid code so that's not really important it's kind of doing what we've already done but the key here is that we're now building these out in Chains and chains are going to be useful as we move on so let's go ahead and look at some more use cases of chains this is pretty huge when I saw this I kind of like had a okay this makes sense moment right kind of the light bulb moment so with chains you can create sequential chains of prompts so what does that mean let's look at an example right what's a good name for a new programming language that focuses on attribute that's prompt we create a chat and then we create this chat chain right second prompt write a catchphrase for the following programming language okay so now you can see how this is going to start tying in together so that's second prompt so then we set up our chat chain two now we have the overall chain that wraps together both chain one and chain two funny how I use the number and the letters here whatever and now I'm running just on the first prompt and as you can see was a good name for a new programming languages for new programming language that focuses on speed Swift flow and now catchphrase let your code flow with swift flow Okay so you know small example simple example but this is powerful why is this powerful when you think about a problem you don't just think about the first answer to the first step right like say you want to set up a new server there are 50 decisions you need to make 50 questions you need to answer along that you are actually creating a Chain of Thought when you're going through that process of thinking about how to do something complex simple sequential chain gives you that ability to tell your llm the steps it needs to run it gives it the the power to solve more than just one problem and chain together reasoning just like you or I would when we're thinking about solving a problem that is a big deal we're chaining prompts together we're building a network of question answer nodes some other notable things about chains you can build your own custom chain class so maybe you want to you know clean up the responses in between maybe you want to you know run arbitrary functions between chains or wrap the chains in some new class you have all that ability you can do all that you have chain serialization and then we have the transformation chain so with the transformation chain you can run custom functions on your input and output of the chains all useful stuff let's talk about memory why is memory useful why do you need to give your AI memory well memory is really the foundation of making good judgment what does that mean so I know that this gets like kind of philosophical but that's where we're at with llms and artificial intelligence what is memory memory is experience memory is skill memory is an understanding of things that have come before which gives you the ability to alter the future to make better decisions AI without memory is not that useful it doesn't really matter how well it can respond to one thing if it doesn't remember the past if it doesn't understand your problem with your data it's not useful that's why we need to give our AI memory memory is knowledge memory is experience memory is skill so let's look into how length chain provides your AI with memory langchain has several different types of memory and several different ways to incorporate memory let's use one of the simplest ones right so just like in the chat gbt interface you have question and answering right so here I'm setting up a chain notice how we're reusing the chain concept right we're chaining together an llm and now we're chaining together memory okay now what I'm going to do is before I even prompt I'm going to say memory save context I'm going to give it information input and output so this is like a conversation that we've haven't had yet but it will have knowledge of right so here's me with a query what are the the top three programming languages and here's the AI responding python JavaScript Java and now I'm saying explain your ranking so there's no way to answer this unless you have some type of memory of something that we just ranked the AI starts responding with the ranking it's explaining the ranking it's explaining why python is first JavaScript for web dev of course and then Java there it is we just gave our AI some memory of course this is really simple memory there are several several memory Integrations SQL redis Vector stores of course are the most popular raw memory and then more there are tons of ways to do this I just want to nail home the concept that without memory your AI is not that useful it needs context of your problem okay so a couple other things there's entity memory we have custom memory classes so you can build at your own memory classes link chain was really great about providing custom interfaces then we have multiple memory sources in the exact same chain all right so agents this is where everything comes together using Lang chain this is how you create a gentix software we have to talk about the big bag of tools that Lang chain provides out of the box what do I mean by that here you can see in three lines of code four lines of code I am running a DuckDuckGo search that answers the question what's the current time in UTC now similarly to our chain we have passed in a tool with our llm with the agent type and we're saying B verbose so you can see here thanks to the verbose flag the agent thinking I don't know the current time in UTC therefore I need to search for it action duckduck go search action input current time UTC it gets a response back and then based on the results it's returning the final answer now my agent has access to essentially a search engine and there are of course you can use Google search you can use Bing this array here is really magical why is it magical because you can give it several tools and then it will decide which tool is the best for the job there's a lot of work I think to be done here it isn't perfect when you give an agent a tool and a problem now we are down the path of creating agentic software software that does things on your behalf automatically here's another example of an agent with several tools this is where things get really really really interesting so I instantiated a shell tool provided by Lang chain and I created a prompt then I created a custom tool it's very likely that you'll have several custom tools and I just created a simple logging service and you need to describe it for the agent so it understands when it should use your tool and then you need to create a function that accepts input and does some work in this case I'm just passing in a Lambda function that just prints special logging service and then dumps whatever comes in I'm formatting the description a little bit this is right from that lines from the link chain box and then I'm initializing our agent and as you can see here I'm passing in both tools I'm passing in our llm um specifying the agent type there are several different agent types that you can use passing in the prompt and then I'm saying verbose true so notice how this is a lot like the chain functionality let's look at my prompt here read all files in the current directory and Report the attribute then log the results the attribute I'm saying is create a date plus file name and then I'm running self ask with search right so that's just the agent I'm basically saying agent run this and let's look at the output you can see it working through this so what is the task thanks to the Shell tool it's running the shell Command right and now it's making an observation based on the response you can see it ran LS L and now that it's done that it is automatically because I asked it to log the results here you can see that it is thinking okay I need to log the results so let me take that action and run the logging service which is the name of this tool here so it's running the logging service and as you can see the output is special logging service a list of files in the current directory with their created date and bam that is the output that comes out thanks to the tools thanks to the prompt and thanks to the agent you can start to see it all comes together here one of the last big Concepts here as we wrap we have toolkit agents so what is a toolkit agent so a toolkit agent is a set of tools that are operated that's operated by an agent so you can see here we have a CSV agent which is answering the following queries how many rows are there and what's the difference between the cheapest and the most expensive home so we have this homeprices.csv we can pass that into our CSV agent so you can think of the toolkit agents as higher level abstraction wrapping agents and tools we have 128 and you can see here it is is using a python tool write python code and determine the response of the highest price minus the lowest price it's using whatever tool it needs to to solve the problem in this case it looks like it just used the python tool okay and you know maybe useful there are different agent types and you can Define your own custom agent types the agent types use the llm to determine what tools and what actions they need to take what's the summary of all this so there are two ways to look at Lang chain the first approach is link chain is stupid it's just a bunch of rapper classes around string templating I think that thinking about link chain like this is not incorrect but it is not doing it justice there are tons of other out of the box super useful features like the document loaders like the agent tools like some of the pre-built agentic tool sets as they call them are quite incredible and help you get up and running quickly a lot of what langtain is doing is just string templating unfortunately maybe 70 but the other 30 percent in combination with the 70 is super super powerful super powerful so that brings us to the second way to look at Lane chain right Lang chain is amazing it's a massive open source tool set for building complex AI pipelines what are we all saying there what are we all trying to get we're trying to build a gentec software what is agentic software I keep saying this what is agentic software agency software is a software agent that solves problems on your behalf without your intervention if you take that and extrapolate it there are massive massive gains to be had by building the right agentic software if blank chain is that tool to help us build a Genting software we need to be learning it and mastering it and using it so let's break down the cases in which you should use Link Chain and when you should not you should use length chain if you want speed customizability and versatility in your llm development as you saw there are lots of three four five line quick setup quick execution interfaces to get up and running okay uh you should use Link Chain if you want llm optionality the list of models that Lang chain supports is uh pretty sizable I think they're really really great about not locking you in to a single model provider and that's what you don't want if you build your interface in a raw way where you're building like an open AI module you're locked in right that is an example of walk-in to be fair it's not a ton of work to switch out but why even go down that path when you have a framework to allow you to switch out of that llm automatically with just a single class if you want out of the box data loaders and indexers the out of the box tools that linkchin provides to me is the biggest reason to use length chain one of the biggest reasons to use Link Chain it's going to save you time it's going to help you not write more boilerplate code that has already been written by hundreds of Engineers and lastly if you want your AI to have memory then you absolutely should be looking at a tool like Lang chain like I mentioned memory is experience memory is skill memory is context the as01 are going to have interesting data sets right they're going to have interesting memory formatted the right way so that their llm can access it when it needs it if it needs it I also want to highlight the agents and the tools they are very powerful you can get up and running very quickly let's get to the negative right so why when would you not use Lang chain you should not use Link Chain if you don't have time to learn a new API there are a bunch of classes here there are a bunch of new interfaces a bunch of new parameters that you're just going to have to take time to learn you should not use Link Chain if you don't have a complex use cases or you don't have complex use cases and only need a few prompts I would say in that case just do the string templating by yourself print any memory or context that you need in a string template especially if you're using gpt4 don't bother with this it's going to burn more time then it's going to help if your use cases are simple and I think that link chain actually says this in the documentation it this isn't for simple use cases right all right so if you don't believe llms have a future of AI then you should probably not use Link Chain I think that there's a big assumption going around right now that llms are like the thing that's going to be the future of AI it seems like they are it seems pretty promising I'd probably bet you know 65 of the farm on that but that leaves 35 of us just burning time here on llm so that's a you know definitely a personal decision if you don't believe llms are it don't use time Learning Lane chain which is basically a giant rapper around llms if you don't need memory in your AI then I don't really think that you want this link chain isn't going to be that useful that's one of the big selling points of Lang chain and other libraries they help you interface with memories they help you interface with Vector databases Vector stores they help you load data and a lot of that is all to give your AI memory if you don't need memory you probably don't need length chain build a small string templating class build a you know yaml file or Json file with your list of prompts with templating and you're good to go really you're good to go that's my summary I am bullish if you're playing poker I would raise on Lang chain I think that it is great software I am going to be using link chain in my development and my efforts to build agents that solve problems alongside and for me automatically so that's my take on Lang chain I definitely want to know what you think you know huge thanks for watching I thought this was going to be a shorter video but here we are um hopefully I can cut this down uh and and you know really compress it for you guys but you know thanks for watching subscribe and like for more deep dives into new AI tools without having to take the Deep dive yourself you know let me do the Deep diving for you guys let me save you some time I know what it's like to you know see a new freaking gbtx software XYZ it's uh you know exciting it's also really annoying it's impossible to keep up so thanks for taking the time I hope this was helpful I hope you have a you know better idea of if you should use Lang chain I'll see you guys in the next one