Long-term Memory: LangMem SDK Conceptual Guide

Key Takeaways

hi Will here this week we're excited to launch the Lang graph SDK a library of utilities to help you build agents that learn and adopt as they go as a part of the release we wanted to review the highlevel concepts driving the development of the library as well as share a mental framework you can use to more effectively leverage memory to build more reliable and personalized agents memory is a broad concept and we believe that rather than building a generic general purpose memory layer most successful AI applications today will benefit from leveraging domain specificity in using memory to accomplish concrete goals if you think about the specific capabilities and knowledge your agent needs in order to successfully accomplish its tasks you can typically map this on to distinct memory types one analogy I find useful is thinking about how computers represent information computers typically distinguish between data which is static or dynamic information and code which are instructions and procedures to follow we can think of agent memory in similar terms semantic memory acts as the data store it stores important knowledge relationships entities and other information that is useful to ground the responses the agent might have to the user procedural memory on the other hand acts more similar to code it encodes the rules behaviors and other information needed for the agent to know how to respond and how to act in a given situation episodic memor sit somewhere between the two it stores information about past events in the form of few shot examples they instruct the agent how to respond in a manner similar to procedural memory based on prior attempts they also encode a lot of information about what the agent did in the past similar to semantic memory taken together the semantic procedural and episodic memories tell the agent both what and how to act in a given situation if implemented effectively these combine with the language models underlying reasoning abilities and your own code to accomplish whatever task you might need in your application let's go step further in grounding each of these memory types with examples starting with semantic memory semantic memory includes facts knowledge and their relationships into some sort of underlying storage this is typically what most people think of when they think of long-term memory people want to jump straight to knowledge graphs or vector databases and both of these are valid ways to store information in our experience we've seen two representations to be the most common Collections and profiles collections referred as storing memories as distinct records in some sort of database in the context of Lang and Lang graph this is typically stored in Lang graph's long-term memory storage layer the base store new memories are formed by extracting new records in the database each memory can either be an unstructured string or take on some level of structure based on the schema you provide these schemas can model out the specific domain of your application if you find that useful memories are Consolidated and synthesized either by taking existing memories and using those to inform and contextualize new memories that are inserted or by updating existing memories in that database these collections can be unbounded so whenever you need to recall them they are typically searched over using Vector search full text search or some comination of the two below is an example in Lang M we've created a semantic memory manager we've passed in a list of messages as a conversation and it's extracted the following memories we've learned that the user prefers Lex short for Alex and appreciate the casual and witty communication Style can also learn specific information about lacks such as he's proficient in Python Programming and is a competitive speed cuber there's also some additional information about personality and other types of traits that it's saved and will potentially recall in order to know what the user likes to do the second way that we see many people representing semantic memory is through profiles you can think of a profile as a way of compressing all the information into particular schema or into a one haager about a particular user these are most common in user phasing chat Bots where you have have specific information you know is relevant to your task such as user's name age friends and other key details new memories are formed by continuously updating this single memory representation doing so helps you reduce the amount of irrelevant content that's extracted and stored for further use however it also means that more information will be lost if there's something that you forgot to model in the profile this type of information won't be retained for future conversations another pragmatic use case for profiles is it's easy to Rend under in a UI if you have a user facing chatbot it's easy to show what the agent knows about that particular user and the user can easily collaborate on the memory if the agent assumes that you prefer your name is Tom and you actually prefer to go by Thomas you can directly modify this memory and the agent will immediately know it the second type of memory is procedural memory this tells the agent how to respond in our experience this is typically taken the form of a subset of the system prompt in your llm this prompt fragment can include the user preference es the core agent behavior and other types of conditionals and rules that the agent should know in order to accomplish its task a common example of this is the response style if you go to chat GPT today and you ask it to draft a blog post about memory for instance the output is not going to look like the style that you probably prefer rather than having to manually write the instructions in the system prompt and rather than having to restart from scratch on every new conversation it should learn over time that you prefer a particular voice a simpler example would be having it remember that you prefer to write with proper types and typescript rather than raw JavaScript or that you prefer to be writing things for front end development rather than backend or it can even infer that since you the user are now an expert in AI it doesn't have to rehash all of the one1 level content before jumping into the details for advanced concepts it can learn the level of detail appropriate to responding to you langum exposes this through prompt optimization this is meant to be learning online from feedback or from natural conversational examples here's an existing example we go into more detail in our walk through on how to use the prompt optimizers in langum in a separate video the final memory type is episodic memory these store information from past experiences this encodes both how the agent can respond and what is done in the past this usually relies on feedback either from an explicit user signal such as a thumbs up or from Auto evaluation by looking at the interaction and recognizing whenever the agent has done something successfully it can save the trajectory of the input as well as the expected output to the store in future rounds it can then fetch these semantically based on what is similar and Achieve similarly good behavior and avoid diverging on hard examples so in conclusion memory is an exciting topic and the core component in building adapting personalized and self-improving agents for the foreseeable future we believe most successful applications of memory will be application specific and that Engineers that start from a point of thinking about the type of information the agent needs to learn in order to know what and how to act will find the most success while this video is purely conceptual we encourage you to check out the Lang docs as well as our other videos instructing how to apply Lang and these memory Concepts in your agents thank you again and see you next time