"MemGPT" Masterclass

Key Takeaways

in this mgpt crash course you will learn from the creators of mgpt Vivian and Charles on five important topics one brief overview of mgpt code base two easiest way to run mgpt as a Discord bot three mgpt CLI setup and basic chat demo four how to use mgpt to chat with documents and finally five running mgpt with local llms without open AI okay so to give you a very high level idea of how the code is basically the main mgbt uh code Bas is inside agent. py and everything is happening effectively inside async agent and what happens is you have the step function and every time the user sends a message this is what gets called so there's a call that goes to step and you can see in the comments here there's basically a few different steps you first append the user message to the internal message queue then you run it through open AI API or like a local language funnel API to get back a response so you have the messages you pass it the language model you get the assistant reply and another thing we'll go over is if you want to add your own functions to mgpt so for example you want to add the functionality for mgpt to use a new tool you basically need to touch the code in only a few places um you need to update the available functions bank here so you can see here that there's actually a lot of functions here many more than mgpt will run by default some of these are aliases for the same function so like conversation search is just the same thing as recall memory Search is with a different name but to give you an example of a function that you could add we have this function here called message Chat GPT this isn't in the paper this isn't enabled by default but this is a function that allows mgpt itself to call chat gbt or call gbt 3.5 basically and when we give m g PT this function it's we tell it that it can use uh it can basically use chat BT to answer questions for it so if it gets stuck it can send a request to chat gbt and to actually Implement that in the agent class itself you add a new function so we can see here the actual call to message Chat GPT is itself like a little um asynchronous API call and it has some concatenated string here that you can kind of see the preview It Starts With Ur helpful assistant keep your response is short something like that passes the message that um mgpt wanted to message Chat GPT with and yeah that's the basic way you would add in your functions so I know some people on GitHub were asking for adding functions to allow mgbt to write to a Word document for example this is how you would implement it You' add a new function that does that and another thing that think some people are interested in is how memory works so in the base repo memory is just held inside python objects um it's still persistent so when you type save it will save it out to a file but if you want to actually Implement a database so you don't have to do saves but memory is sync to a database what you would have to do is you basically create a new archival memory subass or a new recall memory subass depending on what which one you want to replace you can do both and you would call it something like database archival memory and you all you really do is you just replace the insert you replace the insert and the search um with code that's specific to inserting and searching over your database and that's pretty much it and then you just it's pretty straightforward to hook it in um or to instant when you create a new MPG class to use these new memory function are these new memory classes that you created and then make sure uh after you create a new memory function you update the persist you create a new persistence manager which just includes like the classes for recall memory and archive whole memory and then you update those classes handm GPT uh the new persistence manager all right so the easiest way to get started with mgpt is actually with the Discord bot um this doesn't involve you installing or downloading any code you can just use it directly as long as you have a Discord account and then open a API key so the first thing you want to do I guess if you just tag mgpt bot it'll give you some instructions on how to set things up uh the first thing you want to do is set up a profile and yeah this is basically your user profile so what mgpt knows about you so may me say onless Charles on a PhD student uh UC see brly oops uh I really like for long walks I create time around the lake around the lake sure okay St this is stored so if you create any future Bots with mgbt uh the Discord bot it'll use this information to seed the bot then we want to create our we want to link our opening I key yeah so okay yeah so basically you just paste to your openi key from openi it has to have gp4 access um we have some more instructions on how to make sure you have that on the Discord but at this point now we can uh create a bot actually an important thing to do here is to make sure that in your settings um you go to privacy settings and make sure that you're allowing direct messages from uh other members in the server which is mgpt blot yeah so once you create now you get to basically um set up your Bot so you can either set it up with some default personas that we have there's a very basic one there's one that has a little bit more text in it and then you can also just write your own but for the purposes of this we'll use one of the defaults and now you can see it's just confirming that everything is ready to go um yeah we have our our own profile here and then we also have the Persona that was that's our default you can always just change this make it your own and once we hit confirm see that says you successfully created your AI chatbot they'll message you soon and at this point you hop over to your DMs all right so we see that we just got a new message from mtpt bot and yeah it's basically saying hey nice to meet you um it's using the information that we gave it so it's said uh I heard you enjoy long walks by the lake and there's a few things we can do here now so we can obviously just chat with it so we could say something like hey did you know today was my birthday don't forget to celebrate it next time something like this so This should definitely trigger like a memory event right um so you can see that it's cooking yeah so you can see that M GPT bot immediately identifies that this is some important information we need to store so it's going to stick in memory and we can always actually check the status of our memory using the memory command so if we run memory it's going to dump what's inside the current bod's memory and you see it just added a new line for our birthday um there's some other things we can do too uh you'll notice that in the chat we can see both like the inner thoughts of the bot as well as the message and the functions it's running so if you don't want to see some of these things like you just want to kind of see the messages that are running we can always toggle the functions off so you can do type functions it will oh I guess you might you might have to do this twice but yeah you can turn turn it off and that means you won't see the function messages anymore and similarly if you don't want to see the inner thoughts you can just turn off you have to tole it I guess on extra time but you can turn it off and now let's say something else what was the last thing we said to the bot said it was our birthday oh happy birthday so we could change it my say it's Vivian instead of Charles yeah thank you for the birthday wishes oops wishes actually my name is Vian on Charles sure where you got that info in this chat log we're only going to see final messages that come out we won't see the function calls we won't see the in inner thoughts so it'll make the chat a lot cleaner but you obviously won't really be able to see what's happening under the hood and see that all all we're seeing in the chat now is hey I'm sorry about sorry for the confusion yeah and then we can D the memory all right and yeah we see that behind the scenes it updated the memory and if at any point you want to kind of start over you can always like create a new bot and if you create a new bot then the old bot will still be saved and you can just go back to it with select so right now we just have these IDs we're going to improve this shortly so you can kind of identify which bot is which but it's chronological order so basically the last bot or the last number on the list is the most recent bot you created and the further you go up it's like um older Bots yeah so you see we the most recent one is right here and if we ever want to switch to a different one we can just uh select it yeah okay so to get mgpt to run on the command line we first install the P mgpt package you can run mgpt as a simple chatbot uh we can configure it to use gp4 use this Persona um and have this built-in CS PhD uh user profile and then we'll insert nothing into M gpt's archival memory because we're using it as a chatbot and now we can start conversating with mgpt and now it says oh hello Chad welcome um so I can correct mgpt I can say hey my name is viven not Tad and then what mgpt do what mgpt will do is take this uh response and then it will update its internal memory with uh replacing first name being Chad to first name being Vivian and then it'll say oh I'm sorry for calling you Chad and then after this what I can do is I can exit and when I exit it'll save a transcript of this chat or a checkpoint to this uh folder here and when I run mgpt again it'll say oh we found a configuration file would you like to load it in yes and then it'll say we also found in a we also found a checkpoint would you like to load that in if you say yes it'll load in the checkpoint um from the transcript of the chat that we just saved and now mgbt will still have within its memory the fact that my name is Vivien and not Chad so we can test that by asking it what is my name so actually if we just look at its internal monologue it says understanding Vivian's perspective on this so it already knows my name is Vivian here I'll have mgbt use Doc and then it'll ask us what we would like to use so I have a custom Persona here called viian uh you can make your own by adding text files to the directory specified here it'll ask you if you want to preload anything into M gpt's archival memory and I'll say yes I want to load a folder or file and let's see maybe I want to upload mgpts paper and it'll ask us if we want to compute embeddings we'll say yes and then it will'll compute them embeddings over one file and now it'll be loaded into mpt's archival memory and then we can start asking mgpt hey can you tell me what a virtual context is P here arall memory then hit enter so first it'll keep track of this task and then it'll start querying archival memory with what a virtual context is and the definition of a virtual context appears inside the mgpt paper that I fed to mgpt and it says oh we got some results about what a Rital context is it'll digest those results what is your answer looks like it got cut off a little early forgot to send a message so we'll ask it to send a message rate limited okay and then now it'll outl an answer saying oh the concept of ritual context refers to a concept utilized by a models like myself and this is like pulling stuff that uh it got from the paper text you can pause and read its answer but um I guess this demo goes to show that you know if mgpt is it's like a natural language interface so you can be very explicit with it so if it's not finding things in it's archival memory you can tell it to search your archival memory for an answer to my question if it gets to send you a message you can say what was the answer you wanted you wanted to send me yeah so now we're going to go through a brief example of how you can run mgbt with local language models so not routing to but instead of routing to your own model mod that you're hosting somewhere so for this example we're using um Tex and webui and we're going to use the uh dolphin 2.1 mistro model so what you're looking at right now is the kind of model loading screen for webui you don't have to use webui for this this is just uh what we're using as an example because it's quite popular um but yeah we we already downloaded and installed web UI and then we also already downloaded the model and yeah so that's why we can see it we're just going to U make sure that we have enough CH space here hey load now so loading might take a few minutes or longer depending on how large of a model you're loading um and yeah we're running this on a server and we forwarded the port 7861 for the web application but the actual streaming or not sorry not the streaming the actual API for the model is we also fored that part that's on 500 or sorry on 5,000 yeah so depending on what backend you're using if you're using it on like running it on runp pod or something these instructions will be different but the key idea is to basically just host the model somewhere as a on a web server and once we've done that then so one once we've done that all we need to do to uh make it so that mgpt will launch request this back end instead of open AI we just need to do two things we need to set the um opening API base variable to the basically just where the web server is located in our case we forwarded it to Local Host on Port 5000 and then the second thing we need to do is set backend type to web UI so yeah and then just to demonstrate that or make extra clear that we're not using open a anymore we're going to turn off the API key that we had earlier so it's no longer visible to mgbt beautiful yeah so you can see you can use these much smaller models like this so this is just a 7 billion parameter model and you still can get some reasonable um function usage so things don't completely break this is still highly experimental this is something we just added and we're working working pretty hard to make this a lot more stable so yeah we welcome you to try it out and if you try it out uh please let us know like what issues you run into what models you think work best and the GitHub issue that we or the get up discussion that we have open