Memcached Tutorial for Beginners

Key Takeaways

welcome to this comprehensive video course on MIM cached an inmemory caching system known for its Simplicity and Effectiveness in reducing database load in this course Hussein will teach you about MC's architecture and design choices and he'll conclude with a Hands-On demo using Docker and telnet get ready to deepen your understanding of this transient cache system mimc D is a simple inmemory key e Value Store written in c h it was originally I think written in Pearl and then Rewritten in C this is what back in 2003 so it's been a while and it has been popular with companies such as Facebook Netflix Wikipedia Facebook I think pushed it to its limit you know and the most uh and the reason it's why popular is because of its Simplicity and we're going to talk about that I know we throw the word simple a lot these days but MIM casd is truly simple I mean if you're looking for advanced features it's not here it was designed to be simple solve the problems of the web back in 2003 which is we want to help alleviate the queries to the database we're we're you know the databases are taking a hit you know so let's cash things although I usually do not agree with having a cash to solve a slow query because I think personally I think it's a cop out you know to to just add a cash when you have a slow query you have to understand why it's slow you have to understand why it exactly taking time why there are a lot of logic reads and how to minimize it and that's another story for another day but sometimes you need a cash of course right and then MIM casd was born right of course there are alternatives such as redus I made a video about it but this this video is to is a crash course of mhd we're going to dive into the agenda here you're watching this on YouTube there will be uh chapters where you can jump into the interesting part of the things but it's an inmemory database we're going to talk about memory management you might say why well we're going to find out memory management is not easy at all you know it's not just like hey just throw things in memory and then read it it's a little bit more complicated than that going talk about the Ario the least recently used uh which was designed to avoid growing the memory of this instance unlimitedly right CU you have to have a some sort of a a mechanism to evi all entries that has been never used that's why you cannot really rely on MIM cash D to have a value always there it was never the goal of this cash right unlike Reddit right is if you if you store something is going to be there and you told it to be there forever it's going to stay forever it will make sure to stay forever MIM casd does not guarantee that and you can argue that this is actually a feature and you can argue that this is something you don't want you know so tread lightly thread we're going to talk about the threading model cuz you have to have multiple threads if you want to serve a lot a lot a lot of clients with a lot a lot a lot of TCP connection connected to this thing read and WR we're going to go through examples of a read example for the write and okay kind of uh open the hood and look what is inside this beautiful thing locking model obviously uh two people trying to write in the same item it's not as advanced as acid obviously where you have isolation levels now it's a serialized model where we try not to have two people read the thing item at the same time or write uh the same item at the same time so locking and we're going to talk about the old model and the new model that's where you really try to understand what how things are built is completely different from the way we explain it distributed cach I know they say it's a distributed cach but I kind of don't like to say that because in itself MIM casd is not distributed it's just when you spin up a m casd instance three M casd instance they don't know about each other and they will never be right the client is responsible for the distribution so I kind of reluctant to say it's a distributed cash I know people call it a distributed cach I don't like to do that but hey it is a distributed cache if you if you if you put the distribution at the client side and I think this is part of the beautiful simple design they they on purpose they didn't make it distributed to make it simple and then we're going to go through a demo we're going to use Docker because you can spin up a lot of instances in in Docker really easily let's do that so in memory key value St what's that really uh we're going to talk about some terminology here specific to mkd an item that's what they call it an item is really what consist of a key and a value a key is usually should be unique right and a value could be literally anything a key has to be a string and it maxed out at 250 character you can see the limits right that's why I think uh redis kind of won the cash game when it comes to this thing because MIM Cas has a lot of limits and uh this kind of uh you know crippled some people from using this cache because of these limits right but you can Ario also the Simplicity of this design and if you can work around the the design to use MIM casd it it's actually pretty nice right when when they key as a strength they did that for a Simplicity reason to right if if you support like dates uh or or I don't know like blobs as Keys then Things become really complicated and there shows in the architecture and if there is a bug it's really hard to track down right and the value can be any type h it's by default one megabyte again another limitation in MIM casd yeah you can see that I I talk about limitation but these limitations technically to me I I see them as features you know because they didn't claim to be like the best in the world they said hey we are designed to be simple and I appreciate and I completely love that you know when you say I want to build something simple the simple thing has limitations right the simple thing will by Design have limitation when you look at the big picture right it's not going to have like tons of features right so yeah you can configure this to to increase it but again it's not really a good idea all the time uh Keys have exploration date ttls right time to Liv and uh even that don't rely on that right even if you put like a key that has like a 1 hour and your memory is filled the lru will can kick in and if you never use that key it's going to get affected and they tell you that right hey mkd is a transad we're not going to make sure that it's actually persisted it's not supposed to be that right again always go back to the the requirements here they never meant right for this to be a persisted cash forever right you don't rely on that you kind of use it to help you avoid expensive queries but yeah be ready at any time that this value is not going to be there everything is stored in memory right that's why it done cash MIM Cas D again don't confuse MIM Cas D with MIM Cas DB that's a completely different project and I think it's abandoned since 2019 2009 right but yeah m casd is still going and Facebook and I think Mark Zucker give a presentation about M casd as well at some point how they tuned it to its maximal values let's talk about memory management here's the memory right when you allocate items you know when you say hey I want to allocate an array and iate an integer and I want tocate a here's a block of M you know these items that you allocate and even you new program today they going to go in random places now they yes the process has a dedicated memory area but when you allocate these things go random so the grain is allocated they're going to go to random places right yeah initially they might be consequent but as you uh remove and free items you're going to end up with these gaps so you might say what's wrong with these gaps the problem is like this this is called the fragmentation what's wrong with fragmentation well you're going to have like a little bit of few bytes here few bytes here few bytes here few here but if you want like a big bulk of memory location consent like you want one Meg you have one Meg but it's fragmented and guess what if it's fragmented you cannot use it you cannot just allocate here oh my part and then part here and this part here I think that the operating system allows you to do that maybe but then it will it will crash left and right to collect what you have right we had the same problem with hard drive I guess right back in the days with fragmentation where the the seek you know the the needle has to to go multiple places to fit your files because it's a circle right this this desk and has the if if you store a file and you start editing the file the files will go to multiple sectors and to read that file back you have to go sector one sector two this was supposed to be like a rotating disc I failed miserably but you get point right so memory fragmentation is bad we try to avoid it right new items can can no longer fit so what what did they do so me casd at least what they did is they allocate pages instead even if they don't use it they say hey when you start I look at the whole page one Meg again that's that's the design right that that's the reason why we have one value up to one megabyte you cannot go beyond that so they say hey let's start with one Meg so if they allocate they allocate at one Meg the whole thing they don't use it technically to the operating system you the cash has used 1 Megabyte it doesn't know it right that it's not actively using it that client us connecting to M casd we're not probably we're using like part of that memory does that make sense but this avoids frag M mentation right cuzz now all of this just one big page with a lot of empty space right but it is elated and then there is this idea of chunks right so the pages are broken down into equal size of they call chunk so um keep these in mind the terminology in mind the chunk is a fix size and what determines the chunk size is actually something called a slab class right and the slab is think of a slab every time I hear the word slab I remember Dark Souls you know it's a video game where you have the last item that you require in order to upgrade your weapon it's called a titanite slab where it's say like really large rock that you use in order to upgrade your weapon it's a it's it's basically a a a big thing a slab of meat they say right it's just a big thing that's what it means so this idea of slab and slab classes will always show up up here so it's like a b big portion of memory a slab class is what define the chunk size right so there is there will be a slab class of 40 bytes there will be a slab class of a one Meg so the chunk sizes will be 44 bytes right and the chunk sizes for a slab class of that type is one Meg so we're going to show an example to cons to to show that of stuff and the pages consist of fixed chunk size right items are in chunks here's the a very important item so your item will be stored in the chunk your chunk size could be let's say uh 100 BYT right if it's 100 BYT and your item which includes the key and the value is less than 100 you're going to include the whole chunk right so if it's 90 byte you lost 10 byte within that chunk nothing to do about it sorry right that's one limitation here so there there will be a free space a tiny free space in the chunk chunk and obviously each slab class has a fixed chunk size so that's how they are determined it's going to be clear as we go through them obviously avoid memory fragmentation here's an example right so here we have a slab class with a chunk size of 72 at subass one and slab class 43 for example the chunk size is one Meg so you have in a single page right so slap class have multiple pages and sometimes they call them also the is called slabs the word slab in the documentation is so overloaded and I've seen people use it one over the other so I avoid using it the word slap so a slap class and there are pages right so this page in this case we said one Meg right and the chunks are 72 divided that means we have one 14563 chunks per page but each chunk is 72 byte right so if you have an item around 72 byte fits nicely in this right but if your item is larger let's say 900k then oh it doesn't fit this slab class so we need to let's let's find out what is the subass for this item oh subass 42 because the closest one Mig and guess what the one the one Mig class slab class has this entire page is one chunk right so this is really important to understand here right so that's how they allocate memories this we're looking at the internal architecture of M casd here right let's go example new item 40 byte 40 byte the closest thing is this guy right slab class one let's allocate memory and then Boop no we don't allocate memory that this memory is already allocated right we just store the item right in this chunk and then we start adding pointers and stuff stuff like that here so now our item is right here we're going to talk about the hash table and stuff like that but this is just again memory management let's say I have a new item 900k oh this fits right here so one big chunk in one page right so that's interesting let's say another new item 40 byte but guess what slab slab class one because that's the what the chunk size is fits it nicely but guess what we have two pages they're all completely full so I cannot insert this what do we do create a brand new page put that thing here does that make sense so we allocate the new page and when you go to the demo we're going to see all this stuff right we're going to do a stats and see like oh how number of pages allocated is this right so it's just I I absolutely love how they did this it's interesting obviously doesn't doesn't have limitation of course but you're going to see that depends on the sizes of the item you say and once you really understand how things work you you can architect your application specifically the back end here frontend doesn't really talk to M casd at all right the back end here can be architected so that you can choose the perfect items right to fit this entire thing right you're not going to just choose hophazardly right that's how you know your craft effectively all right let's get to the meat lru least recently used you know the main problem with memory is it's limited you know and even if MIM casd allocated certain amount of memory if you store a lot a lot a lot of keys even with good expiration date memory can get full what do you do do you block new inserts I would say that's that's one feature that you can add I suppose but but mimk they don't allow you to do it they won't let you go to that state you if the memory is be about to be full then anything that hasn't been used for a very long time they will release it that's another reason where REM Cas D is a transient memory it do not rely on a key even if you said the expire for an hour do not rely to that key to be to be there in an hour it can go any any time right that's another limitation that's another feature right I say limitation and a feature at the same time because it is it is a feature right and it's to some people it's a limitation and how do they do that right they use something called the link List have you ever heard about this before and 20 years ago 21 years ago uh in the University I took a course and the Cs CS 101 they talked about link list and that is pretty much the the only time in my entire professional career I ever used a link list that's probably I'm not saying that's just the case with my the application I wrote is all high level languages I never had to write a database or a m cache you know so I never used the link list right so doesn't doesn't mean that it's it's a useless structure but it is it's a important data structure why cu the least recently used is a link list there's a head and there's a tail and every item is linked to each other so if all these items that you add are the in this architecture right they are in the link class and there is every slap class has its own lru right so if I for example access an item that happens to be in the tail it will be popped and go back to the head so there is a cost to accessing an item there is a cost there is a cost of removing this chain put it that back pull the head to this pull this guy to this guy pull this guy to this guy that's how you do linkless right so every time you access an item it goes back to the head so items that are not used they will automatically be pushed down to the tail and if the memory is Out Of Reach basically these items will be removed from the tail but also another thing with with this link list is like with threads if you with which MD is is multi-threaded app how can you have multiple threads read at the same items right you can't you have to lock this structure you know if people who done multi-threading you have to you have to lock it right and the moment you do locking if you if you know about something about databases which I a course on database engineering check out check it out it's actually right here database. husin as.com I talk about all this stuff you know in details in fundamental details you know so don't expect like SQL syntaxes in my courses not like that I talk about fundamentals which which should then build up and see how the client is built out right but yeah locks is a very critical concept here you have to lock it to avoid this you know mutation you know corruption right but yeah it's a cost and there's an a you CLW crawler and a demon that does the cash inection from the tail right and again every time it kicks out have to lock and then if it's locked people cannot read people threads cannot read and if threads cannot read latency right block slow right all of this you got to understand when things happen this is why and I'm going to share my opinion about the lru right and I think this is a good time by the way there is an L cache per slab class I think I mentioned that so the one Meg slab class right which has like pages of one Meg and the chunk sizes of one Meg has its own laru and each other sub slab class has its own L by the way I don't make I'm not making any of this up I'm I had to read frankly maybe 20 different documents to collect this information and and kind of present it in a summarized manner here right because there is no one dog to explain all that unfortunately right it's it's not incomplete unfortunately that's what I noticed and outdated so my opinion about the lru in my in my personal humble opinion is I wish MIM casd actually disa this by default you know L is a feature right and they the reason they added it because memory is limited especially back in 2003 when they first built this thing memory was so scarce or scarce is it scarce or scarce scarce right it's very limited and when you do that you don't want to run out of memory right so if you allocate certain amount of memory for M casd it can easily run out right if you have a lot of keys so how do you manage that they say hey we're going to remove we're going to build an laru least recently items K get kicked out from my memory that's a fine but I wish they disabled that by default or give us an option to disable it because the overhead of managing AIO and you can see from the papers I'm going to reference is so large the locks that they have to maintain slows down throughput right and comp and complicate the application so I think they stuck whoever built this 20 in 2003 MIM casd Brad Fitzpatrick who is the original developer of MIM casd he built this for his website live Journal you know I wish he disabled this by default I really wish because his original design is so simple and so elegant I absolutely love it you know you build something so simple with its featur stripped there are no much features U made it not simple unfortunately yeah cool have this a feature but disabled by by default or have an option to disable I don't know if there's an option to disable maybe there is but hey I want to take the responsibility as a client right if I'm going to allocate certain amount of memory cuz I'm responsible I'm going to give uh MIM cash D 5 gig 10 gig and my application is smart enough to know to set expiry date right for this item and yeah if I'm going to get errors if it's failed out it's on me I want to delete an items I want to do this management this way for 95% of the users who want simple things they're going to get it l are you in my personal opinion again this is just my personal opinion you can agree or disagree I think you should this this should have been disabled because now they they created a new lru which is like has hot and warm and cold and and they move stuff around because they have a lot of problems with lru like moving stuff around all the time is so expensive you know so it it has a cost so let us just how about eh give me an option not to use it and go back to a simple model of course I I don't I don't mind if two items two users trying to access the same two threads trying to access the same item at the same time let them be serialized that's fine right but lru as a whole thing I think it's a it's just to me over engineering that's just my opinion you can disagree here is how it looks like by the way aot you in the big picture again this is all drawings I made it uh I could be wrong in small details because I don't I don't know the actual architecture so this is I derive this from reading the source code and the doc so this is how it looks like so this is this is where we talk about the pages right and the chunks so the chunks or the items is what being lru right so the head is right here and this is linked to this item this is link to this item this link to this item this this is the tail so this is how it looks like every item here is actually linked to the next to the one next to it right this is think this of this as a snapshot after many many usages right gets and red that so so things will move to the head and the tail obviously I didn't draw every particular thing because it's going to be a mess of a drawing but you get the point right that's that that's how the L you and you can see how complex things get so let's talk about threading so this is one of my favorite Parts I know uh I absolutely have um I absolutely love networking and if you're interested I have a networking course and and this part is is all about sockets connections the way listeners work the way the TCP connection works I talk this about this in detail in my network course if you're interested go to network. hus n.com learn more about that again network.has nasa.com this URL redirected immediately to UD me with the latest coupon applied so uh you're going to get a discount and you're going to be supporting this channel this work thank you so much there's here's the threading model for uh for mam casd right because it accepts clients it has to have networking right so what they do is they listen on a TCP Port right so that means they support transmission control protocol that's the native transport that they support they also support UDP which I didn't mention here but UDP has been now disabled by default because of an attack that happened 4 years ago 2018 uh reflection attack actually right with with MIM casd public servers so it was the cloud flare actually reported that so UDP has been disabled by default but yeah you can use it if you want but yeah let's stick with TCP right now TCP Port 11211 and there is a listener thread so one spin up listener one thread that spins up it listen to Port 11211 so that creates a socket right in the operating system speak right and that basically creates its own accept Q it's on syn Q this is how the application start accepting connection right so everything every single connection that is happening The Listener thread will accept it so there is a loop infinite Loop here literally all application has this Loop where it's constantly accepting uh connections one thread right so all the connections goes to this thread so once it accept the connection it gets the file descriptor we call it right which actually represents the connection and now what what mimk does is spins up a new thread gives that file descriptor to that thread now if a stream of data if a request to get a key was sent to this connection it will the operating system knows to send it to this thread well technically what happens is the thread pulls the descriptor right this is now this thre is responsible for this connection This Thread is responsible for this connection This Thread is responsible for this connection so you can see that now this model just blows up right if a one connection PA of thread if you have so many connections you can R run out of threads right or that kind also blows your memory and CPU so be careful with that as well I think they put a limit number of connections M casd I might be wrong there but yeah so this is basically explains all of that now the moment you have threading now the beauty here is you don't have bottleneck right if you have one thread that is responsible for all the connections and listening you you will be blocked right one user will send a key and then another user will send a key they won't be served right they have to be serialized because there's one thread actually executing them one by one right but here if one user executes a sends a key request to get a key and this guy want to write they can happen at the same time right this thread will read it and this thread will read it they are different processing this could be in a core this could be in a completely different core that could be also possible versus it's one thread then becomes really a a problem so we had to go with multi- threads what's the problem with that well the problem is these threads will try to access what we call the laru and the items and the memory so everything is shared between all these threads but you can't have two threads right to the same location that's a problem that's why the original design had one Global lock it was serialized so in this case yeah the threads kind of helped with the connection but but you were serialized at the Locking model so nobody can even access two different items has nothing to do with each other they were serious they were locked so one thread has to be served after the other they they fixed that they completely red that now it's a pair item log so if two threads try to access the same item then there will be SSE that's good that's okay I'm okay with that right but yeah if I am accessing item one key number one and then another thre access key number two at the same time they should be served at the same time what there is no reason for locking and the only reason we lock is because we want to update the lru again so there is so much stuff that comes back always to the lru it's like oh really we did we really need an lru why what if we disabled by default right okay that's just me let's go through an example read and this is something we never talked about here which is the hash table if you think about it if you have a key how do you actually find where this key lives right if you think about it you need hash tables so what you do and I talked about hash table in my YouTube channel uh look up hashing and consistent hashing I talk about details hash table is nothing but an associative array it's really just an array and the beauty of an array is if you have an array right let's talk about arrays a little bit if you have an array from an array has to be consecutive if you allocate an array of a thousand elements accessing element number seven access an element number 10243 is biger of one is fast because you know the index once you know the index and you know the head of the array you add the address to the index voila you have the address and the memory of the CPU can immediately go to that location that's the beauty you have an index with hash tables you don't have an index you have a key the trick is to convert the key back to an index that is all what it is a hash table nothing fancy it's just an array so we take that what we do is the do a hash on the key right let's say I'm going to R test key right and then do a hash and then do modul n where n is the size of this array or the hash table right and then you're going to get a value between zero and N minus one I guess right so now okay let's go point it and that's big of one plus the cast of The Hash right now you got here found it now what you do is you're going to get a pointer which takes you to the the page on that specific slab class for that item which is happened to be D in this case that's how a read works so it's a big off one you can argue that this is one read and this is a second read yeah I suppose that works too right I think the the new model have kind of two hash tables if I'm mistaken but I couldn't find detail docs about this so I explain this so I apologies if this is a little bit out of date but this gives you the idea here the new two hashes I think uh they were provided to provide a pair item lock and obviously what happened here is this is the lru you access the item the D is pushed to the head right so now you have a b c d a is now the least recently used item is in the tail and a points to b b points to C C points to D and obviously it's a revers length as well so D point to c c points to B P point to a right so that's how they allot you actually what you if you think about it the the pointers are right here in the item itself right but I drew it this way for Simplicity otherwise it's not readable at all read two this is another example for read I'm going to read Buzz hash the buzz get the N whoa get the item Boop get a c nice now when we get a c the C is pushed to the Head D is slightly pushed and then obviously the allario is updated and that's another lock right you have to do a lock to do that so if buzz and whatever the value before a test was read at the same time they are serialized at the lru level if they belong to the same slab class again this might have change with the new architecture they changed that a little bit so I think that you can you can play with that a little bit but again to update the L you have to you have to kind of acquire a lock so you're going to be serialized right here yeah let's go through a write I'm going to write key new of a value with 44 whopping bites let's do that well to write we need to obviously find the hash right where to write it hash module in get that puppy find where to write it oh happen to be an empty location sure that's good right and now you have questions what if what if I happen to have something that is already written you can you can have collisions and we're going to talk about collisions in a minute right it's a problem of hash tables hash tables are fun and good but the moment they you run into collisions and you want to resize it it falls apart but now I create a new pointer this pointer now I need to allocate a slab class not slab class I need to allocate a a chunk where going to put my item in and that chunk goes into a specific slab class well 44 bytes pick a a slab class right and even the slab classes guys by the way it's not really fixed you can play with those the other configuration called the Chun the chunk Factor growth size I'm not going to mention it here it's just going to make the course a little bit longer but you you get the point there are so many tweaking you can play with right tweak these chunk sizes but yeah I look at a new memory location in this specific page in an empty page in a fit a fitting chunk right because you want to pick a chunk that is almost fits right in the chunk right not too small obviously cannot be larger than the chunk SI I have to fit right into it right but then that's the the that's what the what MIM casy does all right let's spice things up let's say I'm going to write a key called Nanny which is a new key I don't have it before and value 44 but it happened to clash with another existing key because that's hashes always do that right so when you do that you hash Nan and happen to be fitting on a entry that already have a pointer what do we do do we overwrite it no what they did is this is called a bucket by the way right they add more item to the bucket you know we have one item let's call it I don't know test or something all right and then we have Nani which fits right in the same bucket what we do is just we make it into a chain this chain of buckets right actually one bucket with two items I don't know right whatever the terminology doesn't really matter you just read to understand let's turn back the laser here and yeah we're going to add it here and then just do the pointer and do the do your thing now obviously we need to talk about a collision what happen to Collision I want to read the keani right go here hash it obviously go here oh we have two which one Ah that's the cost you have to go one by one through all of them right why because now we have a hash you don't know one of which one of these are actually Nanny what you do is read the first one right check it compare the key oh because if you go to the item you're going to find the actual key right that's stored here so you're going to find it and say oh that's not 90 that's something else right let's go through the bucket go to the next one there you go that's my item so here is a completely different paper that you can write here this P people take phds in this stuff by the way guys you know this is called the the the scale factor you know mimc D measures this growth and if it's too much based on a certain percentage if you're overloading then then you're going to still see performance problems right reading a key is going to have to go through multiple reads to find the actual key versus if it goes right here hey the key is right here of course there's one entry it has to be it right but if there's multiple yeah then it's a problem right I mean you can you can think about it you can argue that you can hash a key that happened to get to a value that is not there so technically you have to read it and compare because your key might not exist but it happened to Hash to a value that does right so you have to read it so there is a cost to reading so that's the problem of hash table so and if that's the case then they do a hash resize and boy when you rehash your table they have to shift everything around and I believe this is when they use the consisting hashing which is this ring concept which I talked about in another video and that just gets really complicated right because they know now the moment you resized your hash table you need to move stuff around because Nanny will not be this index number one anymore it going to be index number 7700 something like that right 1700 is not a number I think I'm going to skip this because we talked about locking in a minute we talk about thre threads and then accessing the lru and how it was a global lock and then it changed to a pair item lock and then still we have ref counting you know every time you read an item you increase the ref count you know and when you release it you decrement the rount this is for so the garbage collection can the garbage collection is written in C there's no garbage collector but the the the the ephemeral application Level garbage collection when Elio kick in can remove the item because you cannot just remove the item if if someone is referencing it that's the definition of meem memory leaks right all right let's talk about distributed cach and how it's uh MIM casd is actually not a distributed cash in my opinion mimc D servers when you spin up a mimc d server mcash D server they are completely isolated you cannot link a server to another server there is no mechanism to do that right when you spin up a mimc d server it's a mimc d server it doesn't talk to another servers and I absolutely love this design how simple and elegant this is put the responsibility if you want to do distributed well the apis at the client side has to do that and that's what we're going to show in the in the code section where we're going to write our own uh we're going to use a nodejs application to do that of course we're going to use also tnet to connect to that and write stuff right but we're going to go through all this stuff now but yeah what happens here is the client actually knows about all the servers it has knowledge so the client site actually does the distribution right so it's like okay key number one go here key number two go here key number three go here so there's a hashing going on consistent hashing to be specific you can build your own mimc D client that does whatever you want right and then distribute that stuff well what happen if I if I add a if I add a server well your client can start Distributing the keys I would definitely not be with that that because why would you distribute the keys for in a transient cash anyway who cares at a server is like oh yeah if if the kid is not there you're going to query the database and pull it up right it's it's not worth it to do this this chattiness to move items around from one server to another that's just a bad idea I don't know if clients do it maybe they do but I don't think it's it's required it's just thrashing for the case thrashing D B Shing again you you might if you know this channel you know that I'm I try as much as possible to push it as the last resort I do not like distributed stuff especially so complex to deal with right I like Simplicity I'm a simple man right but yeah sometimes you go you have you go to the YouTube scale and Google scale then you don't have a choice one machine cannot possibly handle everything I would I would go with raid replicas I would go with partitioning horizontal partitioning in the server itself minimize that as much as possible I would go with raid you know distributed dis storage but the application remain as a single writer the moment you have multiple writers and you have to deal with the Shing it becomes really complex you know if you want to deal with the complex complexity sure but yeah that's that's the idea of distributed cash okay let's do a demo we're going to do a demo we're going to spin up a bunch of MIM cash D Docker instances so for this exercise just install Docker and you're good to go and you have to have a Docker H you have to have a Docker account because somehow they are locked behind an account M casd I have no idea why did that they did do they do that sometimes right so you just just create an account do a Docker login you're good to go right once you do that you can download the image and you can spin up as many MIM cash G instances as you want so we're going to do that I'm going to use tnet because I love the Simplicity of MIM casd you know how many clients these days that you can actually just til it and run commments to they can be counted on one finger you know they don't exist anymore the Simplicity is gone from these from today's applications right the good old days of you just telling it and run and one thing I didn't mention is M casd doesn't have security by default so that might be a deal breaker for you right so you have to you can Implement authentication which doesn't exist by default sassel I believe they call it you can Implement TLS if you want but by defa they don't have any of that stuff right so take it with a grain of salt right they they said simple it is simple right but you have to be careful in a cloud environment when it comes to MIM casd you have to TLS it right they support that there is a support for that and obviously we're going to use no JSM Cas D for this consistent hashing and we going to put all our Docker containers and and play with that a little bit how about we do that so I have Docker installed here on my Mac you can have Windows and install Docker on top of it you can have Linux install Docker on top of it that's why I always like to use Docker just it's an agnostic whether whe whatever your application is you know whatever your operating system Docker works you know we have it on top of all all of this stuff so let's go ahead and spin up a Docker container that have a mimc d instance one MIM casd instance right so Docker run you do das Das name let's give it a name uh let's called it M1 M casd or M1 right you don't have to give it a name but I like that so that we can find it and delete it later easily and then you can expose the port uh by default 11211 right 11211 so this is what is running in the container this is what is exposed in my host right cuz I'm going to hit my host which is Hussein Mac which is that's the actual host that is running Docker and then I'm going to hit that Port which will be Port forwarded to this Con container and I'm going to spin up another one with 11 211 11 212 and 1121 three right later we're going to spin up multiple ones and then finally we're going to do MIM cache d right if you do just like that this will block the uh the terminal you know and it's going to be just work so I I suggest you do that first I know it's going to work for me because I I did it before but I like to do dashd right- d means just hey detach it because I'm going to use this terminal for something else later right so just go just like that we created a container it can do Docker PS to make sure that the container is running obviously if the image is not there it's going to download it for you and you have to do Docker login to do all that stuff so do all of that log to your account and uh all that stuff all right so let's test it out how do you test it tell it what are we telling it into Hussein Mac which is my host and which Port 11211 again this is the port that I exposed that happen to be the same doesn't have to right if I do that all of a sudden I'm logged in how do I do that well let's do a stats give me your stats and this is the stuff that most of the stuff here we talked about currently right up time what's the version of MCD the pointer size maximum number of connections 1024 we talked about that right there's a maximum number of connections uh how many times you run a get how many times you run a set how many times you incremented uh the threads how many threads you have here right eviction how many times they allow you kicked in and evicted stuff and then you can do like stats uh Slack I think which going to give you like how many slabs were active how many is actually allocated obviously we don't have anything because we didn't do anything right so let's go ahead and set something so to set a key you do set and then you do the key let's call it Fu and then um the flags zero I don't have any Flags here Flags you can do further controls over here and the second parameter here is the expiration so let's say it's 3,600 which is an hour right you can set it for for an hour you can say it for a minute you can say it for a second if you want right that's the expiration so if your item ever get to after an hour it will be Eed it will not be returned to you not necessarily will be Eed until the lru kicks in and that actually physically removes it right and then finally we're going to uh put the data length how how how big is the value that you're going to set let's say two characters here right so I'm going to do high you have to exactly match it right otherwise it's going to be uh it's not going to fit nicely right so now we sto the value so let's read that value give yay I know this is just very simple stuff but you get the point right you can you can uh increment you can you can delete that key right and if you delete it you can read that it's not there so very simple stuff I don't really care about the API more I want more to talk about the architecture of stuff here right and that's what matters here there's there are actually two protocols command set this is the old one and there's the new one which is starts with mg like two characters and there's like a different set of syntax right there are two syntaxes here syntax can you say syntaxes I I guess you do you can but you can play with this it's very simple you connect to that and you see I didn't log in there's no accounts or anything like that there's no collection that you create it's just a free floating right some people might like that some people might not like that cuz they want partitioning hey let me create a table or collection let me play with that key value right there so it's a it's a freefor all if I if I destroy this connection so if I destroy it and I connect it again right and I do get F obviously it's not there because we deleted it so if I set it again um black zero let's said it for 10 seconds or or 2000 seconds whatever and then to high stored get Fu right if I killed it now let's kill the session right quit and then do again connect and then get Fu the value is still there obviously right cuz it's it's stort in memory right even if I connect it as a different TCP connection right and now that we did that if you do stats slabs you can see some interesting values that we talked about here right let's chunk size right so we have this number one which is this represent the slab class that we talked about right so we happen to have one slab class because my value is so tiny right if I created another chunk with a large value the large item another slap class will be created and that will have its own configuration so the chunks per page which we talked about 10,922 we have one page only as we add items we can just increase that if we want how many is in used one chunk how many is free 21 109 21 so that's the total we used one how many time we read it how many time we said it how many time we delet it everything here is actually uh accounted for active slabs we have one slab right here effectively active and this is the memory allocated right so it's like what that's like uh one Meg exactly right that's the make is that is allocated we talked about that so yeah so you can start playing with that and add M multiple data points and and look at the stats and play with it a little bit so let's control this and quit so now that we talked about tillet how about we actually go to nodejs and build our beautiful interesting application here all right so I created this I went to my project folder here I'm going to go Ahad and create a directly called no mod M right and I just go ahead and do that right do an mpm init Dy right I have nodejs of of course installed right here to do all this stuff right and then um let's create an index.js file and we're going to do const MIM cached equal I believe it just do require MIM cache literally and that's the library we're going to install so once we have this Library we can call it right how can I call it const um let call it server and then we're going to create a new MIM cached and here's the interesting thing you can pass in an array of servers and you can pass in a string you can pass in an array or you can pass an object if you pass an array then the uh the the keys will be evenly distributed between all these instances today I have only one server so Hussein Mac 11211 all right close the array we're going to add more servers later but that's that's it basically because server I guess server pool is a better name huh let's let's call it server pool because that's what it is it's a server pool here let's create a function called run and this function will be called in right and this function we're going to use the server pool and we're going to set a value so here's how we set a value set pool uh server pool do set and you give it a key say F right and then the value bar right and then expiration day an hour and then the final one is a call back which gives you like in case of an error I'm not going to set it because I trust that it's going to just work so all we have to do is uh do that and this will just set the value but for the sake of time I'm going to actually set 10 values 3 4 5 6 S 8 nine for those who know JavaScript we can do this trick like for each I I think you can do a better job at this than I will but I think this works right this this should work right I like that that so they will have a different key a and the bar will have a this way we'll set what 10 values in this case and each value will have the full a for one and bar one for two bar two the reason I do this is because I want to actually see the I'm not going to read it from here all of us going to do that is just run and I'm going to read it from tnet right that's how I'm going to do it let's go ahead and do save mpm install MIM casd and then npm that's it node index. GS hopefully it runs and of course the moment I say that I have an error so let's go ahead and and check the error here so I going to see what the error is for each so let's go ahead and just add that okay this is going to print the error in case there is an error just in case all right try it again node index toj all right now works I had like a typo so I had effect but look at this get full three get full four get full five we're getting all the values that is pretty cool you guys right it's pretty cool so here's what I'm going to do now let's control exit here quit here's what I'm going to do now I'm going to spin up more containers so it's going to do run Das Dash name MIM 2 right they call it- p1212 2 11211 again this is the actual Port will this will not change this is what's changing here right right here right detach MIM cash d right got to do the same thing or three three that is pretty neat right and uh for for sake of completing let's add so four servers why servers are free we can we can spin up as much as we want now let's edit our application right here and uh what we're going to do is literally just add a comma say hey hin Mac 11212 is another server right there's another one too hin Mac all they are living in the same server if you think about it right it's just different Services right J say mac 11214 that is awesome nothing changes right so I'm going to do it again and then node index.js I don't want you to pay attention to what will happen now The Client app now distributed all the stuff to all the servers so now my foods will be all over the place Let's test it out tet Hussein Mac uh 11212 right let's connect to 11212 the second server and then get f one it's right there get F two not there get F three not yeah it's right there good F four it's not there get it because now the distribution is up to the cine I have no idea how how this will be distributed probably round robin but uh could be something else right so if I pick now another server right three let's do that get f one not there get F two not there get F three not there good full four not there good full five right there so it took us like five five is there right and and you get the idea these keys are now distributed everywhere and when you ask about it now I'm going physically to the server itself to ask about it but if I ask the nodejs app it's going to give me these values right so here's what I'm going to do next right I'm going to do index.js after I run all these which is I'm not going to run it anymore right uh cuz I already store these for an hour I'm going to just go a loop and read right going to create a function that reads and exactly similar uh YY p and then just do a get right the get is slightly different what we going to do is that you don't need a value right you don't need an expir date but it's going to give you a function a callback where it's actually two places error and data right and then we just print the data CU that's what we're interested in assume there are no errors here again this is a very simple app here and then's just go ahead and read so in this case you're talking to the pool directly you're not talking to individual machines so we know that the keys were actually evenly distributed between the servers because we ta knitted into each servers and tested that out right so now what we're going to do is let's run and see node index.js look at beautiful Bar Nine bar two bar four bar three bar six Bar Seven bar eight bar one bar five why do we get the different values it's very easy because we we're we're running asynchronous job we have no idea yeah we executed f one first right but Fu right we we what do we did is like we looped and sent all the 10 request at the same time right all this is what we did we looped and all the 10 request but these are asynchronous so Bar Nine F9 might get give us a result before f8 right all of these are just this is how no GS works it's single threaded and it sends all these request and just Loops through the its main um the the main Loop the the main thread Loop right and looks for the result it depends on the server how fast is going to respond right so is going to send all these things and then hey the server responds for this respond for this for this we just do all this stuff so the client here it really depends what it does as well right so the client really depends on this it took the hash right of the f one determine that F1 should exist on This Server connect it to the server asked for the value pull that value and then return it so you have no idea how fast these servers will reply right and the number number of connections to each server also really matter right so that's basically uh it for the demo guys and kind of explained all this uh idea of MIM casd let's go ahead and summarize this course all right we did the demo let's summarize we talked about memory management memory is fragmented if we didn't do the slab Pages concept then we're going to be allocating values left and right right and that as a result uh becomes fragmentation and fragmentation is bad because now you have all these beautiful gaps of free space that we cannot use unfortunately right because our items might not necessarily fit these gaps right so we need memory management lru again uh very in unpopular opinion I I would like for this to be an option to be disabled so that I don't get locked right and keep my application simple and if someone want to build Anu why don't they build it themselves right or just have the client have the control I wish they stayed simple and didn't implement this just that's just my only this criticism of the MIM casd they they stayed simple they stick to their rules this is to in my opinion I think it's an Overkill right threads I love this design yeah we can work with money threads of course there is a limit for there is another problem with the threads design here is that let's assume you have multiple threads all right and each thread is a connection right let's say about that right and and when we looked at the data I think when we looked at the stat we saw that there is a fixed number of threads right and I don't know if this if MIM casd share connections on a given thread like have multiple connection on thread I don't know that maybe because otherwise it's going to run out of threads right so in this case what you you you can end up with is a thread with a connection that happen to have a very aggressive client a client that sends a lot of d data to the thre you know so in this particular thing you create a bottleneck and that bottleneck really there is no solution to it because you don't know if a connection is going to be aggressive or light waight right you can you can change that complete model to a centralized thread model where is a center thread that takes the messages these requests and these requests will be distributed evenly right you can do that but then the bottleneck is moved to a single thread you lose either way right there is no solution the best solution that's when you when you when you go into deep these things like it's it's fascinating to me I absolutely love it we talked about read and write talked about locking about distribution cach which is completely client side guys I hope you enjoy this uh crash course Deep dive level into MIM casd absolutely uh I enjoyed researching this talk me a lot a month to research this entire course uh absolutely love it uh if you want to support the channel become a member there is a lot of uh member exclusive content in this channel uh if you want to support otherwise there is the there's uh I have a lot of UD me courses uh there's discount coupons check it check them out and that supports the channel I appreciate you so much and thank you all for your wonderful messages hope you enjoy this course I'm going to see you on the next one you guys stay awesome goodbye