How Key value Stores Work (Redis, DynamoDB, Memcached)?

Key Takeaways

Today we're looking at key value stores, a type of database that keeps track of everything from your shopping cart to your chat messages. And what starts as a simple idea quickly becomes a hot problem in software engineering. So what is a key value store? Think of it as a giant dictionary. You have a key, maybe user 1 2 3 4 5 cart and a value. All the items in that user's shopping cart. You can put data in, you can get data out. Here's the thing, though. Amazon stores everything. We're talking about terabytes of data per major region with billions of key value pairs that need to be accessed millions of times per second. No single computer can handle that load. So, you need to spread it across thousands of servers. But now you have a problem. When someone asks for user 1 2 3 4 5 cart, how do you know which server has it? Your first instinct might be just hash the key and use modulo. Hash the key, divide by the number of servers, use the remainders to pick a server. Simple math. But here's where it gets tricky. What happens when you need to add a new server? Suddenly, you're dividing by a different number and almost every key maps to a different server. You would have to move nearly all your data just to add one machine. This is where consistent hashing comes in, and it's pretty clever. Instead of mapping keys directly to servers, imagine both keys and servers living on a giant circle. Think of a clock face, but instead of 12 hours, you have millions of positions. You place your servers at random spots around this circle. Maybe server A is at position 100. Server B is at position 1,000. Server C is at position 5,000. Now, when you want to store user 1 2 3 4 5 cart, you hash that key to get a position on that circle. Let's say position 750. You site at that spot and walk clockwise until you hit the first server. In this case, that would be server B at position a th00and. The magic happens when you add a new server. Say you place server D at position 500. Now keys that hash to position 101 through 500 go to server D instead of server B. But everything else stay exactly where it was. You only move a fraction of your data. But wait, what happens when server B crashes? Now all those keys has nowhere to go. This is where you need copies. Instead of storing each piece of data on just one server, you store it on multiple servers. One way to do this is to keep using that circle. When user 1 2 3 4 5 card hashes to position 750, you don't just store it on server B. You also store copies on the next two servers clockwise. Maybe server C and server A. Now, if server B goes down, you still have the data. Great. Now, your data is safe, but you've created a much bigger headache. Let's say two people are shopping for the same family account. They both add items to the cart at the exact same time, but their requests hit different servers. Now, you have two different versions of the same shopping cart. Which one is correct? This brings us to one of the fundamental concepts of distributed systems. You cannot have perfect consistency, perfect availability, and perfect network reliability all at the same time. You have to pick two. If you choose consistency, making sure everyone always sees the latest data. You might have to refuse requests when server can't communicate. Banks do this because showing the wrong account balance is unacceptable. If you choose availability, keeping the system running no matter what, you might occasionally serve stale data. Most web apps do this because it's better to show an old shopping cart than no shopping cart at all. The solution most big systems use is called eventual consistency. The idea is simple. Given enough time, all the copies will match up. But right now, this instance, they might be a little different. This creates a new challenge. How do you handle conflicting versions? There are different approaches to this problem. One clever solution is called vector clocks. Think of it like a version number but smarter. Every time a piece of data gets modified, it gets tagged with information about which server did the modification and when. When you detected two conflicting versions, you have a few options. Sometimes you can automatically merge them like combining two shopping carts to include all the items. Sometimes you have to ask the users to choose. Sometimes you just pick the most recent one and hope for the best. Next problem. In a system with thousands of servers, machines are failing constantly. How do you even know when a server is down? The naive approach is to have every server ping every other server. But with thousand servers, that's nearly a million connections. It doesn't scale. One solution that works well is called a gossip protocol. Each server keeps a list of all the other servers and occasionally shares that list with a few random neighbors. If server X stops responding, the gossip spreads throughout the entire cluster. It's exactly like how rumor spreads in high school. Surprisingly effective, and you don't need everyone talking to everyone. What started as remember this shopping cart turns into a master class in distributed systems engineering. And this is just the beginning. We haven't even touched storage engines, data center failures, or performance optimization across thousands of machines. The next time you add something to your cart and it just works, you'll know there's an intricate dance happening across data center around the world. All to make sure your data is exactly where you expect it to be. Ready to ace your next technical interview? Join our community where we offer comprehensive courses on system design, coding, behavioral questions, machine learning, and object-oriented design. Learn more at byitebico.com.