System Design for Beginners (Full Guide)

Getting started [music] with system design can be overwhelming. Since it not only involves having the technical basis to make informed decisions, but also having a deep understanding of the problem you want to solve. Today, we're going to solve a classic interview system design problem. And by the end, you have a repeatable framework for any system design question. First things first, what is system design? It is the process of determining the components and architecture of a system in order to comply with a set of requirements and quality standards, as well as determining how components will communicate with each other. The key word in system design is trade-offs. These are the pros and cons we analyze in every system design decision, and weigh against which quality aspects are most important to us. We usually balance performance aspects like speed and response, also known as latency, cost, and complexity. Let's see this in practice. Step one, understanding the problem. Today, we are designing a URL shortener. The usage flow is simple. User submit a long URL, the system creates a short URL to substitute it, and create an internal mapping that associates the short URL with the original, and returns the short version to the user. Then, when a client accesses the short URL, the system recognizes it and redirects user to the correct location. Step two, identifying requirements. The golden rule in system design is, don't make silent assumptions. Ask clarifying questions. Your interviewer is expecting this. Your questions should guide you into understanding exactly what your end product should look like, and any special requirements and constraints. When we talk about requirements and system design, we can divide them into two types. Functional and non-functional requirements. Functional requirements, as the name suggests, points to the functionality of your system. They specify the main features and what the user can expect from them. Non-functional requirements, on the other hand, speak of the technical aspects of a system, covering elements that guarantee system quality, like performance, scalability, availability, security, etc. Out of the box, from the problem instruction, we can derive a few functional requirements like one, users should be able to submit a long URL and receive a short URL in return. And two, when a client accesses a short URL, they should be redirected to the original long URL. Something that was not explicit in the problem instruction, but would be a good question to ask an interviewer, is whether or not we should implement some kind of account registry, so that users can manage their created short URLs to have a centralized view and be able to enable, disable, or even delete the generated URLs. Let's imagine we ask our interviewer and they said yes. Users should be able to have this control. Now, we have new functional requirements. So, three, users should be able to register and log into the system. And four, users should be able to manage their own short URLs, including enabling, disabling, or deleting them. Remember, the interviewer may not hand you all the details up front. They're not trying to trick you or see you fail, but they are checking your ability of thinking about the entire usage flow, user experience, and consider edge cases. We also have to consider the non-functional requirements. You can ask your interviewer if they have any specific non-functional requirements like security concerns or if tech stack limitation. But in general system design quality aspects, let's imagine our priorities are the system should have a one-to-one relation between short URLs and the corresponding long URL. Meaning, we want to prevent duplicate short URL generation or a short URL that points to more than one long URL. The system should have low latency. Meaning, the time from a client request until the response is short. You can set a specific threshold like responding in less than 100 milliseconds, for example. The system should be highly available. Meaning, the system should be continuously available with no or close to no downtime. The system should be resilient to failure. Meaning, it should be able to adapt to and recover from errors and failures. Step three, predicting traffic patterns. This is the point in a system design interview when we want to start introducing some numbers. In a real-world scenario, we would try to make predictions based on data we already have like how many customers does our system have, how many times a day they use each operation, and expected growth for the company. In a system design interview scenario, it's a good idea to ask your interviewer for this information or make up your own and ask if it seems possible. In the scenario, let's imagine we can expect 1 million new short URLs created per day, 100 million redirects per day, an average peak traffic of about 1,200 redirects per second, and expect a data retention of 5 years. So, we should plan for storing a few billion URLs mapping over time. Something important thing is the read-to-write ratio of 100 reads to one write for our system, which is typical for a URL shortener, since a single short URL tends to be accessed many times after being created. This means that our system will require a lot more processing power to redirect than to create short URLs, which is something to consider in terms of scaling needs. Scaling means adapting your system infrastructure to support changes in traffic. In other words, giving it more resources in moments of peak request and removing resources when they're no longer necessary. Considering this is crucial to our non-functional requirements of having a highly available system. This brings me to an important system design concept for us to cover, monolithic versus microservice architecture. A monolith is a single unit of an application that centralizes all components. You can imagine it like a Swiss knife. Microservices, on the other hand, break the software into smaller, independent services specialized in one part of the application, like different tools in a toolbox. The trade-off analysis for monoliths and microservices is a longer discussion, subject for another video. But one thing I would like to point out is that a monolith can add complexity in terms of scaling, since we cannot scale only the functionalities that need it, but have to scale the entire unit. Microservices offer flexibility in scaling rules, because we can scale each component independently. Considering your system has a reality of having 100 times more reads than writes, without extra constraints, I'm inclined on breaking it into microservices, which leads me to step four, designing high-level components. Now that we have our requirements, we can start picturing in very high level what different parts composes our system and where are the connection points. Since we're designing a user-facing application, we can already imagine we will need a front-end and a back-end. As I said in the previous step, I want to divide my back-end in microservices. So, I'm creating a service for each functionality. Let's have a service for creating short URLs, another for redirecting short URLs to their original versions, and a third service to handle user registration and login. These services have to be connected to database to persist and access the mapping between the long URL and the generated short URL, as well as the user data. So, we add a high-level database component. We also need a component for our front-end assets, so users can interact with our system. We call this our client. Let's also add an authentication service responsible for handling user registration, login, and issuing tokens that provide who the user is on subsequent requests. And finally, we need an API connector, which is the entry point for our client requests to be directed to the back-end. Notice that we're not saying which database or which programming language we're using it. That comes later. First, we want to understand the shape of the system. Step five, defining APIs. With the components laid out, we can start sketching the contracts between them. An API contract describes what the client sends and what the server responds with. Let's keep them simple and focus on core operations. So, we have post shorten receives a long URL in the request body and returns the generated short URL. This is the entry point to our URL shortener service. Get by short URL receives a short URL in the path and responds with an HTTP 301 or 302 redirecting to the original long URL. Post users register receives user data like email and password and creates a new user account. Post users login receives user credentials and returns an authentication token the client will use on every protected request from that point on. Get users me URLs returns the list of short URLs owned by the authenticated users so they can manage them. Patch users me URLs by short URL allows users to edit URLs enabling or disabling them. Delete users me URLs by short URL allows users to delete their own short URLs. Step six, selecting a tech stack. You see how up to this point we didn't even think about specific services or technology stack. The main point is solving the problem and the implementation comes as a consequence. Now that we have our requirements determined and the basis of our components, we can think of a tech stack. For this example, let's see what our tech stack looks like using AWS services. Here's a possible mapping. For our microservices, let's add an Amazon EC2 instance for each service. These are instances of servers that you can set up and manage and allow for independent scaling as we mentioned before. To guarantee resilience and availability, we can add at least two servers per microservice. This is called server redundancy and it assures us that even if one server fails, the application will still be running on the other one and will not go down. We could also go serverless and create different lambda functions for each service. Serverless applications have the benefit of scaling automatically, but come with their own challenges. Since we're creating an architecture with managed server instances, we should also add a load balancer with Amazon Elastic Load Balancer. For the front-end client, let's use AWS Amplify, which will host our front-end application with automatic deploys, CDN, and HTTPS out of the box. For the API connector, we are using Amazon API Gateway that will route the requests to our microservices. For authentication, we use Amazon Cognito to handle user registration, login, and token issuing without us having to build from scratch. It also integrates with API Gateway to authorize the protected endpoints we defined earlier. Finally, for database, the most basic decision is choosing between a SQL or NoSQL database. There's a lot deeper we could go here as well, but the main difference is on how they structure data. A SQL database will be structured as tables with rows and columns, while a NoSQL database is built on key-value pairs, which tends to give it lower latency on reads. A SQL database may be preferred if you have highly structured data and want to perform different join operations. Since in this case our data is simple, easily structured as key-value pairs, and our non-functional requirements expect us to prioritize low latency, I'm choosing NoSQL and adding Amazon DynamoDB to our diagram. Again, this is one valid combination, not the only one. Different set of trade-offs could lead you into a relational database, a containerized service, or a different authentication strategy. What matters is being able to justify each choice against the requirements. Step seven, considering implementation constraints. For many system design interviews, what we did may be enough. We showed the interviewer that we understand the problem and its requirements, designed the high-level flow, and suggested a tech stack based on specific technical trade-offs. But you can still go the extra mile and point out any lower-level concerns. A good way to do this is to go over your requirements list and check if you covered everything. So, let's do it. Functional requirements. Users should be able to submit a long URL and receive a short URL in return. Check. When a client accesses a short URL, they should be redirected to the original long URL. Check. Users should be able to register and log in to the system. Check. And users should be able to manage their own short URLs, including enabling, disabling, or deleting them. Check. And for non-functional requirements, the system should have a low latency. Check. We considered this in our architecture choices. The system should be highly available. Check. We can scale parts of our back end independently. The system should be resilient to failure. Check. We have redundancy in our servers so that our application will still be running even if a server fails. The system should have a one-to-one relation between short URLs and the corresponding long URL. Notice we didn't address the requirement of not having duplicate URLs. In an interview, the interviewer could point this out to you, or they could make a silent note on how you weren't attentive to details. There are few options to this that are worth researching on, and each comes with its own trade-offs. One possibility is hashing the long URL and taking the first few characters as the short URL. And before writing to the database, checking if the short URL already exists. If so, we can add a small appendix to the short URL and rehash until we get a unique URL. Congratulations. You just designed a URL shortener. Before we wrap up, I want to leave you with a few things that made the biggest difference for me when I was getting into system design. The first one is that system design is a matter of practice. You're not supposed to come up with a perfect architecture the first time you see a problem, and nobody does. The more systems you design, whether real life or theoretical exercises, the more patterns you start to recognize, and the faster your brain goes from I have no idea where to start to okay, this looks like something I've seen before, I know how to approach it. That's why studying classic design problems is so valuable. URL shorteners, chat apps, newsfeed, hotel booking services, video streaming platforms, they come up again and again in interviews, not because interviewers are lazy, but because each of them exercises a different set of fundamentals. Once you design a couple of them, you start having reusable building blocks you can mix and match from. The second thing is to think critically about every decision you make. Don't just pick a database because everybody uses it, or a technology because it's trendy. Ask yourself why, and more importantly, ask yourself what you're giving up when choosing it. Every decision in a system design has a cost, and being able to articulate that trade-off is what separates someone who memorized answers from someone who actually understands what they're doing. And finally, invest time in the foundations. Things like how databases work under the hood, how caching changes performance, what happens in a network request. These concepts show up in every single system you'll ever design. And the stronger your basis, the less intimidating every new problem or interview feels. System design is less about knowing the right answer and more about structured thinking. If you walk through the requirements, traffic patterns, components, APIs, and tech choices out loud, and you justify every decision against a trade-off, you're already doing it right. The URL shortener we designed today is a classic interview problem, but the framework works for almost anything. Start with a problem, not a technology. If this helped, leave a like, subscribe for more cloud [music] and system design content, and drop in the comments other classic problems you would like us to break down. Happy building, and see you in the next one. Bye. >> [music]