Async Python Explained for AI Backends — I/O vs CPU Bound

Welcome back. In this video, we are going to talk about async. Now, async is one of those things that sounds complicated, but really isn't once you see it the right way. A lot of developer thinks async just means faster. That's not quite right. By the end of this video, you will know exactly what async does, where it actually helps, and where it doesn't. Let's start. So, when Karan saw the fast API and async first, he thought, "Great, I will use async everywhere, and everything will be faster." That's very common assumption, but it's not how async works. Async helps in some situation that does nothing useful in others. If you use it in the wrong place, you just make your code more complex for no gain. So, let's build the right picture from the start. Here's the simplest way to think about async. Look at the chef on the screen. He put a dish in the oven. He has two choices: stand in front of it doing nothing, or go chop vegetables while it bakes. Async is the second chef. While your code is waiting for a response from an external service, it goes and handles something else instead of sitting idle. That's really all async is. Not magic, just not wasting wait time. Now, here's the key question. When does async actually help? It comes down to the type of work your code is doing. Input-output bound work means your code sends a request and waits for something to come back. CPU-bound work means your processor is busy running calculation. Async helps when your code is waiting. It does nothing useful when your processor is already busy. Let's look at both with examples. Input-output bound work is when your code is just waiting. You call the OpenAI API and wait for the response. You query your vector database and wait for results. You fetch conversation history from a database and wait for it to load. These things can take anywhere from a few hundred milliseconds to several seconds. During all that waiting, an async server doesn't just sit there. It goes and handles another incoming request. That's where you get real performance improvement. Now, look at the CPU bound diagram. Request come in, they hit your server, and the server sends them straight to the CPU for active computation. There's no waiting involved. The CPU is busy the whole time. Notice the queue building up at the bottom. That's what happens when the CPU can't keep up. Async won't help here because there's nothing to wait on. The right solution is separate worker processes that can handle computation in parallel. Now, let's see this in a real scenario. A user sends a query to your rag API. Your server calls an embedding API. That's waiting. Then queries the vector database. More waiting. Then calls the LLM. Waiting again. In a regular server, every other user is blocked during all of this. In an async server, while step one is waiting for the embedding API, the server is already picking up the next request. For a production API with many users hitting at once, this makes a huge difference. Here's the simple rule. Ask yourself, is my code waiting for something or is it actively computing? If it's waiting, use async def. If it's computing, use a regular function or a separate worker. And one thing to never do, don't put heavy CPU work inside an async function. It will block your server and make things worse, not better. That one question, waiting or computing, is all you need to make the right call every time. To wrap up, async means don't waste time waiting. Go do something else instead. It helps when your code is waiting on external responses, which is most of what Genie backend does. It doesn't help when your processor is busy computing. Remember the rule. Waiting means async def, computing means regular def or a worker. Next up, we get practical. We will set up the environment, install FastAPI, and write your first running endpoint. Next up, we get practical. We will set up environment, install FastAPI, and write your very first running endpoint. See you there.