Translation Lookaside buffer

Key Takeaways

So next time I read from a a virtual address that happened to be in the same page that I already cached, it's free. It's a hit. It's a cash hit. So that's nice, right? You get all these nice caches. Ah, so beautiful when you hit the TLB. But that is that's the scarce memory by the way. So if you're doing a lot of virtual mapping and you're all over the place then older translations will be discarded will be flushed and then new addresses will be entered. The final we're going to talk about is is the caching. Of course the walk the page walk this is literally what the what we explained is expensive. You don't want to walk the memory every time because the C while the CPU is doing a starting to read something like hey read X address from memory I just want to read X address I don't want to translate in order to read I try to at least do a little bit translation as possible right so if if in order to read a memory you need to translate the memory which requires reading from memory to actually translate to get the address to actually read the memory will never finish right will never be done. So what the CPU does is it takes that knowledge right of the walk that says all right this page address virtual address is is this physical address. So it skips all of that and caches that as a nice array that we talked about, but it's a very compact. It's a tight cache in the CPU. It's called a TLB, translation locusside buffer and in it, it caches pages mappings essentially and it's very efficient, right? Because it knows that it works with a single process. Right? And the virtual translation for that process as long as I'm executing the same process, I can reuse this cache all over all over again. Right? So if I'm at the moment I I read another page, I update my cache like all right this is this page. So next time I read from a a virtual address that happened to be in the same page that I already cached, it's free. It's a hit. It's a cash hit. So that's nice, right? You get all these nice caches. Uh so beautiful when you hit the TLB. But that is that's the scarce memory by the way. So if you're doing a lot of virtual mapping and you're all over the place, then older uh translations will be discarded, will be flushed, right? And then new addresses will be entered essentially to do the translation. So can it can get expensive. Uh and I think one of the the problems uh that people noticed in MySQL 8 was this large cache uh large code base which means massive number of page virtual page translation to physical beta translation which exceeded as you're reading from this physical memory and as you're jumping between different pages uh the TLB gets full and then you flush it and then you keep walking the page again and then doing back and forth and that can kind of suffer the performance because if you have like a large code base that means all of this has to go in memory that's we still have to read that stuff right and as you're reading and you're jumping between different places especially if you have like functions that are that are all over the place that are not next to each other in in code then you're reading from different physical places in memory and that are as opposed to reading like from the stack, right? Stack is beautiful. It the TLB loves the stack because mo most of the stuff is if you're reading from a function, you're calling another function, it's just almost the same page. If it's not the same page, the the page next to it. So the translations are very similar. So you cached at least some of the stuff, right? So as you're reading through things things are contig the locality of data access is so critical here and compare this to heap accesses which are almost random uh randomness can create these fragmented TL entries for the page address translations. So that's that's fascinating stuff there. uh pop quiz uh process A gets in uh TLB's cached right for all these virtual to physical addresses. Now uh colonel does a conte switch brings in process B does the TB remains or no? Can I reuse the TLB that I just cached this all this beautiful virtual addresses? No, you cannot use the TLB for a different process because virtual addresses are unique mappings or unique for per process. So if I have virtual address I don't know seven in my process that maps to a certain physical address the same seven can be mapping to a different physical address for another process or could be even not mapped. So I cannot use that stale entry. It's a private mappings. So you do not you do not use that at all. You cannot reuse TLB entries for between processes between threads. It's amazing. Why? Aha. Why? Guess if I switching threads, it's beautiful. As long as those threads belong to the same process because guess what? It's a shared memory. It's a shared memory, guys. All threads share the same virtual memory space of the parent process. So if I'm switching threads, it's way faster. I keep I get to keep my beautiful TLB. I don't flush it. But if I'm switching processes, I need to flush my TLB. That TLB needs to go. And so you have to rewalk the new page and catch new entries. The exception is kernel pages which are marked as global. kernel code is shared between every single process. It's a kernel. It's the kernel. But the kernel has code itself and the CPU has to read the code to execute system calls for example, right? And interrupts and whatnot. So in order to do that, well, you cannot because the kernel also lives in virtual memory. The same thing that has its own virtual memory. It's the same thing, right? Well, if you think about it, the trick people that we did we do in the kernel is the process top part whatever 99999 or FF FF until 77 something all of this is mapped is reserved for the kernel. So the top part of your process actually mapped to the kernel code. So that's another benefit of the virtual memory. without you knowing your part of process is actually half of it or the top half is actually for the kernel and you just map to it right but you cannot you as a user you cannot access your the kernel stuff so it's all it's another shared memory concept so pages that are if you're accessing these virtual memory that belong to the kernel they there is a bit in the page table that's called global literally says global bit and the c the the the the CPU when it loads the page it knows that it's a global page. So the TLB entry has the bit set on [snorts] as a global. So it will not essentially flush those entries because they can be reused because it's it's so beautiful, right? Because the kernel will be walked a lot. So I it would be nice to have the kernel pages cached, right? Of course, it takes precious uh memory, not memory, cash space from your TLB, but hey, that's the price we pay, my friends. for all this virtual memory stuff, guys. Yeah. So, of course, um it all goes back to the CR3 register in the CPU. Every time I do a context switch, the the kernel writes the page table root physical address translation in the CR3 register so that the CPU just reads the CR3. It doesn't even know. The CP doesn't know about processes. It doesn't know processes exist. It just just, hey, all right, someone writes to my CR3, I'm just going to walk this time. It just just executes code. It doesn't know what's going on. There's no concept of processes when it comes to CPU. Doesn't know. [snorts] Maybe this changed, but it's like it's an abstraction. It's a it's an oper abstraction. This process is idea, right? So if I switch a process, I just update the CR3 and now I'm just pointing to a different location in memory and physical memory which has the page walk that we talked about the levels, right? and and we just walk that and finally guys um what I [snorts] what I like to also talk about this like I talk about the old virtual memory and you see the complication right it's it's not a easy concept take it with a grain of salt challenge it everything can be broken all of this stuff I always like dreamt of building my own kernel but not a general purpose a specific kernel just for my needs. I don't need half of the stuff in the kernel. We don't really need if you're building a specific things, right? It's a general purpose. And the anything that general purpose, you cut corners, right? You do all of this stuff, right? You do all this like, oh, what if someone do this? What if someone did that? What if someone did that? But I was like dreaming about building a database as a kernel itself. The database talks a database system that is the kernel. There is no operating system. Don't need that. Just database on the metal directly. No virtual memory. Why do I need a virtual memory for that? I know my stuff. I'm spinning up my own. There's no processes. I don't even need processes. What? I have code. I have single process doing everything. Why? Why do you need processes? You imagine the limits you can go to. Unlimited my friends. Unlimited where you can go. You can go everywhere with this. Just expand your horizon. Everything can be broken. Don't take anything for granted, right? And challenge everything. Even all of this stuff. There's beauty in what we build. But in the same times you can always challenge that. Take it what it for what it is. Understand why it was built. Go back to the roots and tweak it back. Build your own