V8 Performance from the Driver's Seat (Chrome Dev Summit 2015)

Hi everybody. Uh my name is Seth Thompson and I am a PM on the uh Chrome V8 team, but I'm here to to represent the whole team and uh it's a big team. We have folks in Munich, San Francisco, Mountain View, New York, and contributors from around the world. So this stuff is is uh really due to their efforts. Okay, let's get started. So V8 is shares the same name as a race car engine. And I don't know much about cars or even mechanical engines, but I am fascinated by Formula 1. Uh for two reasons. The first reason is it's just so fast. The cars are going at such speeds around the track that I read if they could actually get enough runway on the ceiling, they generate enough downforce to stay on the ceiling. That's crazy. So I think we picked the right name for suggesting unparalleled speed. But the second thing about Formula 1 that I think is more pertinent to this talk is the fact that every year the cars that the racers drive undergo drastic changes to all sorts of parts of the car. The engine, the brakes, the shape of the car, the steering. So, I mean, here's just a picture over the years of how much each car has changed. And you can imagine how different they must be to drive around that track. I mean, the track is the same and the driver is the same often, but the cars that they're working with have just different performance characteristics. You know, yeartoear, uh, a team might change the tire tread. That means increased traction. That means that the the driver has to be cornering in a different way than they did with tires that that were smoother. Uh, they might upgrade a V8 engine, a V6 engine to V8. This creates more power and torque, but it changes the performance characteristics. Once again, uh team might change the turning radius to accommodate a particular track. I think Monaco, the track has such tight turns that they need to change how steering works to actually get the car around the turns. And the driver has to accommodate for that on the fly. And the shape of the car, the body of it, uh changes in ways that fundamentally affect the aerodynamics. just a small modification to one the tip or the tail can feel different as a driver going around the track. So when I joined Google or before I joined Google, I thought that the V8 engine which turns dynamic JavaScript into native machine code was done when they first announced it. I mean that was such a a major improvement in speed really ran almost as fast as native code for a variety of applications. I assume the team packed up, gave each other high fives, went home, were happy, they were done. But I couldn't have been more wrong. Um, ever since then, the team has been working on an amazing set of features. Things like adding new support for language semantics and syntax as ECMAScript changes and JavaScript changes. They change the uristics that they tune for optimization. things like code caching, inlining, type inference. There's a bunch of hacks in there to really squeeze out extra performance and the results are are dramatic. Chrome today with V8 of today is a turbocharged version of the original V8. So the engine really has changed and unlike a car which gets a new release maybe yearly, V8 gets a new upgrade every six weeks as a new version of Chrome and Chromium uh is released. So this talk is about the engine and the driver working together V8 and Chrome and you the developer writing code for V8 working together to to make the fastest possible web apps. So the talk is in two parts. The first part is what we're doing on our end to make all JavaScript currently out there faster improvements in the engine to ek out extra performance. And the second part is how you as a developer can best take advantage of these changes and adapt to machinery that's being evolved underneath you. because ultimately the goal is to write fast JavaScript that's fast today but then that remains fast as it stays out there on the web and is browsed uh via new browsers within continual improvements. So let's start with what VA has been working on, what the team has been working on in the past year. I'm just going to highlight four big areas. We released a new optimizing compiler. We drastically improved garbage collection by making it smarter and more intelligent. We've added support for ECMAScript 2615 aka ES6 and we have this incubation project which is really cool which is a spelt interpreter and I'll talk more about why we did that in a second. So let's start with Turbopan. Turbopan is the code name for a new optimizing compiler. Now, V8 as an engine, I mentioned compiles dynamic JavaScript to native code, but it has actually a bunch of different compilers and it chooses which one to use based on the code itself, how long a function is, how expensive the function is, what language features a function you're writing might use, and especially how hot the function is. The more you run a particular function in JavaScript, we pull out the big guns and we use a a compiler which is takes longer perhaps to compile your code but results in much improved optimizations. So why do we write a new optimizing compiler? Well, we wanted to start up from the ground up with a new approach. Turbopan represents the JavaScript a in a a sort of interesting new form called a sea of nodes. This reduces the loads and property accesses at a really low level in the memory to this giant soup that we can have more freedom in how we schedule which axises and which loads come first and last. The the scheduling freedom is is unparalleled and allows us to to architect new and improved optimizations. But what does this mean for you? Well, Turbopan is designed from day one to support ECMAScript 2015. So it's true some ECMAScript features even if we supported them syntactically in our older compiler we couldn't use all of the optimization techniques that let's say another compiler used because the the support was spotty but Turbopan will support all of ES 2015 and beyond and it will make it fast. This is really exciting. Turbopan uses static type information for the first time. Previously we inferred type as a function ran. We made guesses about what the type was, but with turboan when there's static type information, we can leverage it from the beginning. So, I'm very proud to announce that we now fully optimize or will fully optimize asjs with turbopan. This is a this is a big improvement. And soon when the web assembly language spec is finished, we'll be able to optimize that as well. So rather than guessing at what types are are you're using in JavaScript, if you use asam.js JS compile C or C++ code. We can run it like native code because we know from the beginning what types are being used. So turboan is great but this next improvement I think users will really feel fastest. So idle time garbage collection is a method of freeing memory in the browser more intelligently. Now I'm sure you use Chrome and you know that when you have a lot of tabs open the memory consumption is non-trivial. So all of these optimizations make the browser feel so much lighter. So what's going on? Well, Blink, the rendering engine in Chrome, released a scheduling API recently. This is amazing because for the first time, a single API has a global knowledge of what tasks are happening in the browser. that is when clicks are coming in, when a scroll event is happening, when the page is actually being rendered to the screen for a frame, and of course, when V8 is busy performing a JavaScript execution on the main thread. So, with this newuler API, we have a sense of what's going on. Now, I'd like to show a quick diagram here that explains why jank happens in the browser, why something that should be a smooth animation feels kind of jerky and stuttery. Well, for smooth animation, you need 60 frames per second. So, the bottom of this diagram has markers for frames going forward in time. And in between those frames, the browser's doing execution to figure out what to paint to the frame for the next screen. And you can see here, it's it's often variable. Some frames have a lot of JavaScript work and some have idle time. Now, as the JavaScript is running, it's creating new objects. the memory profile is increasing. So V8 has to have a garbage collector which comes along at some point and says, "Oh, these JavaScript objects aren't being used. We can free them up and lower the memory profile." But normally it waits until the memory profile hits a certain point and then schedules garbage collection. It does this because it doesn't know any better. You can see here how big that garbage collection chunk is. And if it comes at the wrong time, it may postpone execution of the JavaScript for the next frame and miss the frame. This is a janky disrupted animation. With theuler API though, now we know when these idle spots are happening from V8. So we can proactively schedule garbage collection as smaller chunks before we really need to to garbage collect. So you can see here we've got some free time in that second frame. So, we'll go ahead and garbage collect and reduce the the profile a bit. Having this global knowledge of idle times allows us to make garbage collection essentially happen when you're least expecting it, resulting in smoother animation. Here is a demo of the Ort Online 3D benchmark. And you can see after these improvements just how much smoother this is. In fact, the the video that's playing in Keynote right now is is not representing this correctly. If you try this in your browser on the right, it's even smoother than it looks here. And it's faster, too. You can see how much further we are along in the benchmark than we were before. And again, this happens because we know more intelligently when to schedule garbage collection in between frames. Now, there's another improvement that this knowledge of when the browser is idle brings. V8 uh has this heap of dynamic JavaScript objects. This is what makes the memory profile so big. what takes up space uh in in in your memory and often a page creates a bunch of memory or a bunch of objects when it initially loads. So you can see here this graph uh of a page creating a bunch of objects and increasing memory. Uh right after the page loads, we go and we increase the heap size to that limit prime. But then garbage collection kicks in. We reduce the memory footprint a bit and then I background the Gmail tab. So let's say I go away. I'm not using it actively and the page goes idle. Well, you can see the the usage of of the creation of new JavaScript objects kind of tails off and we don't have much memory consumption. But before these improvements, that heap size had still been raised and we were taking up more memory than we needed. Well, now Blink knows when you background the Gmail tab when you're not really interacting with the page. And we can use that time when it's in the background to clean up after ourselves, shrink the heap, and just take back a bunch of memory. So, here's a another demo. They're flipped this time, so be careful. On the right is Chrome 43, the old version. On the left is Chrome 45 after we implemented these improvements. And you can see when you initially load the page, memory consumption is pretty high. It's around 150 200 megabytes. But the crazy part or the excuse me uh it's it was about 150 150 on either side. But after uh both tabs goes idle only in the new version does Chrome realize, hey, we're idle. We can free some memory. And quickly, I think you'll see it it it happen soon here. We save uh a bunch of of memory as we shrink the heat. Boom. It's down to 80 megabytes right now. So it's it's about half of the the uh memory that the older versions of Chrome took. This is huge and it's really it's out there in the wild right now. You and your users will feel that Chrome feels lighter and snappier because of this. So that's garbage collection improvements. But uh here's one that developers as developers you might be particularly excited about. We are spending time right now working on implementing ES 2015 in V8. So, ECMAScript is the body which determines what the JavaScript language is. And this last year they've announced a new specification with a bunch of really cool features. Got promises, proxies, arrow functions. Uh V8 has shipped a bunch of these features already. So, you can go out today and use classes a newer syntax and semantics for better object-oriented JavaScript programming. We've shipped arrow functions. Not only is it shorter to write this, you don't have to write those anonymous functions anymore, but uh it's much easier to deal with lexical binding of this. So the this in this case refers to the correct thing, whereas in previously when you're writing this longhand, you might have to bind the context of that inner set interval to the outer context. And we've shipped spread and rest operators. These just make it so much easier to uh write arguments that uh create that that take in uh something like an array. And we've staged a bunch more features. These are just two highlights, default parameters and dstructuring assignments, but we've got a bunch more in the pipeline. We are really committed to delivering ES 2015 in the browser. And like I mentioned, uh because we're doing this in Turboan, it's not just that we support these features, but we're really committed to optimizing them and making them fast, too. So, here's a project which is new and it's a bit of an experiment, but it's pretty exciting. Designing V8 is all about trade-offs. As you try to go faster, you grapple with increasing memory and having a a more computationally expensive CPU operations. But as Tall mentioned yesterday, increasingly a bunch of users on the web are browsing from what we call spelt devices or these Android phones or phones in general that have drastically reduced memory profiles. So while you may be using an iPhone or an Android Nexus in your pocket with 3 to 5 GB of memory, many of our users, in fact a huge number of them are browsing with just 512 megabytes of memory. This makes it really tough to perform optimizing compilations for JavaScript. So Ignition is an experiment to trade off a bit of speed. It's true. An interpreter is slightly slower than compiling code to native, but it is drastically reduced memory footprint. So already based on our experiments, this project ignition has shown that we can create a bite code from your JavaScript a that's three to four times smaller than unoptimized code than what we normally would run off the bat in a browser. And this is it's really will be noticeable for users on mobile devices that just are not uh as beefy as some of the things that Chrome or and V8 were originally built for. So really excited about this. But what's next? Uh briefly I want to talk about frameworks. V8's motto is to make all the JavaScript on the web fast. But uh how many of you use frameworks when you're when you're starting up a new website as the first thing to do to find a new framework? Usually it is. People use React, Angular, Ember, um, Polymer. All of these frameworks perform in different ways. They're often setting DOM objects rapidly. They might have a virtual DOM implementation. Some of them use an immutable pattern where they create a new object every time the object changes. And some of these APIs are designed to be ergonomic and really flexible and easy to use, but they turn under the hood into things we call polymorphic functions, which are bad for performance. So V8 historically has made these these types of applications with frameworks fast just by making all JavaScript fast. But next year we really want to dig down into the specific usage patterns that modern frameworks are are using and optimize for them in particular. So we want to reduce boot time of frameworks and we want to turn the patterns that they all use into really optimized uh code. So this is something that historically I think we haven't spent much time looking at but we're really excited to tackle this and we've been working internally on some advanced benchmarks which use uh real websites and are able to load them deterministically for our testing. So we'll we'll be testing on the web on real websites that of course are using frameworks. So this is what V8's been working on and what we'll start we'll keep working on. But there's another half to the equation and that's the code that you guys as developers are writing. Obviously, you know, to use efficient algorithms and you know optimize the the types of data types you use for something. But as Paul showed just earlier and as you've seen at this entire conference, there's times when you're optimizing for something like rail and you dig into dev tools and you see just the in the in the flame graph JavaScript execution taking up a massive amount of time and sometimes you can't do less. So the question is how can you write that same job JavaScript in a way that V8 can make faster? So, I've shown a bit how the V8 engine is like a Formula 1 car. It's this incredibly powerful machinery that allows you to move at blinding speeds, but uh could be finicky sometimes and more importantly over time its performance characteristic and performance um affordances change as we make evolve the engine and make changes to it. So, I'd like to think about how drivers in Formula 1 actually adapt to changing machinery like an F1 car. So, one of my favorite drivers is Senna, the three-time world champion. I actually showed this deck to my team and got uh 40 emails back with different names that I should mention instead, but I'm going to talk about Erin Senna. Uh, and I like to imagine what if he was a JavaScript developer, you know, how would he apply this knowledge of adapting to a new car to the browser and adapt to new versions of V8? I'm going to give four tips that I think are inspired by the sort of things that drivers do. The first tip is understand how modern engines work. Now, Erin Senna doesn't have a mechanical engineering degree. He doesn't neither should you read the source of V8. But having at a high level a sense of what's going under the hood is really useful. So I'd like to share two things which are fundamental enough that they're they're actually shared architecture across all of the JavaScript engines out there on the web because really you want to write code for everything. So this includes sunspider and fire uh excuse me spider monkey and firefox, JavaScript core and Safari and Chakra in Edge. Look at this code right here. I'm It's pretty simple. I'm just creating a a point class and instantiating a new object three times over. Well, I mentioned that the V8 engine is turning this dynamic code because remember in JavaScript your objects can have any number of properties and you can change the shape of them at any time. Under the hood, V8 needs to turn that into native machine code and it does this uh in a way that you can imagine as creating strrus like C like strrus for every object it sees. So when it sees this this point, it infers as the function is run or as this code is being run, it infers that that object is not this dynamic dictionary, but it's actually a strct that looks something like this. It has two properties X and Y and they're both ins. So when you create these three points, it knows that the shape of the object you're creating is the same and it optimizes it for that. So these three uh three points share the same object under the hood. This we call that a hidden class internally. But in JavaScript, you can dynamically set properties. So if I modify my third point to add a new attribute, a new property Z, under the hood, we have to go back. We notice this. We still support it obviously because it's valid JavaScript. But we have to undo that optimization where we said all points share the same hidden class. And we create a new hidden class. It looks something like this. It has an extra attribute Z. And this is really bad over time because we have a bunch of other optimizations that rely on there being the smallest number of hidden classes possible. Here's a quick example. Um, oh, and let me just show you how to fix this. You should always declare all of your properties in the object constructor because V8 can look at that and say, "Aha, all points will share these properties and I can optimize for that from the beginning." So, this is a much more effective pattern. I'll say that one more time. Declare all properties in object constructors. Don't dynamically add or remove them. Here's an example of why this is so important. If you have code like this, say a function twice, this function just adds the input together and returns it. In JavaScript, you could throw an integer at this and it would add them using math. You could throw a string at this and it would concatenate them. You can throw a bunch of different types at this. So when V8 sees this code here, it's a really hot for loop and it runs twice 10,000 times. 9,999 of those times the input to twice is in this case a string or excuse me an integer. Am I reading that right? A string. Um and it sees that. So it says, "Aha, we're going to optimize twice. It's going to be optimized for integers. Hooray." But then on the 8,000th time, youth send an integer down the the the chute and it says, "Well, we we tried to do this special optimization, but it broke. So, we're going to have to deop to a slower version." So, when I when I talk about integers and strings, what I really mean is these hidden classes that I was talking about before. JavaScript doesn't have types, but we think of objects as having particular shapes. So it's important that when you call a function, you use the same shape of arguments and returns so that V8 can turn them into highly optimized machine code versions and consistently use them without deoptimizing. Now this is called monomorphism. That's just a fancy word for it. Now I mentioned two highle parts of the architecture. These are unlikely to change because they're so core to how all modern JavaScript engines work. But I don't expect you and in fact it's an antiattern to go memorize all of the other uristics that V8 and JSC and Spider Monkey and Chakra use because they're changing so fast. So in general it's a bad idea to memorize rules and it's a much better idea to use the tools this suite of tools that Chrome and Chromium provide you and others. So, for example, the V8 engine has a headless version called D8. And if you use it, you can pass in certain flags and introspect what's going on under the hood. There's some information out there on the web if you want to do that. Uh Paul and and others today, all the Paul's have been showing you how to use dev tools and find flame graphs where you can dig into actually which functions are taking the most time. In fact, in the CPU profile page, you can see those same deoptimizations that I was talking about a couple slides earlier, you could see them because there's a little uh warning sign next to the function in DevTools on the CPU profile page. And there's a couple more interesting tools out there if you really want to dig into the nitty-gritty. Something called IR Hydra, which actually shows for any JavaScript how we turn it into native machine code. But I want to quickly give a brief demo of uh a little tip that you can you can do today probably without installing anything extra. So I mentioned D8 which is the headless version of node of uh of V8 but I actually don't have it installed. And if you're not a Chromium developer you probably don't have it installed either but I do have Node installed and Node embeds uh V8. In particular, Node has uh all of the V8 options available as flags that you can pass into Node when you execute something. Now, before I show you this demo, I want to make one important caveat. It's important to check the version of Node you're using because you want to be on the same version of Node uh as the one in the browser. So, if I run this, I can actually see what version of V8 it's included. It's 4.6. That's the same as Chrome version 46. That's what I'm running in in the browser. So check that the versions are the same. But with when I have node installed, I can pass in this flag called and this works for D8 too, but you probably have node already. Can pass in a flag called trace uh deop. And what this does is it executes a JavaScript file and it logs out when those those horrible deoptimizations that I talked about happen. So I'm going to run it on the code I showed in the slide. Remember this is this twice function which has a polymorphic function call. You pass in a string for a while and then an integer. And if I run this node and and V8 D8 under the hood will tell you that. Now, this is a bunch of of uh low-level assembly. But in particular, this first line here is really important. That's the one that's that we want to look for. Um it's actually GP for uh just the word deop. This will print out when it hits a particular function deop. So you can see here it says if you read it across JS function twice was deopted. You can imagine doing this on uh a bigger code a bigger piece of code uh let's say something that has 4,000 lines and counting how many deops happen. This is a good way to sort of figure out at a just by squinting at it kind of how optimized your code is or how much room there is how much room there is for improvement. Uh let's take a second to run. 12. Okay. So I know I can compare this to another body of code or change the code a bit and see if it deops. But let's apply the knowledge to this example. So I mentioned that polymorphic calls are bad where you pass in two types into the same same function. So what if we remove this if statement and just pass in a monomorphic. So in this version we do the same thing. We have this really hot loop, but we're only passing in one type of of object, one shape of object. It's a string. And this time V8 can uh optimize this and not hit those those deops and have to revert to a slower version. And you can see here it runs without a problem because there's no more deoptimization. So this is just a quick tip. Start with dev tools and the flame graph. But if you really want to look under the hood, you can use node. Uh okay. So that was two. Use tools. Don't memorize rules. What are some other things you can do? Well, definitely stay up to date. You can bet that Senna understood when the car got a new brake or when the engine was updated. He was talking with his engineers every day. Well, we have an easy channel for you to figure out what's going on in V8 under the hood and with developer facing improvements. It's the V8 Blogspot at v8 project.blogspot.com. So read this and see what's going on. And you can also read the change log in our source tree, although this is is more uh fundamental architectural changes and doesn't get updated quite as frequently. So the blogspot is a good place to start. And I really want to impart that V8 is constantly changing and we're constantly making improvements on it that are driven by the types of code that we see out there in the wild that you guys as developers are writing. And we can move faster with this feedback loop by just communicating more. So, if you've got code and you're not sure why it's slow, you know, you've you've looked at some of the dev tools, you've dug in and tried to remove things that are obvious, but it's still slow, file a bug. Because in V8 and all the modern JavaScript engines, reasonable code should run reasonably fast. Engage with the team on communication channels like Stack Overflow and in particular, you can send mail to V8 users at googlegroups.com. Talk with us. Show us new frameworks and libraries. If you are using the latest and greatest framework and for whatever reason you're seeing a long boot time or a bunch of deoptimizations when you shouldn't be, show us this framework. Show us this library. It helps us stay up to date with what developers are actually using. And finally, the biggest help is if you contribute benchmarks of real world apps for us to test against. Because the way we figure out if we're making good performance changes to V8 is by running benchmarks against it. And we want those benchmarks to be as close as possible to real apps. It's best if they actually are real world apps. So send us stuff that you use to test speed and we can work to optimize it. And finally, two antiatterns. I mentioned these earlier. Don't memorize hard and fast rules. There's some information floating around on the web about how V8 doesn't optimize try catch statements. Well, it's true. It's true today. But we're working actually we're about to ship optimized versions of try catch. So these characteristics change and if you just used a rule that you read one time on Twitter or on the internet you'll miss it when VA finally ships the improvement. And actually in Firefox they already have shipped it. So these things are changing so fast you can't rely on rules. You got to rely on tooling to figure out what's changed. And lastly don't use microbenchmarks to make design decisions. When I talk about benchmarking and profiling and running a performance audit, it's very important it's on your actual app and not some slim down version, it doesn't help to write a four-line JavaScript statement and try to figure out what way of iterating over an array is fastest because I can guarantee you that when that same array iteration happens in a 50,000line real world production app, a bunch of factors, the context will be different and you can't guarantee that V8 will perform the So, don't use microbenchmarks. And lastly, to bring it full circle, one of the things that I was so struck by Senna and these F1 drivers is how closely they work with the team that built the car. They constantly talked and the team really was just as essential in winning races as Senna was as a driver. So you are Senna, you're the developer, you are the driver, and the V8 engineers are this team of engineers working to make the machinery that runs your code faster. So let's work together. Help us help you reach that checkered flag. Thank you very much. Um, you can contact me at sethompson google.com and send mail to V8 users.com. Look forward to hearing from you and sharing more about the exciting things that V8's working on. Thank you.