Upgrading DevTools' architecture to the modern web

Key Takeaways

hi my name is paul and i work as part of the chrome dev tools team today three of us me tim and jack are going to walk you through how we go about upgrading the devtools architecture to use modern web platform primitives and best practices so if you don't already know devtools is a web app and this means that like any other web app or pwa is built with html javascript and css however it's also a very large web app weighing in at around 150 000 lines of first party javascript along with a good number of third-party dependencies the other thing to say is that devtools is over 10 years old now some code was added to dev tools yesterday or even today but some was added over 10 years ago and this means by anybody's standard the devtools code base has a lot of legacy code and the goal in front of tim jack me and some others has been to modernize that legacy code and the kinds of migrations that we've done and in fact are still doing are these we've been migrating a custom module system to javascript modules sometimes called es modules or ecmascript modules we've been changing from a closure compiler managed type system to a typescript manage one we've been removing bespoke python build scripts in favor of using industry standard tooling like up and we've been moving from a custom non-standard component system to web components each of these migrations and others besides are very challenging to get right and so what we wanted to share with you is how we approach this kind of work and we're going to do that in three sections firstly tim is going to run you through planning changes secondly jack is going to talk to you about how you can make the changes well and finally i'll be back to talk about maintaining your changes the things in this talk the lessons if you will they're hard won and we hope that you will find that they help with your own code base but maybe you'll only be able to adopt one or two of these things well in any case let me hand over to tim to talk about planning changes so thanks paul for that introduction hi there i'm tim a software engineer on chrome devtools as paul mentioned we have performed several large migrations in devtools in this section i will be giving you an insight in how we've planned these migrations as you just heard devtools is web application with over 150 000 lines of first party javascript it's one of the largest web applications i have ever maintained and has an interesting mix of code patterns that accrued over time not only is devtools a large web application it also started over a decade ago when backbone became the most popular frontend library i personally was not a web developer back then and have not used backbone myself not only that death tools has not seen a complete rewrite in this decade and we're still using and maintaining code that was originally written in the first few years this means we are working with code patterns that were designed in a different era than today one interesting observation when working with these code patterns is that devtools had to build numerous custom solutions that were not part of the platform back then the platform did not have the features that we take for granted today therefore devtools gives an interesting perspective on what was necessary to build a large web application back then for example devtools designed a custom module format roughly eight years ago as there was no standard module format available this module format consisted of modular json files that specified the scripts that were part of a module and any dependencies the module had on other modules this module format works for devtools but no other web application was using it as such devtools built additional build infrastructure based on python scripts to process and bundle source files the main script for our build pipeline was build release application which was performing tasks comparable to bundlers that are well known in modern development today it is important to note that the devtools maintainers had to maintain this additional build infrastructure which is outside of our user work on features for devtools users yet we're also making the observation that the platform has significantly improved ever since these custom devtools solutions were built the platform now does offer a standardized module format and there are numerous book tools built on top of these standards so we came to the conclusion that maintaining unique build infrastructure module formats and other bespoke devtools solutions was expensive since any time we had to spend on maintaining infrastructure we could not spend on building features or fixing bugs for devtools users so our first takeaway is to use standards if you are able to since maintaining or using a custom solution becomes an expensive choice later it is important to stress the word can here as there might not be an appropriate standard available that you can use however when there is an appropriate standard available we advise you to favor that option over a custom solution for the long term as we said we've migrated away from the custom module format to the javascript module standard which we have written a detailed blog post about on the devtools engineering blog we will give you the high level details about this migration in this presentation but if you are interested in more detail i recommend reading the blog post and so after finishing that migration we are now able to use and ship devtools with javascript modules the imports and exports that you see on this slide here are probably more familiar to you than the module json format as shown before the migration was difficult at times and we've hit several roadblocks but we are now able to fully use javascript modules as such we are able to use the modern tooling that i mentioned before such as vs code and rollup and so instead of having the custom python script to bundle our resources we now use a small roll-up configuration to bundle our sword files instead this allowed us to remove quite a bit of technical depth in our build pipeline and simplify its approach while thus far i have talked about modules as paul alluded to before we're doing similar explorations in other areas such as widgets the widgets here are our ui components where widgets are the custom devtools component design to give you an example of what a widget looks like this is a code snippet i copied from our code base this particular widget describes the ability to add particular classes to a dom note in the elements panel these widgets are not only a custom component design they also include custom solutions for for example css where this register required css call registers an external style sheet that is included in this widget the registration process is directly integrated with the devtools runtime that handles how we load and bundle source files in devtools and is completely custom to devtools alone as i explained before devtools was built in the backbone era in that time frame components were primarily using imperative dom apis to build ui in this case devtools uses these apis to for example add a class to a dom node and lastly we have custom methods that live on the element prototype to build a tree of dom nodes on this line we're using the create child call which is added to the element prototype to create a dom node and automatically attach it to the parent node back in the day adding methods to the prototype was an appropriate solution but we have since realized that it is not a maintainable solution in the long term however this kind of technical depth is still what we are facing today in devtools and so for any of these large scale problems that you need to tackle with you need to plan and prototype your changes since these widgets are used throughout the code base you need to figure out a plan to methodically migrate the code base to a more modern pattern you will need to prototype your changes to make sure that your plan works when you scale it up to the broader codebase so when you're faced with these large scale changes we tend to plan these migrations using design looks design docks are common at google to discuss designs before we start executing them the design logs contain general information like who performs a migration which bug tracks its progress and what the value proposition of the migration is and they also contain a more detailed design and trade-offs that we made along the way it allows the team to discuss potential solutions and figure out additional improvements an important benefit of this process is it allows us to catch issues earlier and lowers the chances we run into problems later by writing out the design first we often figure out incompatibilities or solder problems which we can make tweaks for later often performing the migration that merely becomes the execution of the plan rather than having to solve problems you're facing on the fly what's important to note is that it usually requires intermediate steps to perform a large-scale migration these migrations on them own are very large which means that splitting up the work in discrete steps makes the project tractable this is even more important for devtools as devtools is part of chromium which ships a new version every day as canary this means that every single day devtools must work there cannot be a point where devtools is broken for a week as our users and fellow chromium engineers rely on devtools working and so when we perform these migrations we have to make sure the devtools remains working this means that the solutions we develop have to take into account both the old implementation and the new implementation that we are migrating towards now that is easier said than done because usually the old implementation you are migrating away from has limitations that warrant migration in the first place as such you are inherently constrained by the existing implementation and therefore you are limited in the possibilities of what you can do during the migration it is often the case that only after you finish the whole migration you are fully able to take advantage of the new solution now for a detailed explanation of what this would look like you can read the blog post on our engineering blog on migrating towards javascript modules it describes how we made both module formats work in tandem so that we can gradually migrate our code to javascript modules while other parts of the code base remained using the module json format now one catch to all of these migrations is that you get to the problem that at some point you get there since you're actively migrating it will take some time until you reach the point where you want to be but it is difficult to know when exactly you get to that point and so what we realized is that with these kind of migrations it's very difficult to know when you're finished for our migrations our estimates have almost always been wrong both positively but most of the time negatively for example for the javascript modules migration we originally estimated four weeks of migration this estimate was based on our preparations with scripts that would automatically transform our code to the new module format sadly we got nowhere close to that estimate and it took us seven months to complete the whole migration partly this was because javascript modules are a lot stricter and uncovered issues we were previously unaware of so we were able to find these issues but we are forced to apply additional fixes to keep devtools running and so ultimately while you can prepare it to the best of your ability you never know what you're running into and so the phrase you don't know what you don't know is appropriate for any work in this area do not feel discouraged when you made an estimate that turned out to be wrong in hindsight it is incredibly difficult to make estimations that are completely correct so for planning your changes here are the key takeaways listed feel free to pause the video and take a screenshot if you prefer after planning the next step is to execute the plan for that jack will now explain how we perform these kind of migrations thanks tim now we've heard about some of the planning let's dive into some of the challenges and things we've learned when putting those plans into action the first thing we found really important is to prove your assumptions in an early piece of work throughout the migration and however well you've planned it you will have made some assumptions about how certain pieces are going to fit together by finding a little piece of the code base that you can use to prove these you'll validate your approach early on or learn that you might need to make changes so for the web components migration on devtools we pick the breadcrumbs element as the element to migrate this element sits at the bottom of the elements pane and it shows the current active element all the way up to the html element this isn't a particularly exciting element and you're probably not going to even notice when we migrate it but what it was was the right size for us to prove some assumptions that we made about how this new migration was going to fit in with the legacy system it had enough complexity and some challenges around overflow and scroll to to challenge us and figure out solutions but it was also simple enough that it could be contained without growing into a much larger piece of work or spiraling into a huge change that would be really hard to land into our code base so from this what we've learned is to keep your changes small a migration by definition will often touch the majority of a system but landing all those changes in one go is really risky and it's a lot of changes to ship in one new version of an application if you can break your changes up into small pieces you're going to reduce the risk and also make it easier for your colleagues to review those changes because they'll come piece by piece rather than one big overwhelming chunk of code for the javascript modules migration that tim mentioned earlier there were 267 individual cls cl here is short for change list and you can think of it like a github pull request this meant that we could review those changes piece by piece also meant if anything unexpected went wrong we could just revert the small change we'd recently made rather than having to undo everything in one go there is a little bit of extra work here because you need to plan those changes up front and make sure that no one conflicts with anyone else we're making these small changes but it'll really pay off in terms of reducing the risk and keeping the code review manageable and if your change does ship and cause problems it's actually a really good opportunity to learn about the migration if it stays in the code base your approach is validated and you can continue however if there's any problems and you get it reverted that's a good opportunity to learn why it was reverted and figure out solutions for the next piece of work maybe you realize that you're missing some test coverage in a part of your application and you can add that coverage to make sure you catch problems like this in the future in any migration there will be situations where this happens you can't plan out everything and you can't predict exactly what's going to happen so don't worry if you have to undo some work and see it as an opportunity to learn we also found it really useful to gamify and track our progress over time migrations can be long and it can often feel like a bit of a slog particularly in the middle of a massive months or even years long piece of work if you can track it you'll have a much better sense of where you're at and also much better momentum to keep the migration going no migration is without its roadblocks you will hit problems even if you've done the best planning in the world there'll be something unexpected thrown at you it's most likely that some work lands then you have to undo it then you learn from that then there's another unexpected blocker that causes you a few days worth of delay there's always going to be road bumps along the way but if you can track your progress you'll keep the momentum high and understand how far you've got to go on the migration so for the typescript migration we created a chance to show how many lines of code we still had to do that's the dark blue line on this slide and also we were able to predict an end date which is the light blue line this let us visualize our progress but also made us realize that the predicted end date was far further in the future than we needed it to be but because we realized this we were able to prioritize the work a bit more highly get some more colleagues involved and we were able to take the end date down from november 2021 to december 2020. we also got a much needed boost of momentum because everyone was working really hard to bring that end date forward and we're all able now to look forward to that end date and keep working on the migration without putting that into a chart and having an estimate none of that would have happened and we'd probably still be on for finishing late into 2021. another thing we found really valuable is to use linting to avoid regressions if you're putting in the effort of migrating you want to ensure that at the same time no one is landing code that makes more work for you to do in the migration later on on a big team where some of you are migrating code and some of you are working on new features your teammates are almost certainly going to regress your code back to its pre-migrated state not on purpose but just because there's so much going on it's hard for them to keep track of everything a good example of this is a small migration we did to move from using assert.equal in our tests to either assert dot strict equal which uses three equals to compare the items or cert.deep equal which compares the structure of objects or arrays this was a pretty straightforward migration but it would be really easy for someone to introduce another assert.equal call into the test that we then need to migrate you can easily imagine yourself typing a cert dot your editor's just equal you accept that suggestion and you move on not realizing that you forgot to use either strict equal or deep equal we were able to add an es loop rule that would actually highlight these for us and fail the build this meant that no one could introduce this accidentally and it also means in code review we don't have to ask our colleagues to actually look for these problems because we know we have tooling that will do this for us we're a big team we ship code every single day and it would be very easy for things like this to slip through the net if you can add eslint or other tools to catch these problems you'll find that's a really good way to offload work from your colleagues reviewing changes and just trust your tooling if you do enforce migration with limp rules make sure you communicate clearly to the team what's going on and why those liberals are in place there's nothing more frustrating than having a change fail or build because of some eslint rule that you don't quite understand we're not quite sure why that's been put in place so what we do is we share documentation we add readmes to the code base the document the rules that we're using and why we're using them so that when someone does have an issue and when those rules fail in their build they can understand why and fix the problem linting is particularly valuable when you want to make sure you catch things that sometimes will work and are very easy to slip in without anyone realizing assert.equal is a good example of this most of the time it will actually work fine in your tests but there's the odd time where you really need to make sure you're using strict equal or deep equal so by catching this in the eslint rule we stop any test sneaking under the net that may cause us problems in the future if you would like to see how we use eslint and all the rules we apply you can actually do this online via the link in the slide finally i want to talk about dealing with roadblocks that we talked about earlier and coming up with the solutions for them you will hit these problems doesn't matter how well you've planned there will always be unaccepted problems but what matters more is how you deal with those and take which approach if you hit an unexpected blocker during some work you've got two ways that you can deal with that you can use a quick fix or a little hack if you like to work around the problem or you can pause the work and investigate the root cause and plan a more robust solution now in an ideal perfect world we would always reach for the robust solution but that might not always be the best thing to do any given solution you have to weigh up the risk the reward and the effort so if something's blocking you you have to wear how much risk there is involved is fixing that going to mean deep architectural changes across the whole code base that might be so risky that it's not worth it and a quick fix that works around it for now might be the better approach similarly there's the reward is it going to be a lot of effort for a small fix that just unblocks you and maybe causes your colleagues problems that doesn't sound like it is worth it but if it's going to unblock loads of issues that all your colleagues have faced then that might be worth it and along with the reward and the risk you also have to wear the effort is it going to take you weeks and weeks to figure this out or can you do it in a couple of days depending on these three things and balancing them you might decide that quick fix is actually a better solution here and it's important to bear in mind that temporary workarounds are rarely temporary every developer has landed the change and added a little to do or a little comment to come back and fix but then something changes different work comes up priorities change maybe there's an urgent bug to be fixed and that work gets left behind this is no one's fault it's just part of the work that we do in the devtools code base for example there are 783 to do comments that's a lot of work the developers intended to do but didn't often though this isn't a big problem these haven't been done because other bits of work that were more important have been done but you should bear in mind when landing what you think might be a quick fix that might only be in the code base for a week or so but actually it could stick around for months or even years so consider the impact when you're landing a quick fix and be prepared for it to sit around in the codebase for a long period of time to unblock your colleagues and yourself when working on migration it's also a really good idea to keep documentation for when you do hit these common issues that you might catch on our typescript migration there's a few issues that crop up time and time again we've created a document that people can refer to when they're working on the typescript migration this means the doctor asks for help and it means they can unblock themselves and use this as a handy reference guide it's also a really good way of sharing knowledge across the team when anyone hits a problem they can add it to this document and then we all are aware of it if we hit it again so here's a slide with all the things we've talked about making your changes if you want to screenshot that i'll take note of that and now i'm going to hand over to paul who's going to talk about maintaining your changes once the migration is finished thanks jack so it falls back to me to talk about maintaining your changes and right out of the gate we've got a takeaway for you that if it's not tested it's broken tests are they're table stakes for any project now it might be fine to have no tests if say it's a prototype or something but devtools isn't a prototype and it's a tool that people rely on every day to do their jobs and even simple code needs tests let me give you an example take this code from our code base it looks fairly straightforward it's this ensure enabled function if the thing is already enabled do an early return and if it's not call enable on agent which is an internal variable and then set the flag to true for next time and we might test it like this because we've decided we want to test things so i've called it food for simplicity uh and you can see we describe it and there's an it block and we create a new one ensure enabled and then we assert that it's true but not so fast because while testing is important bad tests if you have them are even worse than no tests at all and what i mean well it turns out that line that i just jumped over before this agent enable line it turns out that actually that returns a promise it looks like it's synchronous but it's in fact asynchronous work and so what we should have done is we should have written our function like this it should be an async function and we should await that agent enable call and correspondingly our test should look like this now if you have complex and complicated code it really pays to go slowly with writing your tests and examining any assumptions that you've got let's talk about our test types all right so testing is important but not all tests are right for all situations so for us business logic is something that we want to be able to import side effect free to call and confirm that it gives us the right kinds of outcome this means that our code is predictable and it provides a lot of confidence straight out of the gate sometimes you'll need to architect very specifically for that case for being able to unit test your logic end-to-end testing in the dev tools front end is the place where we actually follow our user flows and we literally click the buttons that you click and we observe that we get the right outcomes this protects us from testing the implementation and instead we care that the outcomes themselves are correct so we do this with puppeteer and a special form of the devtools front end called hosted mode uh which is where we can run the devtools front end in a browser tab and treat it like any other web app now be careful here though as end-to-end testing tests the whole system so it's inherently more likely to flake than say unit tests all right next up slightly contentious one boring code is the best code now fair warning what follows here is very subjective because i'm going to be showing code and i'm going to be talking about readability and that's something that we all have our own individual preferences on but here goes anyway so here's some code here again from our code base and i'm just going to point out a couple of things that might prevent someone from easily reading this or at least these are things that they would have to keep in their head if they were going to review this code and the more they have to do that the more cognitive overhead there will be so first there's this bind call here that means a copy of the function is being made and any references to this inside of that function will be set to the instance of the class then there's the function itself which is defined as an inner function but because of variable hoisting that function will be in scope when the bind call is executed so there's a few things there now you can think of this then if we say reduce everyone's cognitive overhead it might be to think of that in terms of applying to your teammates in code review or you can think of it as applying to you in a few weeks or months when you come back to the code or in my case a few minutes in any case the longer you spend decoding the intention the harder it will be for everyone so what could we have done differently with that code well let's start here start at the code we can see that we had a promise at the end of the the dispatch event so that bound function could be an async arrow function instead that would still return a promise if that's what we need and let's bring that one line that was in the old bound function bring that in there and that's it done so in this case making it more readable involves relatively few changes now i hasten to say this is not a case of code a is bad and code b is good you likely have your own preferences about what is readable but this is about tuning the code for team-wide readability and often code that doesn't require specialist knowledge is far more readable the bigger the code base the more necessary a type checker the next one and in fact another contentious claim now like we said earlier devtools has around 150 000 lines of javascript written by a range of developers over a long period of time and there is no way you can keep all of that in your head so naturally you'll need as much assistance as you can get type checking can help here and the first thing to say is that type checking is going to help with auto completion as you type out the code something like vs code will pop up prompts that will help you navigate the code more completely so it's a good thing but you have actually got a fair amount of options when it comes to how you do this one option is to inline your types like in the case of something like typescript that happens to be our chosen path because we found it to be expressive and that it provides high quality feedback if you do something like typescript though a downside is that you're committing to a build step but the upside is that the build step will enforce your types which can catch errors and that's typically speaking good but another option is to use js dock and you can still use typescript here to keep it in check here you won't need a build step but you are reliant on author time checking so something like the editor actually prompting the developer and that might be just fine for your project as i said there are options you could also use the closure compiler here too and run that as part of your build step loads of options similar to our linting point earlier though having something warn you as you author the code or as somebody else authors the code and certainly before somebody else reviews the code uh the better off it's going to be for everyone and certainly for us on the devtools team type checking has helped us and our team enormously and i would go so far as to say it's actually it's essential next one today's modern code is tomorrow's legacy code here's a set of features that you might recognize from es2015 to 2017 arrow functions classes async awaits the spread operator template streams there are more you've probably got your own set of things that you uh think of when you think of these kind of javascript features that are more recent however javascript's not done and there are more things being added so for example knowledge coalescing private methods wheat refs finalizers logical assignment operators again the list goes on and on so what do i mean then well let's come back to this code from earlier this is something that tim touched on when he was talking when this code was written arrow functions weren't a thing nor was async awaits so this would be pretty much the right way to write code like this but things have moved on so when you can update your code to use newer platform primitives it is worth considering if you're able to use asts to take old patterns and upgrade them to newer patterns then it will be all the better and that's certainly something we've done as we've been migrating the devtools code now of course if the code isn't broken and it has tests you might not wish to touch it at all in fact if it has tests you can update the syntax and easily confirm it still works as intended so there's something to think on so there you go those are my list items for maintaining your changes and you see the list here if you want to pause the video let me run into the conclusion and finish up this talk firstly there are always risks any change to code carries risks we're not in the business of removing the risk but we are in the business of reducing it and we can do that by using the things that we've talked about uh today secondly i want to say that this is an art not a science the things that we've talked about today some of them may work better or less well in various situations than with various teams knowing what and when to apply these things that we've discussed is part of the skill and something that simply comes through practice so let me leave this list here uh you may want to pause the video maybe take a screenshot if you found this talk helpful and with that i will say thank you to tim to jack and to you as well enjoy the rest of the conference