Protocols and Tools for Testing Web Apps

Welcome to the Chrome Enterprise Technical Community Hour. Today I'll be talking about testing best practices for web apps. My name is Rich and I'll be your host for today's presentation. Joining me today we have our speaker Mateus Romer who's a developer relations engineer for Chrome testing. For today's agenda, I'll start with the quick introduction of the Chrome Enterprise Recommended Program and the Technical Community Hour. Then I'll hand it over to Mateus and he'll cover today's topic and the call to action for our Chrome partners. Lastly, we'll provide um some additional resources for you to learn more information. Today's Chrome Technical Community Hour is brought to you by the Chrome Enterprise Recommended Program, which is Google's partner program for third-party solutions that are optimized for Chrome OS or integrated with Chrome browser. This webinar brings you the opportunity to engage with our team about new new features and updates, enterprise development best practices, and our enterprise strategy. Now without further ado, I'll pass it over to Mateus and he'll kick us off. Mateus? Hi everyone. I'm Mateus and I work with Chrome's browser automation team. Uh just as Rich already said, I want to talk about best practices for web app testing. I'm sure some of you already are familiar with the topic or might even have dedicated testing teams with them. However, let's briefly look into why testing is important. Let's take the simplified web shop as an example. From the store's homepage to checkout, in this example, a visitor will have to go through four distinct pages until the purchase is complete. While going through the process, the customer needs to interact with the page, click on links, buttons, scroll up and down, and fill in forms. Looking at three of the globally most popular e-commerce websites, on average, the user needs to successfully complete at least 16 of such interactions to successfully complete their order. Now the challenge is each of those 16 interactions are a potential point of failure along the user journey. The layout could be broken making links or buttons hard or impossible to interact with. An API might not respond as expected, causing content to not show at all. Your client-side logic might not initialize correctly, preventing the page from becoming interactive. Or external dependencies, like a form validation library, might have changed unexpectedly, blocking user input. And that's only what could go wrong in this simplified example. In practice, a real-world website can have had hundreds of possible interactions across dozens of unique pages. Testing all of those journeys manually and meticulously on every release, including edge cases and side effects, is impractical and resource-intensive. And this is exactly where browser automation comes in. By automating testing within the browser, it enables accurate, reliable, and extensive testing. Now, let's look into how browser automation actually works. Browsers are automated by tools and there are a lot of them. Here is just a glimpse into the tooling landscape made up of tools like Puppeteer, Selenium, WebdriverIO, or BrowserStack. Finding the right tool in this sea of options can be difficult. That's what we constantly hear from developers as well. Developers just like Tommy. Tommy is a browser automation engineer and yes, Tommy does not have only three fingers on his left and four fingers on his right hand because I'm bad at drawing, but because he's AI generated. Here is Tommy the other night. He had a headache trying to pick the best tool for cross-browser automation. This interaction doesn't work with magic spells like Tommy's trying to cast them here, but protocols. The protocols used today are CDP, which is short for Chrome DevTools Protocol, and WebDriver. Some might be familiar with them, but for those of you who aren't, let's do a quick recap. The Chrome DevTools Protocol was, as the name suggests, initially designed and intended for Chrome DevTools to debug the browser's internals and make them accessible for developers. CDP runs on web sockets. The Chrome DevTools Protocol was never standardized, but also never meant to be. Nevertheless, Mozilla one point partially implemented CDP in Firefox anyway. So Firefox would be able to be automated with the same tools as Chrome, like with Puppeteer, for example. Later on, Puppeteer also became the foundation for Playwright. CDP is also a bidirectional protocol. This allows CDP-based tools to hook into browser events rather than polling and waiting for them to happen. Now, let's look into WebDriver, the other main protocol. WebDriver is really a major W3C standard with its roots dating back almost 20 years into the Selenium project. By now, it is implemented as a driver for every major browser. So there's ChromeDriver, GeckoDriver, EdgeDriver, and SafariDriver. Over time and by having its origin in the Selenium project, today there are language bindings for JavaScript, Python, Ruby, Java, and even C, making it easy to adopt by any developer. So with its age, WebDriver also has its problems. It cannot deal well or sometimes at all with newer browser features that rely on a deeper connection to the browser internals like extensions. And being based on HTTP, it's also primarily unidirectional, responding to a sent command, but never forwarding events. So to summarize, the Chrome DevTools Protocol is not standardized, which allows it to quickly change and be extended. But on the other hand, also requires tools that use it, like Puppeteer, to be kept up to date with the protocol. And as you heard, it's bidirectional, allowing it to receive events rather than waiting on them to happen. On the other side, there's WebDriver, an official major W3C standard that is supported by all browsers, but only unidirectional with a limited command set. As that, today both protocols are widely used by tools in the ecosystem. For example, our very own Puppeteer is a reference implementation of a tool automating Chromium-based browsers and Firefox via CDP. An idea that Playwright built upon and took three steps further by building patched CDP support for other browsers than Chrome. And then of course, there are all those tools, some of them around for ages, that have historically been based on WebDriver like WebdriverIO or Selenium, Sauce Labs, BrowserStack, and LambdaTest. To be fair, over time most of them also implemented features based on CDP in order to support more advanced automation use cases for their users in Chromium-based browsers at least. For example, Selenium started to support event-driven tests with version 4, also enabling those in downstream products like BrowserStack. However, that led exactly to Tommy's headaches mentioned because now whenever setting up a test suite, he needs to ask himself, "Which tool allows me to test the most use cases on the widest browser palette?" Today, for many of there is an obvious answer to this question. Playwright, of course, with its simple-to-use API and cross-browser support can be a great choice. As you know, it's using the Chrome DevTools Protocol to automate Chromium, Firefox, and WebKit, the same engine Safari is based on. This works because Playwright is patching the Chrome DevTools Protocol into the open-source versions of the browsers that do not originally have it, like Firefox and WebKit. This can lead to challenges. For example, if Tommy wants to test something on an older version of Firefox, this would not work because he cannot simply point Playwright to any Firefox binary, but only the ones shipped with Playwright. So in some circumstances, the question still stands, "Which tool allows me to test the most use cases on the widest set of browsers?" Something that just Tommy asked himself, but also we did together with all other browser vendors like Mozilla, Microsoft, and Apple, but also tooling vendors like WebdriverIO, Selenium, Sauce Labs, BrowserStack, and LambdaTest. So a few years back, we sat together and thought about how to combine the strengths of both protocols that are used in today's ecosystem, the old and major WebDriver and the handy Chrome DevTools Protocol. The result of this work is WebDriver BiDi, a new protocol to automate modern browsers with the same set of commands. WebDriver BiDi is specified as an open W3C standard and widely backed, not just by browser vendors. Just like CDP, it's bidirectional, so you do not need to poll for events like with the old WebDriver, but can listen for them instead as with CDP. Another great thing is WebDriver BiDi is designed for the modern web. It's meant to be extensible with new browser features able to be automated under the same spec. And by being based on a standard, there's an unmatched chance your automation runs in every browser to come without any changes. And last but not least, WebDriver BiDi brings the same granular low-level control that CDP brought to Chromium before to other browsers as well. To this point, this might all have sounded fairly familiar. But if that protocol by that is so amazing, why isn't everybody using it already? What's the catch? It's still work in progress, and we've been saying this for quite a while now. But you know what? This year, the wait actually comes to an end. The ecosystem actually made more progress over the last year than you might think. Selenium, WebDriver IO, and Puppeteer are all already supporting the new BiDi protocol. Using them, you can automate Firefox, Chrome, and Edge with the same powerful commands. While the Safari team is collaborating on and supporting the standard, the Safari browser for now only supports the former WebDriver standard. Now, let's have a look at some examples to see how WebDriver BiDi can improve your testing workflows. But first, here's how you can create a driver or respective browser instances in Selenium, WebDriver IO, and Puppeteer to automate Firefox via the WebDriver BiDi protocol. The same would work with Chrome. Those instances then have access to APIs based on WebDriver BiDi, allowing use cases that were not possible before. Let's look into some. I want to highlight three things. How APIs based on WebDriver BiDi allow to listen for log events. How to intercept requests, and how you can emulate web APIs. The cool thing is all of these use cases are or will be equally supported in all tools and browsers using WebDriver BiDi. No matter which version, channel, or environment. Now, after the first example, how to listen for log events. Let's assume the following code snippet. It's checking for WebGPU support and rendering a fallback in case it's not. To make it clear for testers what they are looking at, you emit a console warning in case the fallback is rendered. Wouldn't it be great to have this information available in your automated tests as well? As I said before, WebDriver BiDi is bidirectional and therefore now allows to listen for events such as log messages. This allows us to do something like the following using Puppeteer. You create a browser instance using the BiDi protocol. And then listen for console events with the familiar dot on set text. Inside your event handler, you then react to the event like, in this case, exiting the test since we're expecting to test an environment that supports WebGPU instead. Next use case in line is request interception. Even after a page has initially loaded, traffic, of course, does not come to a stop. For example, let's assume your app is using an external payment provider. Configuring everything correctly to hit a proper staging API instead in your tests can be difficult. The good thing is with WebDriver BiDi, you can now intercept requests and even modify responses in your automations. Here's how this would work in all Chrome, Firefox, and Edge with WebDriver IO using WebDriver BiDi in version 9 to be released soon. It works by creating a new mock, optionally limiting it to your URL path or a certain method like we do here. And then defining the intended response totally independent from your production back end or any complex back end mocks. And this brings us to our last example, emulating web APIs, and that's a personal favorite of mine. A while back in my career, me and my team were asked to implement the logic for a raffle. The idea was that to participate, people should use their phones to visit a certain website from the coldest place possible. And the hope, of course, was that extreme athletes would travel to adventurous locations and participate from there, creating excitement in the explorer community. But as you might have guessed, most participants did in fact not, but used location spoofing tools to virtually beam themselves to the coldest locations from the comfort of their home. Just as we did during testing. However, mocking hardware APIs was difficult back then and can still be today. But WebDriver BiDi makes it a little more convenient. Again, looking at an example from WebDriver IO using its browser emulate API. You need to specify the API you want to emulate, in this case, geolocation. And then, similar to a browser mock API, define your intended return value. In this case, we want to test a submission coming from the Mount Everest. The same browser emulate API can also be used to emulate a certain color scheme, useful for visual testing. You simply call browser emulate, this time with the keyword color scheme, and either light or dark as a value. And though on the brink of being phased out by browser vendors, user agent sniffing is still a common practice. So, the same API can be used to force a certain user agent even when using another browser. By now, you probably got the API's idea. You simply use the user agent keyword together with the desired value, and off you go. Now, those were just some examples of how WebDriver BiDi already improves testing in tools available today. And the ecosystem is really moving fast in enabling additional BiDi-based APIs on a regular basis. I think it's really cool how quickly tools like WebDriver IO, Selenium, Sauce Labs, BrowserStack, and LambdaTest have been adopting the standard and built APIs around it. So, make sure to check the docs of the tool you're using to see if there is something in your test that could be improved using the new protocol. Now, if you want to learn more about what my team is working on besides WebDriver BiDi, then have a look at our summary at google/chrometestingtools. Or just scan the QR code on the screen. And if you're just getting started with testing or have colleagues that do, I would love you to check out our recent course about testing on web.dev. And now, back to Rich. Thanks, Matthias. In closing, please visit the Chrome Enterprise Developer website for additional information to supplement your learning. That concludes today's presentation. We look forward to seeing you at the next webinar. Thanks for joining, and have a great day.