Build next-generation UIs with the HTML-in-Canvas API

Key Takeaways

[music] >> Hi everyone. My name is Thomas and today I'm here to introduce the HTML in Canvas API. This API lets you draw DOM content directly into a canvas or a WebGL and WebGPU texture while keeping the UI interactable, accessible, and hooked up to your favorite browser features. All the demos that you see here are using the HTML that you know and hopefully love in ways that you never thought possible. We'll get back to these, but first let me tell you a little bit more about this API. In this talk, we'll go over the basics of HTML in Canvas and why each is great and even better together. I'll then show you how you can start using the API today for your own project through some simple lines of code. And finally, I'll show off some more of those cool demos and show you how you can apply the API directly. To really understand the API, let's first look closely at the DOM and Canvas to understand their relative strengths. I'll use DOM and HTML somewhat interchangeably, but generally speaking HTML is the static source code you write while the DOM is the live interactive model the browser builds from that code. As you'll see, this API changes both the static HTML on the page and updates the live DOM structure to enable interactivity. The DOM is the staple of web UI and has incredible features hooked up through the browser. It offers great UI and text layout solutions out of the box that leverages semantically understood content to create rich and interactable interfaces. It enables users to do common operations across web pages including simple things we take for granted like selecting text to copy or right-clicking an image to save it. There are also a ton of browser integrations for features like accessibility, translate, find in page, reader mode, extensions, dark mode, browser zoom, and auto fill, which are all pivotal to how users interact with the web. Canvas, along with WebGL and WebGPU on the other hand, give developers low-level access to enable highly advanced 2D and 3D graphics. The power of canvas is that it is a low-level system that can use various rendering stacks like WebGPU or rendering frameworks to drive that grid of pixels. Games and many web apps like Google Docs that have complex UI need this performant low-level access. However, because the canvas is just a grid of pixels with some basic functionality, you'll need to handle a lot of logic like responsive text layout yourself or through a framework which adds to your bundle cost. Additionally, all these features I prattled off earlier for the DOM break completely when the UI is rendered into a canvas. Until now, that is. The HTML in canvas API is the bridge that gives you the best of both worlds by letting you paint DOM elements into a canvas or WebGPU texture. And the best part is that by putting the HTML inside the canvas tag and updating the transform, the content not only remains interactable, but all the browser features I mentioned stay hooked up, too. The API is available in Chrome today, and we're actively engaged with other browsers on the open standard and hope to see it shipping across browsers in the future. Using the API is done in three phases. Setting up your canvas, rendering into the canvas or into a texture, and finally updating the transform of the element to tell the browser where it actually is. Let's look at each of those steps through some code. The first thing to do is to set the layout subtree property on the canvas tag you're painting into. This causes the browser to be aware of the content in the canvas and sets it up to be hit tested and exposed to systems like accessibility and browser features. You can then just write the HTML code within the canvas tags such as adding a form element and an input and you can write this HTML directly as you're authoring or programmatically add it through JavaScript. Putting the HTML inside the canvas tags is a prerequisite to rendering it, but it is not sufficient. Rendering the element will require an explicit API call through a 2D context, WebGL, or WebGPU, which we'll take a look at next. For a 2D context, the call is draw element image, which takes the DOM element and an X and Y location of where to draw the element into the canvas. In this example, we're doing this through the on paint event, which triggers whenever the element is getting redrawn, including when the user or something like find in page highlights the text. This is the best way to make sure the UI is always updated, but you can do this wherever it makes sense for your application. The API method returns a transform that you can then apply to the CSS style transform property of the DOM object to let the browser know where the element is being rendered. This is so that interactivity and browser functionality works automatically with no additional integrations. It's vital that you update this transform whenever you repaint the element. If you stop painting the element, it's also critical to remove it from the canvas element, or else it'll keep being presented to the browser and accessibility systems as being part of the page. An important step is to set the width and height of the canvas to the right scale factor so that you avoid blurriness when rendering. To do this, you just need to make sure to add these few lines of code to set a resize observer. For WebGL, you still need to set the layout subtree property and add the HTML you wish to render inside of that canvas. To then paint the element into a texture, you'll use text element image 2D, which matches exactly to the text image 2D WebGL method and takes the same arguments except instead of an image source, you pass in a reference to the DOM element. For rendering using WebGPU, we also set up the canvas in the same way. Then we use the copy element image to texture method, which is similarly analogous to WebGPU's copy external image to texture method, which you may be familiar with. So, those two functions just paint the DOM element into a texture, but just like in the 2D example, it's also critical that we update the CSS transform of the element so that the browser knows where it is on the screen. Doing this in 3D space is a bit more complex and I won't show the full code here, but you can see it in this code sample. At a high level, we apply a viewport transformation to map the 3D coordinate spaces to the CSS coordinate space, and then use the canvas provided get element transform to get a transform we can then apply the same way that we did the 2D case. That's all a little difficult and confusing, but luckily that's why we have amazing frameworks to help do that heavy lifting so that you can focus on easily using the feature. I'm happy to share that 3.js has already landed experimental support for HTML and canvas, which you can dig into more at this link. Using the API with 3.js is very simple. You can create a material and then set the map of that material using the new 3.html texture method passing in the DOM element to be rendered. Then you just create a mesh using some geometry like a box and applying that material and boom. Another framework that has also landed support is the PlayCanvas framework which lets you build stunning HTML5 games and visualizations. The setup here is also very simple where you set up your canvas with layout subtree create an HTML element and then create a PlayCanvas texture. We then add an event listener that sets the source of that texture to be the HTML element we want to be rendered. Finally, we add an event listener that just uploads the texture whenever the browser repaints the element so that it always stays up to date. Now that we have the HTML texture, we can use it as the diffuse map of a material which we can then apply to a cube and once again, boom. It's worth highlighting that this example will also detect if the API isn't available and fall back to using a polyfill for rendering. While this doesn't hook up the browser features, it is always a best practice to provide a fallback for your own projects as well. You can see the full code sample at this link. For a more complex example, you can see the shoe demo where the style selector moves along with the cursor to give a stunning 3D effect while leveraging simple HTML for the UI. Of course, nowadays amazing coding tools like anti-gravity and a host of others can also help do a lot of this coding for you. Because this is a brand new API, a lot of coding tools won't know about the API unless you provide it with context. You can pass in the explainer link and suggest the coding tool to reference the examples, but even better is utilizing the modern web guidance linked here which will help the coding tool understand everything it needs to know for using the API. So, now that we know the basics of the API, let's walk through some examples to showcase the full power of the API in practice. All of these examples can be found on the page at this link. In this first example, we have some text being rendered both right to left and up and down. And if we look over here into DevTools, we can see that it is indeed inside of the canvas element itself. And we can still do things like drag the image around or right click to save it. And there's even a little SVG working there as well. Accessibility information is now also exposed, even though this element is inside of the canvas. And if we come over here into the accessibility tree view, you can see all of that content which would be accessible to, for example, a screen reader. And if we go over here into Lighthouse, we can see that we can even get an accessibility score of 100, even though all of that content is being put inside of the canvas. In a slightly more practical world, we have this simple WebGL example where you can see this UI is painted into a texture. But I can actually still utilize the text, utilize the slider, or input text myself. And then because it's all just hooked up through DevTools, I can even right click on this and say inspect to see this UI right here. And I can actually even say, "Well, this is actually a super simple control." And you can see that the UI will just update there automatically, while staying fully interactive and exposed to all the other browser features as well. And finally, if your CSS respects dark mode, then you can actually go in here and say update your browser customization to dark mode, and the UI will just update accordingly. Now, if you wanted something that looked a little bit more like a real billboard, you could also make something like this. And this is cool because this little pencil that you see here is actually an animated SVG that is now being rendered not only into the canvas as an animated SVG, but directly into the WebGL texture itself. And so now you can just play these SVGs without any kind of additional code, and of course all of the text and interactivity remains fully functional as well. If you wanted to have something like a visually stunning overlay utilizing a 3D model, you can also do that. You can see here that you can still utilize the slider and it'll actually refract through the 3D model itself. And similarly, if I wanted to do something like find in page, I can go in here and search for a specimen. And you can see that even that piece here will also refract through the 3D model appropriately. You can also do this really neat text distortion effect as you drag the mouse around and learn about this experimental API for rendering HTML directly into a WebGL texture. And this again just works as basic UI. And if you wanted to, you can even select the text and see that the effect is being applied to that selection of text. And if you wanted to again find in page, all of this just works automatically and hooked up through the same HTML that you know and love. Now we have perhaps again just a slightly more practical example of something that you might actually want to do. Here you can see a kind of 3D book that has all been rendered using WebGL. And again, this all just works as text being laid out thanks to the HTML in canvas API. Because this is all just laid out with the DOM and HTML, I can even go in here and decide I maybe want a more classical font to write Sherlock Holmes in, and it'll just update in there directly. Then I say, "Okay, maybe that isn't really too easy for me to read, so I want something more like blocky and funny looking." And can select out here this different text and boom, it all just updates that font and you can really just choose anything that you want. So we'll go ahead and switch it back to a more normal font so that we can kind of see what we're looking at. And now I want to talk about perhaps one of my own personal favorite parts of this API. Speaking as somebody who lives in a country where I don't speak the language perfectly, a really important feature for me is the browser provided translate. And so I'm able to go in here and just use that translation to say translate this over into German and read The Adventures of Sherlock Holmes. And it just automatically updates all of that different text. That means that I can now have these beautiful 3D scenes while still letting my users pick any language that they want to experience that in. Now, it wouldn't be Google I/O unless I also had the chance to at least somewhere mention AI, and that's actually another place where this API comes in handy. Because all of this is just text exposed in the HTML tree, something like an indexing crawler or an AI agent would be able to engage with all of it as just text. So let's say I wanted my agent to extract all of the text in all of these different pages rather than having to leaf through them. Maybe I wanted to read it out loud to me. I can then go in and have something like a agent or even a simple extension like a text extractor use that as an access to text directly. Because it's all just text, my cell phone agent can then copy out this text maybe somewhere to read it out loud. This last example is actually a WebGPU example, and here we can see this fun jelly slider that some of you may have seen before, but now it actually refracts fully and correctly with this little member right here. And of course, since it's all just hooked up through an input slider, we can even go over here into the dev tools and say changing the step from a one to a five, and boom, it automatically updates so that rounds to the nearest five and is still very fun and engaging while of course also staying hooked up to those browser features. Even in this early stage of the API, it's already getting some amazing apps like Figma, Adobe, Miro, and JetBrains who have all started prototyping and exploring what the API can enable for their web apps. As of May 2026, the API is in origin trial, which means you can already experiment with deploying it to production in Chrome, but you will have to sign up your origin at this link to enable the API. So, hopefully that gives you an overall sense of the API, how to use it, and what it enables for you to create. So, now all that's left is for you to go and create something amazing, and of course, share what you make. We have an awesome HTML and Canvas Readme file where we're hoping to keep track of some of the incredible things people are making, and there's a submission link for you to include your own amazing creations or finds. And with that, I just want to say thank you for listening today, and I can't wait to see some of the amazing ways that you'll use this API. Thank you. >> [music]