WebMCP Explained: AI Agents for APS Viewer Apps (No MCP Server Needed)

Hi everyone. My name is Peter Broz. I'm a developer advocate at Autodesk. And in this video, I want to share with you something very new and very exciting, Web MCP. A new draft proposal providing model context protocol to browsers. If you're building APS applications using the viewer SDK, this opens up a powerful new possibility. Letting AI agents interact with your viewer directly in the browser safely and in a structured way. In this video, I'll give you a quick overview of what Web MCP is and how it compares to the model context protocol. I'll show you how tools are registered. I'll add Web MCP to our simple viewer code sample. And I'll demonstrate the Web MCP functionality in action using Chrome Canary and the MCP Inspector extension. Let's dive in. Web MCP is a native browser adaptation of model context protocol. If you're not familiar with model context protocol, I will provide some links in the description under this video. But at the high level, this is a a protocol that was introduced by Anthropic in November 2024 and it has seen huge adoption since then. It at the high level it describes MCP servers and clients in the following way. MCP servers expose functionality such as tools or resources to clients. Um tools can be used to trigger certain actions and resources are typically used to access information. And then on the other side, MCP clients such as Claude, ChatGPT, or GitHub Copilot can connect to these MCP servers, invoke tools, and access resources exposed by the MCP server. Uh now, the MCP servers can either run locally using something called standard IO transport, or they can be hosted in a cloud and accessed through streamable HTTP. WebMCP brings these types of ideas directly into the browser. Uh it lets your web application expose certain kinds of tools, and these tools can then be discovered and invoked by AI agents running directly in the browser. Um there is no need for the agents to try and parse the DOM, the HTML, or take screenshots of your web application trying to understand if there are any buttons. Your application gives them a structured description of actions that the agents can take. This may be very useful in web applications that have a complex internal state, as is often times the case in applications built with Autodesk Platform Services and the viewer SDK. WebMCP is still a draft specification, currently being incubated under the Web Machine Learning Community Group. However, Chrome Canary already includes experimental support, and there is an official WebMCP Inspector extension, as we will show in a moment. So, experimental, evolving, but very promising. At a high level, WebMCP works like this. In your web application, you first check whether the model context object is available under window.navigator to check whether WebMCP is supported by the current browser. And if If is supported, you can start registering tools providing name, description, optionally an input schema or JSON schema defining the inputs that your tool expects and an async handler with the actual logic. The browser will then expose these tools to compatible AI agents and they can call the tools and your app will execute the custom logic. So, if you worked with model context protocol before, this will feel familiar. Except now, it runs directly in the browser and you don't need a custom MCP server. Now, let's see how we could add Web MCP to an APS application. To demonstrate this, we will use the simple viewer code sample that's part of our getting started tutorials and we will add two tools to our application. One for searching elements in the viewer by their property values and one to isolate elements based on their IDs. And later, we will show this functionality in Chrome Canary using the model context tool inspector extension. Let's get to it. All right, we have the simple viewer code sample loaded in VS code. So, let's add Web MCP to it. I'll open the main JS file where we keep the client-side logic for our application and inside the init viewer function, because we actually will need the viewer instance in our tools, we can start defining our Web MCP tools. First of all, let's check if the model context object is available, making sure that Web MCP is actually supported. And if it is, we can register our tools. navigator model context register tool All right, so the register tool method expects a single object with a bunch of parameters. Name of a tool. So, in this case, we can let's say our first tool will be one that lets agents search elements based on some text in their properties. So, let's call it search um elements by property. I'm getting some intellisense here from GitHub Copilot. Next field is the description, which again will make it easier for agents to decide whether a specific tool is suitable for a task that they've been given. But, this could be something like search design elements by their properties. Okay. Next, we need an input schema defining the data that our tool expects. In this case, we really just expect a text, a piece of string to search in the design metadata. So, we will define the input schema as a as an object with a single field called text. And the text field will be of type string, and we can also add some description to to the input as well. Again, making it easier for agents to understand what this tool is and how it should be called. Let's also say that the text field of our input object is required. Right? And finally, the execute um a sync a sync handler for when our tool is invoked. This will be passed our input object, so we can directly de-structure the text field out of it and use it to actually search for the text inside the design metadata. Now, the viewer provides a search method that accepts at least three arguments: the text to search for, a function to call when the search was successful, and a function to call if the search failed for whatever reason. So, we can't await that viewer search method directly. Um so, instead, what we want to do is say something like object IDs equals wait new promise and call the viewer search inside the promise, passing in first argument the text we want to search for, and second is the success callback. So, in this case, we can just use the resolve of the promise, and the third argument is the error callback. So, in this case, we can call the reject of the promise. Okay. So, we now have our list of IDs um retrieved from the viewer SDK, and we want to return them back to the agent who called this tool. So, we do that by returning an object with a single field called content. And content is a list of results from this tool calls. In this case, we will just return a single result, which will be of type text, and the text will say something like text found in elements with IDs, and then we actually include the numbers, the the object IDs in the response back to the agent. All right. That's our first tool, and let's add one more. More in context, register tool. And in this case, let's add a tool that will allow agents to isolate specific elements in the viewer by their IDs. We can call this isolate elements. The description can be something like isolate design elements by their IDs. And again, we need to specify the input schema, which in this case we expect a an array of numbers, the IDs of the design design elements we want to isolate in the viewer. So, let's say where the input schema is once again type object with a single property, let's call it IDs. And this will be of type array, and each item in the array will be of type number. And you can see again, I'm getting some nice recommendations here. Okay. And the IDs field will be required. All right. Now, in this case the execute handler will be somewhat easier. We can just call directly viewer. isolate to isolate the elements in the viewer, and we can also add something like fit to view to even zoom in the camera to focus on these specific elements. Um again, the tool execution should return some kind of answer. In this case, it may may not be super valuable, but let's just return um some simple message back to the agent, something like we've isolated isolated certain number of elements. That's it. All right. There's our two tools, so let's try them out. Uh as we mentioned earlier for this uh for now, the Web MCP draft is currently supported as an experiment in Chrome Canary. So, let me start my application in VS Code first. Okay, it's running. And let's open Chrome Canary. And actually, before you do that, what you want to do is first install the extension called Model Context Tool Inspector. You'll find it in the Chrome Web Store. And if you read the the overview here, they also make sure that they're asking you to enable Web MCP flag in Chrome Canary. So, that's if you go to Chrome {slash}{slash} flags, um you will see that the Web MCP Web MCP for testing is right there at the top. In my case, I've already enabled it. And I have my and I have my Model Context Tool Inspector extension installed. Okay, so with that, let me open the simple viewer application that's running locally on my machine. Wait for some model to load, and then activate the Model Context Tool Inspector. So, now you can already see it gives us a an overview of the tools that our web application implements. So, in this case, that's the Search Elements by Property and Isolate Elements tools. And we can also call the tools from here directly. So, if I pick the Search Elements by Property tool, you can see that it expects a single field called text. Let's search for something like HRU because I know there are some heat recovery units in this specific HVAC design. And we're getting back a bunch of IDs for elements that contain the word or HRU in their property somewhere. So, I can select these IDs and switch to the Isolate Elements tool because this tool requires a list of numbers, and I can pass just paste the list of IDs from the earlier tool call in here. And that will isolate the heat recovery units and zoom the camera in on them. Now, if you wanted to, the model context tool inspector also gives you a way to enter your Gemini API key and actually talk to an LLM that will also see these tools and call them as it sees fit. So, I hope you see the benefits of Web MCP because specifically for APS applications, right? You can now give AI copilots access directly to your viewer applications without having to develop any kind of MCP server with rest end points or posting messages back and forth. Here, the AI agents are talking directly to your viewer application. This enables things like natural language interaction with your BIM or CAD data inside your APS application and also provides, you know, safe and structured access to um the functionality provided by the viewer SDK or your web application in general. Um as Web MCP evolves, this could become a foundational pattern for AI native web applications. Now, to wrap up, as we mentioned, uh repeating one more time, Web MCP is still in draft stage, but it's already usable in Chrome Canary and it pairs beautifully with APS viewer-based applications as you've just seen. So, if you're experimenting with AI in your viewer workflows, this is definitely something to explore. Let me know what you build with it. I'd love to see it. And thanks for watching. And if you find this video helpful, don't forget to subscribe to the Autodesk developer channel for more content like this.