Leveraging AI Tools in Workspace Development

Key Takeaways

As uh PR said, I'm on the developer relations team for Google Workspace and um I'm going to come over cover some of the basics here of like vibe coding, AI tools, some of the problems you run into and so on. The kind of the thesis here for my talk is that you know to move beyond basic prompts and you know build more robust applications, you have to understand kind of the foundations of these AI tools. Um they're constantly changing but they have you know the basic core pillars there to kind of understand. Um and as part of that we've probably all heard the term vibe coding um you know the generate this app that does this amazing business case um in one kind of prompt um doesn't necessarily work and it kind of becomes controversial in that way. There's different ways of kind of looking at this term. Um for me you know it's generally basic the concept of like directing AI with natural language to do a thing instead of writing the code on the left on the right we have you know move all the rows with the status and column D is completed very different kind of um workflow for your code and for um workspace development. Um but natural language is not enough. Um, many of you have probably tried to do something with the AI tools and it spits out something that's completely gibberish or doesn't realize what you're actually working on. And so you have to add in some context to basically, you know, get that relevant code. So I'm going to talk about the kind of context problem, but first we'll go back to workspace development. Um probably everyone in here can probably add to this list of problems and challenges with workspace development. And so these same problems are applicable to the AI tools. And so we have to improve our you know docs and our use of AI tools um for both the human developer and the AI developer. You know AI tool you'll maybe see it generating JavaScript code or trying to call like a normal fetch call instead of URL fetch app. Um, you see that probably pretty often. It'll make up, you know, methods that don't actually exist. I look at a lot of the issue tracker issues that are submitted, for example, AppScript. And, you know, at the bottom here, you can see a little graph that shows that type error is becoming a very popular error type submitted for Appcript issues. And that's because it's an AI hallucinating basically a method in Appcript that doesn't actually exist, isn't in the docs or anything. So while that tool helps that developer that maybe or non-developer write appcript code, um it, you know, gives them a power that they didn't really have before. And so what we want to do is kind of move beyond that code that is maybe brittle or is full of bugs and you know make it more robust. Basically as part of doing that um we need to kind of know the different tools that we're using here. I think there was a bunch of people that raised their hands for various things on this list. Uh it's been my experience talking with a lot of people. It's especially like appcript developers. They're still using, you know, the chat app, talking there, copying and pasting code back and forth. They maybe haven't used the CLI tools like Gemini CLI that are more specialized to development. They're just using kind of the, you know, the AI chatbot. Um, and there's all kinds of versions of these at kind of all the different layers here. Regardless of which tool, whether it's the, you know, the chat app, the CLI app, they're kind of all in the same foundation of kind of context, you know, like the persona like Jasper was talking about. um permissions or maybe you call that tools what's available to that uh you know agent or tool or you know whatever it can do basically the permissions you've given it and then the kind the orchestration patterns does it run asynchronously in the background does it going off to you know do some long deep thinking process so back to the context problem you know how do we provide the information to go with our natural language prompt actually accomplish something. There are a couple kind of ways to solve this is one is like project specific guidance. These are rules files that you may have that describe the frameworks, the tools, um other information about your project. Uh another part of this problem is actually pulling in the most up-to-date information. Um the models may have been trained, you know, 6 months ago. There could be a new API available, something deprecated, something that no longer works the same way. And so, how do we get that recent information into those same tools to the project specific guidance? Many of you have maybe seen like gemini.md files. There's agents.md is a new pattern to kind of make it more universal for different AI LMS and tools in your coding uh environments. Um, and so, you know, you could have your persona as Jasper had names for all his that we just talked about. Um, you could also have, you know, your kind of your business logic and business rules and so on. I don't really ever write these files. I just basically use this the AI tool to basically look at my repository, investigate all the, you know, the frameworks and the patterns that I already am using and to, you know, create a document about that. Uh I emitted some of the actual details from the generated file here just to keep the the titles. Um you know another thing you can do is tell it to keep updating this as it learns. So you know when you go do an action or you know you write a prompt that says uh you know implement this feature. Maybe it'll learn something about the codebase as you're interacting with it and it would go append that to another markdown document. So it kind of keeps its own library um using Jasper's term again of you know what it's learned. Another useful kind of pattern to use here is llm's.ext. Um a lot of frameworks and websites even have these files available which basically provide a markdown version of you know the resources that are available whether it's a list of websites the um in terms of like coding libraries it might be the entire reference docs of a certain library or framework that you can just copy and paste and put into your you know git repository as a markdown file. Um for our developer doc democ do documentation for workspace uh we have the um llm.ext which is a list of every single page in that directory with um not always the greatest description to go with it in that markdown document. We're working on that. Um, and then there's also for every kind of leaf node in the documentation, there's a a markdown version of it. Um, eventually there'll probably be a little copy button. So, you can just copy the markdown of that reference page and you can use that to, you know, insert that specific context into your um, development environment too. Getting back to the live and you know more accurate data um you know APIs change as I mentioned earlier uh some LLM tools such as Gemini include you know grounding in some ways you might have to turn that on um another pattern is to use MCP server uh MCP uh model context protocol is a um standard tool pattern for AI agents to call tools access resources and kind of just interact in a kind of a standard predictable way. We now have a Google Workspace developer MCP server. So this isn't interacting with say Gmail or anything but it is um providing access to the developer documentation um for Google Workspace. So you could be in Gemini CLI and you know asking about something in workspace um a specific API and it could go and search the documentation and get the most uh recent results back. Here's a little example about this um asking Gemini CLI to create a joke about AppScript. Don't use any tools. Don't go do anything. Just use the information it has. And it comes up with a joke. Why did Appcript function break up with infinite loop? because it couldn't handle a commitment longer than 6 minutes. Kind of it's okay. It's nothing special. We all know that 6 minutes is a limit for Appcript for the most part. Uh and so I ask it again to um use the search tool to kind of look through all the developer documentation to find a better joke. And you can see here it's making calls to the MCP server and just going through the docs. search strategies for the docs, assessing joke potential, exploring error themes. And so it it has a joke here. It says, "Why do AppScript developers have trust issues? Because even their commented out code asks for permissions." So this is a little bit more of a hidden issue that I identified in the documentation that if you write in a certain kind of string pattern in your document your commented code in appcript it's going to request you know drive permission or whatever scope um even if you don't actually use it in code. These tools though have kind of a little bit of a double-edged sword. If you have hundreds of these MP MCP servers, each with many tools, it the AI tool isn't going to necessarily know should I use this one or should I use some other one. Uh if any of them return incorrect information, you know, it's going to kind of poison that context and you're going to get errors and then, you know, as your context window fills up, it kind of loses the objective of what you're trying to accomplish. Um, one way around this context problem is to, you know, use orchestration. This kind of gets into, you know, Jasper's talk around like there's a sub agent that only focuses on this thing. It has this context, these rules and this objective. Uh, this helps to kind of, you know, manage that context and kind of delegate correctly. Um you can even set this up in a pattern where you have you know instead of Jasper saying you know this agent go do this this one do this you have one that is like organized in all of those in the orchestrator itself and uh you know there's you know for example some of the VS code extensions they have modes and they'll have that pattern built in already. Another kind of tool kind of in this orchestration and more asynchronous is using jewels. So Jules now has a CLI that you can just you know prompt and send off a new session which will go and eventually create a you know pull request in a GitHub repository. So here I asked it to uh create a slide in the slide deck and it went off and did its thing and here was the slide it decided to create based upon looking at the existing slides I had. I didn't try to use the slides API with AI. So I I did try but this was not using that. Um and so here's the Jules interface where you can kind of go and see what's going on and you know talk to the AI um and iterate and of course it created a pull request in GitHub and Gemini code assist was there to kind of provide a review. So there's, you know, layers of these tools, each with their kind of objective. And yeah, it's basically very exhausting keeping up with these. I think, you know, some of this is maybe a little out of date. There's, you know, skills now. And, you know, yeah, Gemini extensions. When I first did this slide, the night before, I had to go edit another slide for talking about the Gemini CLI extensions. um things are changing all the time and so how do you deal with that? You know, it's I originally wrote this really complex like eval system where I was like trying to test with and without an MCP server for these different use cases in um Google Workspace development. uh use an AI to then like judge the outputs and rank them and you know it's there's so much non-determinism it's really hard to actually write a proper eval is the lesson I learned there um too much noise without spending thousands of dollars of just running AI over and over in loops um and so you know some of it is just back to the you know vibes how does it feel for your development how does it you know kind of work for Uh and then just knowing you know what use cases you're trying to do. Uh explaining or summarizing code it's great finding and fixing bugs maybe depends on the bug. Uh writing unit tests it can be pretty good you know kind of constrain things like that. Uh you know [clears throat] building an application from scratch maybe not so much for workspace itself. One of the things you might want to try when you have AI interacting with workspace AIS is creating a layer of abstraction. Um, instead of trying to like I did have the AI interact with the entire slides API, you could have something that's like a a tool or a function that is, you know, update the title of a slide or change layout of slide. Um, simplify the interface for the AI tools. Uh that's kind of more of the if you have an agent interacting with a workspace AI API also you know having goals when you have these prompts you know expand or fix the implementation as functions so that all unit tests pass you know that's a pretty easy one for it to kind of manage it'll try to make a change see if the test pass and it stays in that loop you know using you know rules files and personas to kind of separate different types of um projects and actions and prompts that you want to take on. Um and then you know just asking to verify for example if you're working on a appcript API that maybe you know you know new bean or whatever that you maybe advanced service that you're not familiar with you could you know ask the AI to verify it and for example here's a case of verify it and use the documentation so to go and look at the functionality search the docs and kind of go through that process iterative iteratively and then know the limits um use it more as a pair than maybe a replacement. Yeah. And you know embrace vibe coding, understand the fun, the foundations, you know, context, permissions or tools and then orchestration. uh evaluate and then you know you know build the or use the purpose-built tools Gemini CLI maybe more than say the chat app and maybe that means you are moving to a different editor where you're using clasp and Gemini CLI versus you know just the Gemini app itself and copy and pasting back and forth for kind of next steps like I mentioned uh there's a workspace MCP server available it's a remote one so there's no running locally You might want to say trust. It's a read only. It just reads the docs. Also, there's now a Gemini CLI extension for uh Google Workspace development and it basically includes this, but it's easier to install with just like this. Potentially also a VS code extension that also includes MCP server. So, you can have it in different surfaces. For example, the gas fakes Gemini extension also already had uh this MCP server installed. If you install conflicting versions, it's smart enough to kind of figure out which one is there and which one needs to be added to your settings. And here's some QR codes for those different parts, this slide deck itself, MCP instructions if you want to add those to your development environment, that CLI extension, and the VS Code extension. The last two here are on GitHub. So if you have anything you want to add or any other features, let me know.