AI Tasks Ep. 2 || Summarize and Categorize

Um, so again, by the end of this live stream, you're going to understand how to use these AI task features. Um, you'll be able to master those features. Um, and this week in particular, we're talking about summarize and categorize. Um, that'll help automate your data entry. Cam's going to show you how to configure some bots, um, as that's where the AI task is, uh, set up. and then we'll learn how to integrate those and scale those into your apps that you're working on. All right. So, with that, I'm going to toss it over to Cam for the explanation on these different categories of AI tasks. >> Yeah, for sure. Um, last week we had a webinar covering the extract and extract rows. And essentially what that is is um within app sheet there's bots um and these bots are automations that are triggered by an event whether that's um a data change or that is um an email being sent in or um whatever you would want to trigger an automation um and then extract and extract rows. It's part of the AI task suite um of Google putting Google's Gemini into app sheet natively, no code, easy to set up. Um we were able to show you guys how easy it was to set up last week or two weeks ago. Um but this week we're going to cover categorize and summarize as well um down low. And so super excited about those two. Um, but to cover the extract and extract rows real quick, those two depend on you uploading an image or a PDF. And based off that image or PDF, Extract will look at that image and fill out X amount of fields that you select that you want it to fill out. And so if you have a a PDF of a legal document, a contract, um it can extract the um the title of it. It can extract things like um who are the parties involved in this contract, what is the total value of the contract invoice. Um and maybe it can also do some inference as well. That's not just OCR grabbing data from the paper, but I can also do some inference and being like, oh, from this contract, these are clauses that might not be friendly to you, the receiving party. Um and so it's able to do a combination of using AI to do OCR and reading the document abstra like extracting things from the document but also using the inference of Gemini powered by that uh that extraction that it's making. Um extract rows is very similar but instead of just filling out one record you take a picture or a PDF and then it creates multiple rows from that. Um we were able to showcase last uh two weeks ago um me using vinyl records and on the back of a lot of these vinyl records there's uh song lists and we were able to create a song record that is related a child record to the parent vinyl record and create a record for every single song capturing the characteristics of each of those songs with it like what disc it's on in the vinyl record what order it is inside the disc and so one. But finally, if I go ahead and click to this next slide, I don't know if I have control over that. There we go. Um, we will be focusing on categorize and summarize. And so, categorize and summarize, they look at your textbased columns rather than images or PDFs. So it looks at your it looks at many of your textbased columns and it will fit your maintenance ticket in this situation what we will be covering into a priority or it would fit your vinyl record into a genre or your image into a color grouping or something like that. it it takes a group of um text values and creates a a category um from that. It's able to use your enum fields. That way you have predetermined categories for it to fall into. And then summarize takes multiple multiple text fields also and will take the um the aggregation of all that data. it'll understand it and spit out a summary that you're able to kind of uh point in the right direction with a a static text field um to kind of point in the direction of what you're wanting out of the summaries. But yeah, you can go ahead and click the next screen. I don't know if it's needed, but yeah, here's the demo. So, I'll go ahead and screen share my side. >> Nice. >> All right, let me zoom in that way. It's nice and uh friendly for you guys. All right, so welcome to App Sheet. If you see a sidebar right here, that's just me using Arc Browser. Um, but right here we have a maintenance tracker app. And the first thing that I'll show you guys is that it is using AI extract. And so we're using AI extract for us to upload an image. And so we go ahead and click this. and we'll just do this broken AC. So, right now, um the office place is really hot. It's 87°, let's say. Um this is affecting the full building. And we'll go ahead and hit save. So, App Sheet is using AI extract to look at that image and filling out a title and a description. Um, and so that's super helpful because um, a lot of times with maintenance tickets, they don't get filed because of the the traction, the friction that is caused by having to type in all this stuff. It was able to type in a lot more than um, the average [clears throat] person would do when filing a maintenance ticket. So, even with just an image, it was able to put out a more productive form entry than a human likely would have. Um, so real quickly we can check out what the title Gemini has given it. So thermostat is stuck on cooling. Temperature is too high. So it's not properly cooling the the unit. Um, the thermostat is stuck on the temperature and it's reading 87°. Please check the thermostat. See if it's functioning properly. So on it gives a little bit of instructions on um diagn diagnosing the issue and how to solve it. Um likely >> uh can that >> animation on the header is that like >> does that happen automatically or do you have to >> the generating title? >> Yeah. >> Yeah. So that's a fun thing. Um whenever I create a new record for maintenance ticket I have the title set with an initial value of generating title. So no matter what when I start a new record the title starts off as generating title and then as the bot runs it goes in there >> gets the response and overwrites that original title generating title with the new one which is the >> log logical populated one. Yeah. >> Yeah. Um, so going back into the app, what I did also was I fed Gemini using AI categorize. I fed it this information right here and this information right here in order for it to give me a priority level. So I have priority levels of critical, high, medium, low, and trivial. Um, this is pretty hot. Um, in Texas, you probably have seen your house get up that hot a few times. Um, but that is pretty hot and it should be fixed. It's pretty high priority. Um, also added some format rules that have a different icon for each of the different levels. Uh, so trivial is like uh a clock is like you got time, it's okay. Um, uh, fallen tree on top of the carport that is not trivial. That is a critical. So, we have a filledin um warning triangle, but for high, I have a non-filledin warning triangle to show that's still high priority. It's still critical, but it's not like quite as high and as bold as the other one. >> And then medium priority is like a scale that's evened out >> and so on. Um, so let's let's go ahead and check out what this did. Let's rehlight that. The first step AI extract or the first step is me taking a picture with my phone. Second step passing that image onto AI extract and powered by Gemini and it returning this value and this value right here. Um a title for the issue as well as a description of what's wrong and how we could potentially fix it. Um from those two things that AI extract gives us, we go ahead and create a priority as well. That priority is created by Gemini by understanding the different text fields. Um and then we have this thing right here. So whenever um this thing gets completed um the maintenance guy goes into here and he goes in ahead and types in a description of what he completed. So, I got Gemini to understand the situation a little bit and um got got to um give me a potential summary of what a maintenance person would do. >> Oh, wow. >> So, I just got it to do that and it did a little bit of a a time log. And so, that is going to go ahead and save. And what this is doing is it's passing this information to Gemini as well as all the other information above and it's quickly creating this right here. Um, a high priority issue regarding a thermostat stuck on cooling reading 87 degrees has been resolved. The problem was identified by a loose capacitor wire with a and was resecured. So blah blah blah blah blah. And so it was able to summarize this was the issue. This is how it resolved um without it being all of this stuff right here. So let's go ahead and escalate this situation one further just for fun. Um let's do a a bad thing. And that is going to be a dumpster fire. I don't know who took a picture of my life. Um [laughter] we'll say that's in the back alley. We'll go ahead and hit save. Um, so it's generating a title for that. >> I'm interested to see what it categorizes it as. >> I mean, hopefully critical. >> Yeah. Okay, >> there we go. Critical. So, we know that Gemini has some sort of uh reason and understanding. Um, dumpsters on file. Fire files. Dumpsters on fire needs immediate attention. >> Um, This is probably good information. [clears throat] I think that's probably honestly in this situation >> the best advice of just hand it off to the professionals. Um, >> but we went ahead and got Jim and I also to create a little bit of a time log of how it was resolved. So, we'll paste it into there. So, uh, 2:15 called the fire department, cleared flammable debris. Fire department distinguished everything. Um, assessment was complete afterwards, all that. And we saw this popup right here. Um, a quick summary of the how the whole event folded. Um, this comprehensive response appears to be a viable fix for the in incident. So, it's a pretty cool summary of just like a manager looking over these maintenance things quickly seeing like what was this critical thing. >> This is a full summary of what happened. >> It's cool to see. >> I almost wonder if it you could combine all of them with this sequence of like the time cards could be uh >> different records on this table. Mhm. Yeah. Like this be a child record and it brings in the >> it brings in all the child record values and like you probably have a virtual column >> thatiz pulls the text in from the child record >> items and then it summarizes all the child records. Yeah, exactly what I was thinking too if I were to build this out more. Um, but you as a saleserson, a CRM, >> being able to pull in the text from the different sales activities >> and then have a summary like so maybe you have sales activity like a virtual column sales activity um >> of current month or of last month >> and it takes in all the text from those sales activities >> and then summarizes >> last month's sales activities. and you just have a little chunk that tells >> you or your manager or whatever >> the story of what is going on in the most recent place. >> Yeah, >> could be pretty cool. >> Nice. >> Um, so let's go ahead and check out how this is done because this is cool as what everybody's talking about AI, but if it's so hard to work on, then it's not really going to be that valuable to us. Um, so this is the AI extract, the one that we talked about in last week. We could talk about it more today, but I think it's better for us to focus on AI categorize and for the other bot, AI summarize. Um, so let's go ahead and um just focus in on this AI task um bot. Um, [snorts] this one is triggered when a new task is added. So when a maintenance ticket is added, it starts this workflow. The first step of this workflow is that AI tasks looks at the image that was uploaded. It understands that image. It uses Gemini's LLM capabilities, its understanding of the internet, understanding of dumpster fires, how critical they are, so on. Um, and assigns a title and a description. um based off of that title, based off the description, based off of the room ID that it's in. This is a ref column. So even based off of what room it's in is having that context. Um we could go ahead and add when it was created. Um we could go ahead and add um who the employee is who created the ticket. Um because maybe if it's a manager raises a ticket, we want to feed it like >> if the user of who submitted it is an admin or is a manager >> increase the priority. Um you could add that as additional instructions. It would have understanding of that more full context other than just the image. Um, and then based off of the aggregation of all this data right here, it fills out this right here, the priority. And inside of priority, we have critical, high, medium, low, and trivial. Um, I'll walk you guys through how to build that in a second also, instead of just showing you guys the final product. Well, let's go ahead and move on to the next one. And this bot is the one that updates the AI summary. Um, and so this one is on the update of a maintenance ticket and specifically not every update. We only want the updates where the summary of what was done by the maintenance employee um that they changed the value. So anytime the maintenance guy updates that column, it's going to update that summary. So if they say, "Oh, turns out the the dumpster caught on fire again." um AI is going to look at that and be like, "Oh, maybe it's not resolved. Let's update that." Um so this is going to run every single update where before the change, >> the value of that column is not the same as the column was after the change. And then it's simply going to look at all the columns, very similar to categorize. It's going to look at all those columns and it's going to create a summary that goes into here. Uh all the all the guidance I gave it was create a quick summary of what was done for the task and if it seems like a viable fix. Um notice I used no code. Um but to prove that point a little further, I went ahead and copied this app and this app doesn't have that AI categorize and doesn't have that second bot for AI summarize. So let's go ahead and build that out right now. So, I'm going to go ahead and create this extra step after the AI extract, and I'm going to call this AI um categorize. All right, I'm going to select AI tasks right here. And like I said, I want to do the categorize right here. Then we have the context of what understanding we want it to have. And then below we have the output of what we want it to produce. Um so we wanted to use the title, the description, when it was created, when it was updated, the status, the employee, the room ID, what was done. Well, what was done wasn't created yet. So it's using all this information and it's going to go ahead and create a priority. Um, we could go ahead and type out, hey, critical should only be used if something's literally on fire or if something is going to destroy our office building or if someone is in danger. Um, everything below that at max is high. Like you could add that type of instruction. Um, you can see the example right here. They kind of did a data dictionary of like inexpensive is less than $10 if you're categorizing something off of like how much it could cost. So I could say like the cost of repair and we could say like it's an inexpensive repair if it's less than $10. This is the example that has right here, not something that I typed. Um so we actually just finished this AI categorize. I did a lot more talking than development. Um, and we've already completed that. Um, so let's go ahead and do the second one. So I'll go ahead and call this one AI task um, comma or colon um, what should I call this? Um, task completed. All right, we want to configure this event. The event is when this happens. What what happens that causes this to be triggered? Um, we're going to say this is the um mech updated um description. >> Your keyboard's clacky, Austin. [laughter] >> Oh. >> Uh, we're going to go ahead and select maintenance ticket right here for what table it's watching. And it's only going to watch for updates because a maintenance person doesn't have an update on how it was fixed um when it was added and if it's deleted, it really doesn't matter. Um so I'll go ahead and go into condition because we don't want it to happen on every update. We only want it to happen whenever that specific pre-type summary is completed or changed. Um so app sheet whenever you're using a bot has this row before and this row after. Super useful. allows you to dreference a value of a column before the change or after the change. Um so again this is happening on the update of a maintenance ticket. Um this is returning the value of this column before the change and if we go ahead and compare it making sure that it was not the same as what it was after the change. And so that just for us confirms that whenever there's a change to the ticket, this change included the the column that we're wanting to watch. So we'll go ahead and hit save. Now that we have this being triggered, what do we want to trigger? What action do we want this bot to take? Well, go ahead and create step. I'll go ahead and rename it to AI summarize. And we'll click on AI task right here in the right side. And then instead of extract, we'll do summarize. We want it to take in. So we got our input and our output. Just like the categorize, we want it to take in all this information that we can, all valuable information for it to understand. Um, and then from that we want to give it the what was done summary. So of all this information we want it to spit out this one column summary. >> Um and then we can guide it that summary a little bit. So, um, in this response, please add a summary of what was done and if it was a viable fix. All right. and we had a fun idea of what we could do for um our test. And so what we were going to do is get Nano Banana um to go ahead and generate us an image. And so in spirit of us using nano banana, let's let's involve a banana. Um, so let's just say I I want an image of a ice maker that is broken. An ice maker that is broke broken. Um, this should be one that you'd find at a >> fast food joint. >> Um, or the back of one. >> We also said it was >> broken because of the bananas. The issue is that it's not making ice, but making bananas. All right. So, we're getting Gemini to create that just fun little little prompt. Um, and we'll use whatever image it gets us. I have thinking mode on, so it's going a little bit slower than if I did the >> the speed one. But look at that. That's an issue. Do not use makes bananas question mark. [clears throat] [laughter] >> I love the question mark on the side. >> Yeah, way better. >> So, let's go back and let's submit this into this. I don't have the correct I don't have a kitchen inside of here. We'll say the full building is the issue. Um, let's go to my desktop and use that most recent screenshot. Broken, do not use makes bananas. Let's see. This is a really odd thing. >> Yeah. >> Happen. Let's see what it's what it thinks of it. All right. Broken banana cooler causing safety hazard. The banana cooler is broken and leaking water and ice creating a slip hazard. So, it's a little bit of an extreme example. >> Um, but these this is it it has recognized that this is an issue and >> of the things that need to be resolved immediately. probably the ice and the loose bananas. We don't want anybody slipping. >> Yeah, >> but let's go ahead and put in I go went ahead and created another Gemini thing for us to do that helps me create a um a description for a maintenance person to fill out. So, if we go back to Gemini, go into here, I can just drag in this maintenance ticket. And this is gonna this guy's gonna pretend to be >> the maintenance guy for me. >> Nice. >> So I I put in that screenshot from our app sheet thing and then just that way I don't have to type out the whole maintenance thing. Um I'm getting Jim and I to like kind of role play as the maintenance person. That way I don't have to type it. And that way I can also prove to you guys it's not me following a script to get the perfect response for Gemini to work. >> Real time prompting. >> Again, I have it on thinking. Gemini does not need to be thinking that hard on this. I'm [laughter] using the the the thinking mode model of Gemini and it's just overkill. But we'll we'll wait a second for it to be done. We see that there's the fast >> that would probably be better in this situation. >> Yeah. So Cam is most of the columns in the database. Well, one are you using app sheet database or Google Sheets to um as your data source for this this app? >> I'm using uh Google Sheets. >> Okay. >> Yeah. And so in that case it doesn't matter if you're what database you're using you can still use. >> Yeah. Absolutely. So there's no limitation on which database you can use. You can use uh datab app sheet database. You could use Google sheets. You could use Microsoft um >> Excel view um or you could use SQL if you wanted. Um, but this is the response it's given me about um, so it roleplaying about a timeline of how things were fixed. >> Um, and so let's go ahead and put that into our app and prove. So one, uh, it was able to choose critical. That was the categorize in and work showing that the the categorize is working. Now let's see if the summarize is working. And so it's going to send that it's saving that data that we just put in here. It's sending it to Gemini as well as all this other information. And Gemini very quickly gave us back a summary of what was done and how it was fixed. So super cool. >> Um I did a lot more talking than development and of development work probably took me about like 30 seconds total. >> Yeah. in order to do all this automation. And [clears throat] so >> it's crazy. >> Um, a cool thing about App Sheet is that all of these different SAS platforms all over the um SAS environment, SAS ecosystem world, all of them are trying to grab your attention, get you to upgrade to higher tier licenses, get you on enterprise stuff. um with like single use case AI features and App Sheet says, "No, we want you to build your own AI features in your own SAS platforms. That way you can use AI to do what you need to do instead of Trello or uh like something off like a Trello or um Jira or any other SAS platform giving you like very niche AI features. App sheet just gives you opens it up for you to build your own instead of waiting on another platform to develop something for you that might or might not be useful. >> Nice. >> So yeah. Yeah, this is great. Um, it's it's cool to see how it just takes like a couple clicks and you've got a full uh like AI feature built into your app. >> Yeah. And I think like we said last time, the like I think the huge value ad for me and AI tasks is like there hasn't been a great way to do like bulk updates or bulk data imports. >> There there's like you can download an Excel sheet and then load up the CSV. Uh but with the extract rows >> Mhm. >> you can now add multiple rows at once. Um yeah and and it's you don't have to do some technical looping mechanism with uh with actions or automations >> to create that. >> Yeah. >> Which we've done webinars on that one too on how to create loops the oldfashioned way um in app sheet but now you can do that with extract rows and a few >> few button clicks. So yeah, this is great. Um, I think what we'll do is we'll add um the app link as well um to to the description of this video. So, y'all stay tuned for that one. Um, so that way you guys can uh go on the back end, see the editor, copy it, um, and add some of those features to any of the apps that you're working on. Um, but yeah, thanks so much, Cam. That was great demo. Um, excited to see how everybody utilizes this new AI tasks feature. Real quick before we leave, y'all wanted to [clears throat] make sure you knew about our free resources. So, we have the Chrome extension. So, if you wanted to get access to the app sheet toolbox, it's free to use. Uh, kind of adds on to your expression editor. Also gives you great access into the views that you've made. Um, and you can save your own expressions. Um, so it's a handy tool there. Um, that's probably one of my favorite features about the Chrome extension. Um, and don't forget about appetitrading.com. We've got access to um a bunch of free resources over there and you'll get in the loop for any deals that we have um in the pipeline. So, one happening right now um if you head over to ashtrain.com, you can get access to all of our courses for $100. So, it's only open till the end of the year. So, make sure you go check that out. I don't want to miss that opportunity. Um, it doesn't look like we have live Q&A today. Cam, was there anything that you felt like you wanted to share with the audience that we didn't have time for uh before we wrap up? >> Um, not particularly about what we talked about today. >> Yeah. >> But is app sheet training back? Are we doing webinars now? Are we back to live streaming consistently? I think so. We got this the schedule coming up. Jenna sent out the email. >> So, if you're not on our newsletter list, make sure you're on that. Uh so, you'll get up-to-date information on the release schedule of these. So, we've got the next three lined up next week. I don't remember what we're doing. >> Do you remember what we're doing, Cam? >> We're covering app sheet database. >> There we go. In the week after that, we're doing a um a review of 2025. Um app sheet training. >> Our team has been a little bit quiet this past 2025. Um and we wanted to make sure that nothing was missed. Um. >> Mhm. >> Even though our our foot might not have been on the gas as much on our YouTube channel, >> that's not the case for um us doing development and us really buying in and believing in what App Sheet is doing. >> But it's also not true that the App Sheet team has had their foot on the gas over this past year and has announced a lot of cool stuff, released a lot of cool stuff. Um we're also doing a lot of cool stuff >> that they're um opening up office hours again. So there's a lot of uh double >> Yep. back back to the good old days as they say. >> Um so we'll be we'll be um back in action with our live stream. So be on the lookout for those. If you're not on our uh email list, make sure you sign up so you can get the most up-to-date information there. Like and subscribe to the YouTube channel so you also get notifications just on here uh when we're going to be streaming. I'm also going to be doing some daily no code briefs too. So um those will be lined up as well in the coming weeks. Um so you'll get a little bit of daily information from the Crew team. Um and yeah, I think that's it. Um, I'm excited for the upcoming year and all the new content we've got coming out. So, thanks Cam and thanks to everybody that joined today. Um, hope you found this valuable and we will see you guys on the next one. Bye. See you guys.