Hands on with AWS Lambda Durable Functions

Key Takeaways

Have you ever wondered how video social media platforms like YouTube or Instagram or Facebook analyze videos before they put them on? I mean, they need to check things like, you know, is there profanity? Is there hate speech? Is there, you know, one person saying something bad or good? We don't know. But we want to check it and see what is the tone of the video. Well, with the invention of AI and with this coming out, there's an easier way to do this than a bunch of people sitting around watching the videos and saying, "Yeah, this is good or bad." So, today we're going to see how to build that. Okay. So, let's talk about why I'm even building this system. Let me introduce you to two of my five children. Meet Sophie and Gracie. Sophie's 15, Graciey's 13, and they have their own YouTube channels and they love to produce videos. Now, the deal with them having their own YouTube channel is the dad would watch these and approve them, make sure we're, you know, putting out the right message and that we're, you know, we're we're not being unsafe, right? So, with Sophie, that's fine. She would do maybe two videos a week and I could keep up. But Gracie is a lot like her dad. She would do, you know, 30 videos a week or a day and they were about 17 to 20 seconds long. She loves her shorts. Uh, and so I was having a hard time keeping up. So I needed to build an automated system or a pipeline that could handle these. And I was thinking, how do you know large, you know, video social medias like YouTube, how do they do it? And so this is what I came up with. Welcome to the video approval uh and analysis pipeline using serverless and AI. And the actual hero of this is the new AWS Lambda durable functions. So let me walk you through how this works. So number one, a user uploads a video to an S3 using EventBridge that then invokes the scanner function. And the scanner function, first thing it does, and there's a couple steps, but mainly it starts the transcribe and recognition jobs. And these are done in parallel, meaning at the same time. And once it starts those, it actually pauses. Now, one thing it does before it starts is it saves unique tokens for each job in Dynamob so that we can look it up later and then it pauses and it waits and there's no billing going on or anything until these jobs are done. Now, transcribe and recognition once they complete the job they will then send events to eventbridge and SNS respectively. Transcribe generally uses event bridge. recognition requires SNS to do this and so it sends those events. However, they both invoke the response handler lambda. So step number four is the the response handler is then invoked and it looks up the corresponding token that we saved earlier and we use some some logic to identify those jobs and I'll show you when we're doing the code. Uh and it actually invokes that lambda. it looks it up and then it calls to the Lambda service and says, "Hey, start that invocation of the scanner function back up. Here's the information." So both of these actually start up their branch of that parallel process. So the scanner function resumes and then step number five is the scanner function fetches the artifact. So it actually goes and grabs the comprehend data uh or fetches the artifacts from transcribe and recognition and it sends that text to comprehend and it does three things and it does these in parallel as well. It actually uh checks for toxicity. It checks for sentiment. And most importantly, it checks for PII. Are my girls posting phone numbers they shouldn't be? Things like that. Once it gets that back, because this is a synchronous call, it calls, it waits, it gets that information back very quickly. It then takes that information and it sends it over to Bedrock. And we use the Nova light model. And and when we're kind of explaining this, I'll explain why I chose Nova later. Uh and it gets a summarization of that text. Okay. And after that, step number seven, the scanner function saves the full results to S3 up here. And then it uh saves the metadata to Dynamob. So we have that for historical data. And then it sends a notification for human approval. And we do that through AppSync events. And actually goes back to the little front end that I built. So they can see that. We can also send it out through an email or a text message, however you want to do that. Then the scanner function pauses again. Again, you couldn't do this with Lambda before, but now you can pause that scanner function and it'll wait. And for this, I think I have it up to three or five days that the uh human in the loop has time before they can respond. All right. So once the human approves the rejects, when we say human, it's dad. So once dad approves or rejects, so I'll get a paragraph and you'll see in a bit where it'll say here's what it is. Uh and I can approve or reject it after watching it. Then uh it actually uh sends it back to the API and same thing our response handler lambda looks up the token and it sends it back into the scanner function service and it starts the scanner function back up again. The scanner function then updates Dynamo DB with the final status reject or approve and then the system shuts down. And there you go. We've orchestrated using AWS Lambda durable functions. We've orchestrated this pipeline to analyze and approve videos. So, now that I've kind of explained the overview of it and why we've done it, let's get into the code and watch how I built it. Okay, before we get into the actual code, let's look at our configuration because there's a few things I want to point out and kind of explain uh and get your take on. So, this is all going to be in the index.ts file. This is the code for my durable function. Now, normally when I'd build a TypeScript file or I'd build something this kind of big, I would break it down, but it's easier to show you if I have it all in line in one file. So, uh later I might break it out and and logically kind of group that up, but you get the idea here. All right, so let's jump in. Obviously, we have our imports. Uh some are saying they're not needed yet, but obviously we're going to be adding more code later. Uh but here's our configuration. I have my logger, which I am actually using the Power Tools logger. Uh, and so we do a lot with power tools and then our clients for transcribe recognition so on and so forth and our variables scanner table. But the thing I want to point out there's two things. First is timeouts. Now remember when we're doing steps inside of durable functions much like AWS step functions uh we can set a timeout. we can say, look, if this doesn't happen in a certain amount of time, kill it. Or if we don't get a heartbeat back in a certain amount of time, basically take it down or retry or whatever you want to do. Uh, so it's really important to set those timeouts and and kind of think that through. Don't just slap random ones on. Uh, for instance, this 259,200, I believe that's f 3 days or five days. I have to do the math. Uh, that's for my human approval, so I'm gonna let that sit for a while. Whereas 1,800 is a lot less time. Uh, and so I think that's uh I think that's 15 minutes. It's probably bad math. 60 times whatever, you know. Uh, but uh, yeah. So, call back seconds. Now, I'm really embarrassed, but I'm not going to worry about it. Uh, and finally, we get to the call back strategy. And the call back or the retry strategy, I'm sorry, for this is for the call backs. And this says, hey, if you if something doesn't work, how many times should we retry? Well, you'll notice I've got mine to false right now, but that's just for development. During development, I want things to fail fast. I want to see the errors. is I want to get, you know, I want to work on that and and work through it. So, I'm going to turn that on. However, in production, I'm going to turn it on so that I have retry up to three times with an exponential back off of two seconds, 4 seconds, and so on. So, uh again, just kind of configuring that out. Uh and now we can kind of climb into the code. So, let's do that next. So, now we have a working function. It doesn't do much, but let's walk through it. First of all, we've introduced a handler, which every Lambda function needs a handler. And we're wrapping that handler with the durable function call or the durable function term with durable execution. So that lets the Lambda service know that, hey, this is running uh with durable execution. And so it has the ability to stop and start and replay as needed. Now, we're going to pass in, you know, our event and our contest just like we normally would. We grab some information. And all that's not important right now, but we have our first step. And our first step is to generate a scan ID and a time stamp. And you may say, well, Eric, why are we doing that in a step? Well, these are nondeterministic responses. Every time I run UUID V4, I'm going to get a different answer. Every time I run date to iso string, I'm going to get a different answer. So I want to wrap those in a step so that once it plays and we get a result, it stores that result so that when we replay the lambda function, we get the same results every time. This is the magic or the power of durable functions is that replayability. Uh and so it's really important that our non-deterministic code is wrapped in a step. And you'll hear that about every video I ever do with durable function, but just kind of I had to bang it in my head. So, I'm trying to help you get it there. And then finally, we log that information out. Uh, and then we also return it. So, now we can go ahead and see this in action. So, I I don't need to deploy this really. Uh, I'm just going to run it locally because it's not doing much. Uh, so let's go over to my terminal. I'm using a big terminal so you can see this. So, the first thing I'm going to do is as I've done new code, so I'm going to go ahead and build. So, I'll run SAM build. Okay. So, that happened really quick. Then I'm going to do SAM local invoke. Uh, and what this is doing is this is going to use actual local terminal to do this. And you can see I've done this before. So we'll load that. And there's two things I want to talk to you about. The number one is my event file. So demo events S3.json. This is a an event that has an actual what an event should look like when an S3 create has a new object created in it. The second one is a locals.json. That's just what I name it. You can name it anything you want. And this is a JSON configuration of the environment variables. I have actually deployed this uh out. Uh I'm just testing locally, but I want it to be able to call later. Probably not for this one because we're just doing UU ID and dates, but later I want it to be able to call and the AWS services and have the right to know the variable names. So this actually collects that all in and I'll put some links in uh on how to how to use those. So couple things about AWS SAM just so you know. All right, so let's go ahead and run this. So this is again the local invoke. It's going to use Finch. I use Finch for Docker uh locally or for containerization. And so it uses that. And there we go. There's our results. So you can see here I got a message here where it actually output the scan ID and the uploaded at. Uh you can actually see the outputs of those as well. So that this is what our logger produced up here right there. And this is what our output produced. And then finally, you can see here we can get a little more information. If I ran this, I would get that execution uh details also. But let's go ahead and get the execution history. So let me copy that. And you'll notice we've got a an invocation ID. So let's copy that and paste it. And this is actually grabbing from the local container. And now you can see a table that shows uh here's everything we did. We generated the scan ID uh and it started and succeeded and we the invocation was complete. And now we also generated the update but if you remember if we go back to our code we just called this the generate scan ID. So that's the step we're running. So when we do a step we pass a name in and then we pass the functionality. Let's go back here. If this if you don't if you're not crazy about the table you can also run this uh with a format like this. and we'll do JSON. And you get the same thing again. And it grabs that execution history of that ID. And there you go. There's all the information. Okay. So, that was our first step. After this, we're going to actually start doing a video analysis. So, we're going to start adding more steps, parallels, and different things like that. So, let's take a look at the next step. Okay, we're in section two. I'm actually going to call this two-way because in section two, if you'll remember, we'll go back to our graph here. We were going to do transcribe and recognition at the same time in parallel with wait for callbacks. And we're going to get there. We'll do that in 2B, but for the moment, I want to go ahead and uh kind of keep it real simple and just introduce you to the wait for call back uh at the moment. So, we'll walk through this. All right. So, you can see here we're all inside our handler. From now on, we're going to be working everything inside our handler because remember I'm keeping it real simple and and straightforward. So, we've got the step one. So, back down to step two. We're going to do transcription results and we're going to use a content.weait for callback and it's going to give us a call back token uh as a string. I'm going to export a a job name uh just for me to use later. But here we go. First thing I'm going to do really is call an DynamoB. Now, this is inside the the wait for callback step. So this is non-deterministic uh and it'll get replayed if there's a problem. And we're going to call Dynamo DB and we're going to put that call back token along with some other information to kind of start this job out. Okay. Then we're going to start our transcription job. So we actually call transcribe and we send it we send it the information and then in this call back we're actually putting our timeouts uh you know h how do we want it to how long do we want to wait the retry strategy that I mentioned earlier. Now after it sends this this is where the workflow stops. it pauses and it waits and it'll just sit there and wait till either it gets a call back with success or fail or it times out. Now, after we get that call back, it'll pick back up. Let's say we get a success and it'll say, "Okay, let's go ahead and get that transcription data." So, we'll grab it from S3. Uh, and then we'll do some parsing and pull that data out. We'll do some erroring if it doesn't work. Uh, and then we'll return it. All right. to test local. I'm going to run my same SAM local invoke uh here. And you can see that it populates it out because I'm I'm using the uh the terminal with uh type ahead here. Uh and so I'm going to go ahead and run that. Now, you may say, "Oh, Eric, you didn't build that." And you have to build every time you change it. Well, I have a little trick here. I run sam sync, uh the actual command of samsync-watch, and that'll tell it to watch this. And it actually what it's doing is it's building and deploying. I'm just using the build part of it, but it's super helpful. So, all right, we come back here and you can see here I've got uh it's wanting me to send a call back success or failure. It already has the token here uh in line. So, what we're going to do is we're going to go ahead and send a success for the moment. And then what I want to do is I'm going to grab uh my code is going to actually accept expect an ID. And this is a job ID from trans transcribe. So, let's go ahead and build this out. See if I get it right here. Job name is all right. All my quotes are in place. Then a comma. And then my status. Okay. And my status is completed. All right. And so let's go ahead and finish that. All right. So there's my response. Just simple JSON. Let's go ahead and enter that. And it was sent successfully. So then what happens is if you remember in that code then it goes out and it gets the transcript uh here. So this is that that call to fetch the transcript from S3 and we're going to actually output it. So I'm logging that out if there's a problem but I'm returning the transcript data here. Uh and so let's go back and look at that real quick. And you can see here oh yeah she's saying my my dad never lets me post anything. I asked him to do something like negative so that it would pick it up. Uh again uh down here I can check the history uh just like we did before uh with the the execution and the history. All right. So that's just a simple you know wait for callback functionality and kind of wanted you to see it happen. In the next section we'll go ahead and add this parallel and both wait for callbacks. So now we're back in section two. We're going to call this 2B. We're actually roll. And it changes the way we code this a little bit. So let's walk through this again. We're inside our handler. And one thing you see here is we're going to start out with a parallel results. We're going to set that uh equals to a wait context. parallel. This is a new uh you know durable functions call here. Um and so one thing that does is that provides a child context. So it's a context within the parallel that still has you know things like wait for call back things like that but it scopes it to this parallel uh process. So we've got two branches now. The first one is should look real familiar, right? So we've got this um constant transcription results equals weight child contextweight for callback. So I'm not using the context of the outer, I'm using the inner context to do this. That helps isolate that and run it here. Going to name it the uh transcription result here. Uh and then we're going to go through and and this should all look familiar from last time. Again, this is where it pauses. This is where it picks up and runs. and we wrap it up. However, now we have branch two where we're also getting the child context and we're going to do recognition result and we're going to do the child context.weight for callback. Same thing here. We're going to go ahead and store the token for this uh just like we did in transcribe and then we're going to start the recognition and it too will pause. Right? So now you've got both paused inside this parallel. So one may pause before the other and then but the whole lambda function comes to a pause when both are set to pause. So it'll just sit there and it'll wait until one of them starts back up. Uh and so when recognition comes back up and recognition we actually call back into recognition we have to loop through with the next token kind of a pagenation thing to get all the recognition data and then we'll process that out. If there's a problem we'll we'll handle the error down here. you can see there. Um, we also do the the transcription error. Uh, and there you go. So, we're going to output the actual transcription data, the recognition data, and any video text data because remember what I'm doing is I'm grabbing any text that they're saying, but also maybe that they're putting on their screen. Uh, okay. So, this time we're going to instead of testing locally, I want to test in the cloud. And I could certainly pop out to the console and give it an ID or or an event uh and try that, but I can also run it locally. So, there's a couple things we're going to do here uh when we do this. So, I'm going to go ahead and go back to my SAM console here or my SAM terminal and we're going to uh change things up. All right. So, I've got a couple of windows. This now I have a big wide screen and normally all these will be showing but I've got them tabbed so they're easily for you they're easy for you to see. So the first thing we have is we have this same sync still running. So every time uh and let's see if we can make that happen fast enough here. So we'll just put a comment in here. Um so if I go log out. Okay. So, I'm going to go ahead and save that and come over and you can see that it does a quick build and then it'll actually sync it with the code in the cloud. The build is what I was using earlier, this part, but now it's synced up. Uh, it sends my version. Okay, so that's always running. Every time I make a change, I add a little more code. This is running and it's usually done before I can even test anything. The second thing I'm going to turn on that I like to run is a simple command called SAM logs minus T. Now the T is a tail. And what this is is this will actually aggregate the logs. Let's go ahead and start it from all the resources, API gateway, uh, Lambda functions, anything else. And it'll bring these in, and it'll update this every 5 seconds. Obviously, we haven't run anything yet, so we're not going to see anything. But another little trick I like to do is I like to do another one that Sam that Sam logs, and then I like to filter it. Now, you don't have to do this. I'm just showing you that I'm a nerd basically is what I'm doing. I'm a nerd. Here it is. So, I filter for the word info. So, I'm looking for comments that I basically put out to me, right? Again, we don't have anything there. Uh, but now they're ready. Okay. So, we're not have anything here. This will show me everything. This will show me just info stuff that I've got. Uh, I forgot to put the minus t on this. So, you put the minus t on that as well. And that'll actually tail it. So they're both just sitting there pulling for that data. Okay. So now we're going to invoke uh the function again. Now this time remember I did SAM local but this time I'm going to say SAM remote uh and then I'm going to call the scanner function. And then same same as the uh you know when I'm doing uh local I'm actually give it an event file. And so that event file is and I'm actually not gonna try to pull pull your leg here. I'm going to copy and paste this so it goes a little faster. But there you go. So I want to explain this again. We're doing SAM remote scanner uh and then uh the scanner function and the event file and I pass the demo events S3 that I have. And then there's two parameters that I want to explain. There's the qualifier which I have if you um in my SAM template I have an auto alias where I'm always alias publishing an alias as live. You have to have a fully qualified uh do or or or endpoint for lambda functions. With normal lambda functions we've allowed you to do the latest but with uh durable functions you need to have a qualified one because it helps us know which one you're running. And then the second thing I'm doing is a parameter called invocation type event. And this sets it to a an asynchronous event because this is an asynchronous Lambda function that I'm running. All right, so let's go ahead and start this. Hopefully that makes sense there. Uh, and I did not uh hang on just a minute here. Let's just fix this right here. I'll actually just copy the whole thing real quick here. This is always fun when it doesn't work for you. Let's try that again. And let's paste it. Uh, okay. So, it's invoked. So it's invoking. So so we really don't see anything, but now we've got logs coming in. So let's see what's happened. So the first thing is we had the platform started up. Uh it's the initialization. Here's the start. Uh we get a platform report because there may have been a quick stop and we're starting again. And as we're going through and it won't read all this through, but I'm going to note that here the transcription is complete. And here she is. My dad never lets me post anything, you know, much like we're seeing. Uh and then here we've got a lot of data um coming in from recognition. Where is text located? Now you can see this also uh over here, but it's a little more uh just just the info ones. So this is the one I'll usually stay on and you can see that the log is grabbing there. Uh but you can see here's the text that was on her screen. HD adventure camp. Looks like she put a lot of different things. Call here if you agree. So it's picking different things and so you might see some duplicates in there. Um, so there so that's the information and it's a lot of information. This is all like bounding box of where that text is. Here's the text if uh where it happened in the video where you find it. So like I said just a ton of data coming back in and uh now we've got we're starting to get our analysis right. We've actually pulled all the text out of the video both in the audio and the screen. So our next job is to kind of prep it and then we'll analyze it. So we'll move on to the next section. So the next section is going to be quick. I just want to kind of explain this. This is where we build our combined corpus. Meaning we take all this text and kind of put it together so we can send it. Now you may look at me and go, "Well, Eric, that's all internal work. Why are we wrapping that in a step?" Because you can see here I'm doing a context. And I'm wrapping it up. Well, sometimes when you have order and you know, we're making sure that that we that we pull that. Sometimes it's not all guaranteed. So, we want to make sure that we get it once, we get it right, and it's the same thing each time. So, once this successfully runs, if it has to replay, then we get the same thing every time. So, basically what we're doing is we're grabbing the transcription items, we're grabbing the recognition uh data, and we're combining it all together and pulling it in. And then when we return it, we return the corpus data. Now, I won't run the test on that because you've already seen me run it several times. We'll be testing a few more times as we go. Uh but just to kind of get that idea, this is how we prepare the data in a replayable way um to to make sure that our durable function can replay as it needs to. And with that, let's get into the actual analysis. We'll move on to that in the next section. So now we're down to one of the big parts of this and and the important parts is the actual analysis of the text. Want to make sure that everything's safe, everything's nice, everything's as it should be. So we've created this corpus and we're going to pass it in. And this is another parallel or branch parallel task with branches, right? So again, we're going to do our our analysis results and we're going to do context.p parallel and it's going to give us a child context just like before. So, we're going to do branch one is a toxicity detection and I'm going to use comprehend. Now, you may say, "Well, Eric, why don't you use one of the LLMs or something like that?" And that that's fine. I can do that. But I've got these machine learning models that are trained, these these deterministic models that are trained to do this, very highly trained and very good at it, and they're very fast. So, this is a less expensive and more efficient way of going. Uh, so I'm going to use uh comprehend. Same same reason I use transcribe and recognition. I use LLMs all the time but only when it makes sense, right? Uh okay. So we're going to do our toxicity data is a childc context. So this is a step in here uh on the child context. So we're to detect toxicity. Now these are not a wait for call back. So we don't these this is an asynchronous call. It's going to call and wait and get the answer back. It's going to be pretty quick, right? So then it's going to process that. The next thing we're going to do is we're going to come down to branch two and it's going to do the same thing and it's going to do detect sentiment. That's another call and comprehend. It's going to process everything coming back and give me as a positive, negative, and it's percentages. Uh, and I'll return the sentiment data. And finally, we're going to look for PII. So PII is personal uh identifiable information. So I don't want their phone numbers or their, you know, last name or address or or my bank account number or or anything. I don't even want them on on there, but they're growing up and boyfriends and all that, you know. So, here we go. Uh, so that's going to deliver all that out. Uh, and then we've got uh then we're just going to respond with it all. Toxicity results, sentiment results, and PII results. And then I also log it out uh to to help me, you know, as as I can see that in there. All right. So, now that we put this together, we can go back uh we've already got everything uh in our in our terminal. So, let's brought that up. So if you look here, here's what we're doing. Uh, we're doing the comprehend part there. So, uh, all right. So, let's go ahead. We've got our logging going on here. So, let's go ahead and run that remote invocation again. And I'm just doing the same image over and over. Um, and we'll watch our info here. Um, in just a minute, you'll see more information pop up. I have to scroll down. It was going I didn't see it. Let's scroll down. Yep, there we go. All right, it's done. So, it was done. Uh, so let me give you a little bit of what was going on. So we have our analysis results. We've got our profanity 0.15. Nope. Nothing there. Hate speech. No. Insults, graphic. Uh, sexual, no. Uh, violence. So, it's okay. So, it's neutral, right? Uh, and oh, we do have a phone number. Looks like we have a phone number. So, yep, that's not good. So, we're going to have to uh say no to this one. Uh, and so that gives you an idea of how I'm running that. That's the analysis itself. Uh so once we have this data we can move on and in the next section we'll prepare it and then we'll send it to Bedrock for an LLM to process it, evaluate it and give me its opinion. We'll see you there. In this next section, we're actually going to take the results. So we've done this analysis. We're going to take these results and we're going to put them back together uh so that we can use them in a way where we send them uh off to uh off to to Bedrock to be summarized. Right? So, it's another context step and and I just wanted to show you here that again this is one of those places where we're doing some local work. Uh we're pulling in this information that we've got and we're going to do some some combining of this. We're going to map the PII uh back to, you know, locations and get it all in. So we have a listing of our audio issues and our screen issues. So again, we combined them all. Now we kind of broke them back out, mapped them to where they need to be, and we can pass them over to Bedrock to do that. Now, in the next section, we actually generate an AI summary. And so this really again, this is going to be a step because this is this is if ever there was non-deterministic, this is non-deterministic, right? So we want to send this information and we've built our prompt. Uh we've got, you know, the prompt set up here and uh we're we're having it give us toxicity, sentiment, uh PI, look at all this uh and and you can look at the prompt later if you want to look at this code, but you get the idea. And it builds that whole prompt out in the step, and then we're going to send it uh and come back. Now, one thing I want to do is we're pulling the model ID from uh from a variable. So I want to go back up to the top and show you for our model ID. We are actually you can pass in a model ID or I'm defaulting to Nova Light. Now you may say why Eric why why are you not using one of the one of the you know powerful reasoning like quad sonnet or a llama or something like that. They're great models and I use them a lot. But for this we've kind of collected all the information and I just needed to summarize what we have and give me give me an answer. And Novalite uh is is very fast. It's It's less expensive and it's fine for this job. I can even do a Nova Pro if I need to. I can do a prompt router to pick which one is needed, but for this uh this work we're doing, the Nova Light uh does great. Okay, so we're going to go ahead and remotely invoke this again. So, let's go back over to our terminal and we're going to uh invoke this again. The same video and then we can look at our logs here. Uh so, let's take a look. we'll see logs come back in. Okay, so we've got some information coming in here. Uh, and you can see here, and again, I'm just doing the info once. So, we've got the PII source mapped, uh, and the summary is complete. And here's the summary. So, the whole job finished. So, I just have this coming out as an info. The executive summary is the content safety assessment for video for file gracy1.mpp4 is rated as caution. Oh, my 13-year-old's a rebel. I'm going to have cautionary videos. I'll have to check it out. Uh while no toxic content was detected and sentiment is overwhelmingly neutral, the critical finding is the detection of one instance of personally identifiable information PII. You can read the rest but it's done what it needed to do right and so so with this uh and if we go back to our graph basically what we've done is you know we've we've transcribed recognized it we've done an analysis on it and now we've asynchronously called Bedrock and we've gotten that information back. Uh so our next steps is we're going to save that information out to S3 uh and the Dynamo database and we're going to get a human in the loop on this. So let's go to that next section. Here we are and we need to now save that data. So we've done this analysis, we've got a summary, we need to save that data. We're going to save it two different places. So step seven, uh we have a step uh where we're actually in the context step. We're going to save results. And so we start building out. we determined in the overall assessment uh safe, caution, unsafe, uh whatever we need to do. Uh then we're going to come in add a when did we complete it some some non-deterministic data that we're adding inside the step obviously. Uh and then we're going to build out that complete result object and finally when we're done we'll go ahead and save the JSON report to S3. Uh and then we'll save the metadata to DynamoB. Now I could break these into separate separate steps as well. I haven't here. I'd be curious to know what you think. Uh so please leave comments. And finally, uh that's where we then come down to the human approval. Uh and so we've seen this before, right? We see the contentwait for callback. Uh I'm going to get a call back token. I'm going to go ahead and kick that call back token out for my testing here. And then I'm going to save the token into DynamoB here along with all our other metadata. And then give it a timeout and retry strategy. And again, this is that long timeout uh of several days. And then we pause, right? So we actually pause and it waits for a response much like we had we did earlier. Uh when it comes back in we parse the response and then we move into the uh final step uh which is save it all to the database. So we've kind of collect it all together. Um and so and we've seen this before. We're going to do it in a step. Um and so we've got the final action. We're still sitting in the step uh here. There we go. The context.step the final status. and we gather all the information up. We return the information that we want out of that function and the final return that happens is all this data coming out and then we catch and throw errors if need be. So that's the full thing. Now we're going to do one more version of this where I add some events and stuff that we can track. But for now this is the full analysis pipeline. So let's actually see it uh in action. Make sure I save it. Uh, and then of course, uh, we look and make sure there we go. We look and make sure that Sam has, uh, done that. So, it's already finished syncing that version. Uh, and then we'll go ahead and we've got our logs going here. So, let's start another, uh, invocation. Now, while that's doing that, we've been all about the terminal, and that's where I like to work. I like to work in the terminal and get that information move fast and stuff, but sometimes you want to be in the console. So, we actually have the console as well. There we go. So, let's go ahead and refresh this here. We'll get the latest version. Now, that would eventually refresh. And here's our running one. We can actually go in here and see what's going on. So, here's here's our step one where we just generated let me spread this out so you can see it. Where we generated the scan ID. Notice these names are ones you can see here. Uh you know we we know that um so uh you've got your s your parallel branch. What did we do here? We did the transcription and then afterwards we fetched it and here we did the recognition and afterwards we fetch that. Uh then you can go in here and see the parallel branch where we were doing the detect toxicity uh detect sentiment. Go on here I believe in the output. Yes. Uh, nope. We didn't put an output on that one. That's okay. Uh, but you can see all the details from these jobs that are running. And now it is actually sitting at human approval, right? So, we can go in here and we can say, all right, uh, let's just it's started. It's waiting. And if you kind of hover over this, it's kind of fun. You can see our callback ID. I've also kicked it earlier. And you can also go over here and you can send a success or failure from here as well. Now, I'm not going to do that because I like to do it through my uh through my system, but I will go ahead uh just for for giggles. I will grab this call back ID. And then we'll go back over here. And I want to show you this. I don't think we've done this yet. Um you can actually say uh a or I'm sorry, SAM remote call back. Okay. And then I'm going to say I want it to be a succeed, not a fail, right? And then I give the token number. All right. And then I'm going to pass a result. And this is what I want it to say. And I'll just go ahead and and uh copy this real quick here. All right. Now, so I'm going to send say SAM remote callback succeed. Give it a token. Here's the result. I want to put approved. True. Reviewed by Eric DJ at amazon.com. Comments looks or content looks good. Now, this is actually probably one I would fail. So, all right. So, there we go. Approved, true, reviewed by. So, it gives you that call back success. Now, I could pop back over to my logs here. Uh, you can see the human approval token that was kicked out. And let's actually go back to the console. Uh, and you can see that our status has gone by. Me scroll back over so you can see this. And there you go. Everything's done. And we have our deta statuses succeeded. And here's our output. all that data that I said would be there. Uh this is the metadata, right? And then we've saved a lot of data in S3, but you still get that uh summary. So, we've worked in the console, we've worked in the terminal, we've obviously shown some code, I've showed how you can use AWS SAM to do that. Now, I want to show you an actual working version with a little front end I built. I'm not an expert on front end, but you can kind of see it working. Uh so, uh let's do that in the next segment. So, we've built out our analysis system and let's take a look and kind of give an overview again and see where we are. So, so if you look back at our at our chart here, our our architecture, got our scanner function and all the work it does, the transcribes, recognition, the bedrock, uh, talking to Dynamob. But one of the things that we weren't accounting for is that it would be constantly talking to the front end. It would send data via the AppSync events API. So you would just send posts to the AppSync events API with the SDK and then that data would get sent back to the customer because we want the customer to know what's going on. And so what you've got is an event driven architecture in the back end but also an event driven front end. Uh so both APIs and websockets on our front end. So let me show you what that looks like. Okay, first of all let's look at the updated code and we'll go we'll go back to my front end which obviously is awesome. So, one of the things we did here, let's scroll back to the top, is uh I've added this handler. It's an app sync events handler that deals with the front end and post to it and handles any URI uh or URL uh stuff. And then at each level, I'm publishing an event. Sometimes it's in a step of its own because that's kind of where it needs to be. And sometimes we've added it to a step uh where you could like come down here. Uh it's inside a step, the last thing we do. And basically we're just publishing an event the type that it is uh and then their time stamp things like that. So so this allows us to keep information or events flowing to the front end. So let me show you what that looks like when we interact with the front end. So I'm in my kids profile here and so I'm going to act as either Sophie or Gracie. And so they're going to go and they're going to upload a video. So we'll we'll make it uh we'll make it Gracie. It's actually the one we've been using. So we'll upload that. Okay. So once that's uploaded, you can see that it's processing here and we've already moved. It's given us the here's the branches and we're doing a transcription has started and the recognition has started. Now it'll stay here until both of those finish because in parallel but those are still uh those are wrapped in the same parallel but they're each their own wait for callbacks if you remember in the code. So we'll give those a moment. These are the longest part of that. Oh there it goes. It ran but boy you couldn't even see those. It happened so fast. But it went through, it broke out, it did the toxicity and the PII and the sentiment. Uh then it did the bundling and then it did the summary. Uh and now finally it's pending review. And then what I can do is I can go in here and like I can look at the the details which it generates a uh this is my daughter's site. No thumbs all fun. Uh and so I can watch it and this is uh gives me that report we talked about. Here's the audio transcript where she talked about how I never let her do anything. Here's the text on the screen where you see we have a phone number. So I can analyze all that. Now uh this is from the you this is from her point of view but I can also go over to the admin point of view where a websocket has updated my page and said all right here it is. I could go in and view the same details here. It pulls all that up. Uh I could watch the video there. Uh I won't we won't do that at the moment. Uh and then I can look at the details but I'm going to go ahead and reject this. So, I'm going to say, uh, not good. Oops. Spelling spelling counts. Not good. No. PII. She won't know what that means, but then she'll come and ask me. So, I reject that. It disappears from mine. I go back to hers. Hers is updated and now rejected. And if she looks in the details, she gets the response of why it was rejected. And there you go. That's building a video analysis pipeline with durable functions. All right. We covered a lot of information here. I threw it at you left and right. If you have any questions or comments, leave it for me in the comments below. I want to hear what you have to say. I want to hear what you do with this. With that, keep building, keep having fun. I'm Eric Johnson. We'll see you next time.