Building Agents with AWS: Complete Tutorial (Java, Spring AI, Amazon Bedrock & MCP)

Key Takeaways

Hey Josh. Hey James. I've heard this AI thing is pretty cool and I want to dive in and start learning about it, but I'm a Java developer who has to integrate things into servers and all sorts of different services. This seems like it's going to be hard. Actually, as a Java developer and a JVM developer, we are uniquely well positioned because 99% of what people are doing when they do AI engineering these days is talking to existing models via REST endpoints. That's integration. And you know what? Java is uniquely amazing at integration. You better believe it. We are well situated in the enterprise. We connect to everything. These Java apps are at the heart of your organization's business logic and your and the gateways to your data. They are the right place to build this kind of logic. Okay, so let's dive in and first just walk through how we set up Bedrock so that we can use it from our Spring AI application. I'm in the AWS console and I'm going to go to Amazon Bedrock and then we're going to be using models and there's two different types of models that we're going to be using. one is for our chat. So when we actually have this conversation with the LLM. And so we're going to be using the Nova model for that one, the Nova Pro model. So we're going to be able to come in here and select Nova Pro. And then I can uh request access to this model. Um so this is something that we'll need to do to be able to use this model. So we'll hit that request access. And it'll take a minute for it to enable. Uh you should read the enduser license agreement. make sure you're not doing anything against that. Um, but then we're also going to be using an embeddings model. And there's a model here called cohhere. And so we can come down to see all the different models and find the cohhere model and enable the cohhere model for uh also so that we can do our embeddings which we're going to do in the application. So there we go. Let's go back to the list full list here and we can select embeddings. Oh, there's the Titan embeddings model. We're going to use the coher one. Um, so we'll be able to request access to that uh cohhere model and use that for embedding. So now that we've got everything set up, we'll get our API keys and we'll be able to then access Bedrock from our application that you're going to build in uh on your machine. So you've already gone through all this and set up your API access keys. And so now we can dive in and start writing some code. Let's do it. Which brings us to I think arguably the most important question, which is what are we going to build? Everybody loves pets. Uh, and it's very relatable. Super super relatable. I love dogs. I love cats. I just don't love my dog. Yeah. No, no, he's he's not a good dog. His name is Peanut. And Peanut is a terrible, terrible dog. He kind of reminds me of another dog about which I learned during the pandemic named Prancer. And Prancer, this is this advert that was put on the internet uh by this uh by this woman back in 2021. And she was trying to find a new home for this dog. And she went viral. This thing was on the news and like it was everywhere. She put out this ad trying to rehouse this dog. And she says, "Okay, I've tried. I've tried for the last several months to post this dog for adoption and to make him sound palatable. The problem is he's just not. There's not a very big market for neurotic, man-hating, animal-hating, children hating dogs that look like gremlins, but I have to believe there's someone out there for Prancer because I'm tired and so is my family. Anyway, she continues, um, and she says, um, I was excited to see him come out of his shell and become a real dog. the I am convinced at this point that he's not a real dog, but more like a vessel for a traumatized Victorian child that now haunts our home. So, I was thinking, what if we could build something to help people reunite these dogs uh with their forever home and and and to uh sort of help people find the dog of their dreams or or in in in my case the the dog of my nightmare. So, I want to I want to facilitate that process with an AI assistant for our fictitious dog adoption agency called Pooch Palace. Okay. So, we're going to go here to start.spring spring.io. We have some questions that we need to stipulate or answer rather uh before we can proceed. First and foremost is what is the name of the service? I'm just going to call it adoptions because I'm really good with names. Then we have the choice of what version of Java we want to use. So we're using Java 23 and we're using the Java language. So now we need to add some dependencies to this. Yep. It's going to be a web app. So bring in the web support. We're going to talk to a SQL database containing our data. So we'll bring in Spring Data JDBC. uh I want to talk to a postgris database that is also uh capable of vector store capabilities right and we'll talk about that in a second so I'm going to use pg vector store okay uh there are many different implementations of vector store of the vector store interface in spring AI but this is the one we're using and uh and then of course we need to talk to bedrock right yep so there's two different bedrock uh things that you want to add here so one is for the chat client that we're going to build and then the other is for the embeddings And so we're going to bring in both of these uh dependencies in to talk to bedrock and it and it's just it's just already there for you at start.spring.io. And then finally, we're going to, you know, we want to take advantage of uh uh MCP, you know. Oh, that's right. We are going to dive into MCP. So be good. And we're going to Yeah, we're going to bring in the MCP client and we'll build this server later. Yeah. And then, you know, to make it so I can get live reload, I'll use dev tools. Great. All right. I think I'm happy with our selections. Let's hit enter. Open this up in the IDE. It doesn't ma matter what ID you use so long as it supports the version of Maven or Gradel uh that you've specified and uh the version of Java which all of them will at this point. Right? So let's dive right into this. First things first, let's establish uh connectivity to the data store. Right? So we're going to say uh Spring data source URL JDBC PostgresQL localhost my database. Okay. You've got your your Postgress database up and running already and it's got uh some pets in it. It does. Let's go ahead and take a look at that. Actually, you know, one thing I like about Intelligj. Click on that little button there. Hit test. Hit apply. Hit okay. Go over here. Refresh. Wow. Uh and uh there's our database. There's our database. There's our dog database. We've got IDs, names, and descriptions. And there's our boy, Prancer. His ID is 45. His name is Prancer. and he's described as a demonic neurotic man-hating, animal-hating, children hating dogs. Uh that looks like that looks like gremlins. So, first things first, I think we need to actually uh establish connectivity to bedrock. So, bedrock converse chat enabled is true. And then we want uh chat.options equal uh model equals Amazon.nova. So, this is where we tell it that we're going to use the Nova Pro model. So, we're going to use coher for the embedding. So, we enable that. So now let's build this. Let's actually talk to it. We're going to, you know, we're going to talk to the data store. We're going to have we need some types to represent the type in the database. So I'll create a a simple entity here to represent that table. String name, string owner, string uh description. And you know, I'm going to use Java records here. I just I love I love I love love Java records. Huge fan. Big fan, right? Very very nice uh compared to the status quo. And uh we're using Spring Data JDBC. So I'll say dog repository extends list of CRUD repository. Okay, who's of entity of type dog whose primary key is of type integer. Okay, integer. So now I think we're good. Let's actually get to the sort of the uh meat and potatoes here. We're going to build a simple HTTP controller that answers questions for people who are making inquiries as you know about the the dogs in the shelter. Okay, so adoptions uh assistant controller, right? Right. And let's just try a simple hello world endpoint. We're going to talk to our model via the Spring uh AI chat client. Okay. I'm being a little tongue-in-cheek here by calling it singularity, but obviously there's no such thing as a singularity just yet. Um and we're going to say, you know, given given a ID for the username maybe or there you go, user. Uh we're going to have an inquire endpoint and we're going to return data. Okay, we have to configure that chat client. I actually have to have an object of that type configured in the code somewhere. So I'll do that here and we'll return it just the default configuration. So the chat client is what's going to give us access to that model on Bedrock to be able to send prompts and and um get responses back from the the LLM, right? Yeah. And the first thing we want to do is send the user prompt. Now a user prompt is the question that the user sends uh and that you send to the AI model, right? So, it'll be specified as a request parameter like that. And we'll send in the user prompt. Oh, I passed in the wrong thing. I passed in the user. Let's try that again. Okay. It gave us pass in Oh, the question. Yeah. Yeah, that's correct. You've got to you got to be sure to pass in the right thing. Do you have any neurotic dogs? Okay. It says, "While I don't have personal experiences or pets, I can certainly provide information about neurotic behavior in dogs." All right. So, we're going to make this request and the way we're going to propagate that that history, that chat history is by configuring an advisor. This is how you intercept requests going to and from a model. Okay? And so, we're going to configure that here. Uh, and we're going to store a you know, we're going to we're going to cache we're going to memorize that advisor for each unique user. Hence the uh distinguishing user path variable there. So we'll say private final map string and it'll be prompt chat memory advisor import that advisor map and a new concurrent hashmap advisor map compute if absent passing in the user there and if the user's uh there uh we can actually uh if it's not there rather we can create a new in-memory chat advisor. So, uh, new inmemory chat, sorry, the prompt chat prompt chat memory. Thank you. There you go. And then passing in the new in-memory chat memory. That's what I was trying to do. Yep. Good. So, that'll give me back either the the existing or a new and then updated advisor. And I'm going to configure that on the call to the model. Okay. Like so. So, you just add that advisor into the prompt call. Yeah. And now we're going to have memory. Let's try it. What is my name? Your name is Josh. Great. We've got memory. Yeah. Awesome. Okay. This is good. This is a start because now we can have a conversational experience with our model here. But it's still not giving us like if we ask that original question like uh the original question is do you have any neurotic dogs? It's still not giving us anything, is it? It doesn't know about your dogs in your database. Yeah. So this is not going to cut it. And we have a higher standard of uh interactions with our customers than this. Uh and so we need to give it what is called a system prompt. A system prompt is a a prompt that overrides it, sets the tone and tenor of the responses in uh you know governing how we respond to those user prompts. And so I happen to have a system prompt here that I've pre-written. Okay, there it is. There's my system prompt. Let's just cat it and I'll and I'll copy it to the clipboard. You know what? This doesn't feel right. Let's try that. Much better. Much better. Okay, there's that. Okay. Oh, you can't cat it. You got to dog it. Dog it. Exactly. copy and paste that and we're going to put that in the default configuration for the chat client. I could also do it here at the call site, right? I I could specify it there. Um, but I find it's just one of those things where if you're going to reuse the same chat client for one particular purpose, why not put it here, right? So, there we go. And this says, "You are an AI powered assistant uh to help people adopt a dog uh from the adoption agency named Pooch Palace with locations in Antworp, Seoul, Tokyo, Singapore, Paris, Mumbai, New Delhi, Barcelona, San Francisco, and London. So now every conversation that we have with the LLM is going to have that context in it, right? It'll Yeah, exactly. It'll precede every thing that comes after it. So the responses will be in terms of that persona. Okay, so let's try this again. Um, do you have any neurotic dogs? Okay. Ah, okay. Currently, we don't have any dogs labeled as neurotic in our available listings. However, each dog has its own unique personality. Okay. And may exhibit certain quirks. So, we've got our system prompt. We got a system prompt. It's more in line. It knows what we're doing here. It knows how to how to play the game, but still no dice or in this case, no dog. Yeah. Right. No dog. Yeah. So, we need to give it access to our data. And where's our data? We've got it running locally in our Postgress database. Exactly. So let's trim this down by sending only the data that might be pertinent to this request. And we can do that by putting it in a vector store. Vector stores support similarity search, semantic similarity searches. And it's not limited to just text. You could put uh you know audio and images and whatever depending on the the embeddings and all that, but but for our purposes, we're just going to put text. And we want to find things that might be relevant to it. And then we sub and then from that result we'll send the sub selection that comes back from the vector store and then send that onto the the model for further and final analysis. Okay. So let's uh let's bootstrap our vector store. Okay. Now I'm using uh Postgres for my SQL database and I happen to be using it for the vector store but there are many other vector stores out there, right? Different options you can use. Yeah. Yeah. Okay. Okay. So let's go ahead and uh set that up here in the chat client. And in order to do that, I'm going to inject the old dog repository here to get the data from the SQL data database. And I'm going to inject the vector store to get the access to the vector store. Okay, there are many different implementations of this abstraction available through start.spring.io. We've got three on the class path here. Okay, so dog repository.findall for each dog and we're going to create a document that we're going to write out uh to the vector store. Now, I'm going to put an arbitrary string in. It doesn't all that much matter what string you put in there so long as it's consistent. So, we're going to say dog Iid dog.name dog.escription and we're going to say vector store.add list of document. Okay, that looks okay to me. No, wait. I just realized. Yeah, not no. This is much better. There you go. Document. Okay. Document. Exactly. Now, we have the data in the data and the in the uh the vector store there, but our chat client doesn't yet know to sort of consult that vector store. So, back to the advisors, right? We're going to plug in one more advisor. This time it's just going to be one singleton adviser. And that is the question and answer advisor. Okay. Question and answer advisor. Thisquestans answer advisor equals new question answer advisor passing in uh the uh the vector store. Okay. And there's no reason this couldn't have been a bean. For example, I could have had a singleton throughout the entire application. But oh well. Okay, so there we go. One advisor and then the other one, right? This question answer. So now we got two advisors in play and um we're going to when the program restarts it'll re write everything to the SQL data store. Do we have a vector store table? At the moment we do not. So here let's have Postgres and PG vector in particular initialize the schema for us. Okay, great. And then we're also going to need to make an embeddings tweak. Do you want to do that now? Oh yeah. Yeah. Another interesting uh aspect of what's about to happen here is that we're going to take our text and turn it into a vector which is a you know like you remember math right like it's a vector a matrix of number yeah an array of numbers a matrix and uh we need to constrain or define the size of the vector so that both the thing that's going to create the vectors for us which is the embedding model hosted on Amazon and the thing that's going to store the vectors which is our postgres PG vector implementation line Okay, we got to get those to align. And so we want vector store PG vector dimensions equals and what do we have here? 24. Okay, good old good old uh round number. Yeah. Okay, so now do you have any neurotic dogs? If we do, if we've done everything uh as we expect, it should result in Yes, we do have a neurotic dog available for adoption. Amazing. So we've just This is called rag, right? We've just done uh retrieval augmented generation, right? something like that. Exactly. I get that right. And so we've been able to augment the LLM with some data from our vector store. But it found PR, right? It did. So it's worked. So we found our dog, right? Um before we go too much further a field, let's comment out the initialization. I don't want to reinitialize it each time because this takes, you know, it's expensive to do this kind of thing. You might want to do this per request per as you add dogs to the database, you also vectorize the data, right? Yeah. But we need to figure out how to get this dog a home. Got to get him a home, right? How do we schedule uh pickup for him? Yeah. And here, this is an example of a system where our model won't have access to the most up-to-date or relevant data. Sometimes that's just a question of, you know, how stale is the model, right? Models typically have cutoff dates. They're valid up until a certain point in time, and they know about what's happened in the world up until that point in time. And then after that, you kind of have to like plug in the gaps. And then there's also just a question of like reach, right? Your model probably doesn't have access to your company's private customer database. That makes sense, right? So we need to give it access to that or at least to the parts of it that it needs to answer these these queries. And um and then uh and and we also want to give it access to the real world. So we can do that through tools. Tools are how you give it the ability to ask questions and get responses and and do things on its own. It's like a integration point into the LLM to some other system somewhere else. Yeah, exactly. The the common most easy to understand example is how does my model analyze the weather if it doesn't know what the current weather is in my local right. So you give it a tool saying hey give me the weather for this city name and it'll you know given a response from that it can say oh it's raining you should bring a jacket or something like that. So in our case, we want to be able to adopt this dog, which means scheduling an appointment, which is going to hook up to our, you know, some business application. And so there's going to be this tool that is going to enable that integration between those. Right. Exactly. So let's build a uh a tool here. Okay. Do adoption scheduler. And we'll have a uh it's going to you know schedule uh dog adoption appointment. This is our functionality that is going to be called as part of this right request cycle. And uh we're going to let's see we have to very important here right remember you were always taught to be to be uh clear and uh descriptive in your communication. It's all the more important here as a function of your AI engineering skills. So, schedule an appointment to adopt a dog at the Pooch Palace Dog Adoption Agency. So, and this is important because this is how the LM is going to decide which tool corresponds to what needs to happen. Exactly. And so, when you say something around schedule an appointment, it's like, oh, I have a tool that can do that, right? And the model and the model will figure it out. So, it's actually going to orchestrate the stuff in a sense uh for us. So we're putting these descriptions not just on the function name but also on the function parameters again so that the LM knows how to fill in the parameters in the right places with the right data. Okay, so here we go. And we're, you know, we just just keep it simple. We'll hardcode a response that is no matter what the date it is today, it'll be three days later, right? So we can say you can come on in uh three days from now, right? So uh there's our instant and we're going to just I don't know, let's just return uh a string for that like that. How about that? Okay, there's the uh exception. Um, and we're going to print out, you know, uh, confirming appointment for put in the parameters so we can make sure that this is getting called with the right data that we expect. Huge. It's going to be huge. I'm just love this part. Okay, so there's that, there's that, there's that. Let's uh, and now we have to tell our AI our chat client rather to take advantage of this tool to to make it available in its processing. And the way this works as you can imagine behind the scenes is that it's we're going to make a call to the downstream service. It in turn uh is going to know about the the the tools that are available to it and if it decides it needs one of those, it'll ask us to then invoke it locally and give it the response uh as required. So here's this. We're going to say scheduler just uh oh yeah, let me save that to a variable there. Do adoption. Okay, add that. Add that to the constructor. got two variables there for no reason. Okay, there we go. And now we go back down to tools and we can just point it to the spring bean, right? And then uh reload. Okay, so uh do you have any neurotic dogs? So the user is asking if there are any neurotic dogs available for adoption based on the context provided blah blah blah. Prancer with the ID45 is describe. Okay, great. So we clearly found him. So now next question which is when can I schedule an appointment to pick up Prancer at the London location. Okay, see what happens. Look at that. It says based on the tool results an appointment to pick up Prancer has been scheduled for March 17th. And remember we said 3 days later it's the 14th as you can see up here. Pi day. So three days after Pi Day. Three days after Pi Day. Very important. March 17th. Yeah. So it called our tool and we can confirm as much. Of course, that could do whatever would be needed in a real back-end system, but but it actually all worked. And we can see sure enough, we got the right parameters sent to that function call. Nice. And this is all local though. In the enterprise, local is not your friend. Maybe you want to centralize this business logic so that other parts of the ecosystem, the organization can benefit from that functionality. Right? In this case, the scheduling algorithm is a it's it's institutional knowledge. We want that to be reusable for all the AI applications in the organization. So we got to decompose and pull this tool out from being just built in. We have to make it a service. Yeah. If only there was a protocol that could only there was a protocol for this. Thankfully to Anthropic, there's the model context protocol MCP which creates this protocol for doing remote tools, right? And we've already got the model context protocol client configured here. We did add that in already. Let's before we get to that, let's go ahead and stand up the service so that this thing has something to which to connect. Okay. We're going to pull that piece out into its own server, its own MCP server. So again, I'll I'll call this service. Again, I'm I'm really good with names. Okay, so MCP model context server. We want the web support. Hit enter. Going to open this up in the IDE. And uh you know this is not hard. We're going to copy and paste, cut and paste even uh the logic for the scheduling. So here's our proprietary algorithm for refactoring. Okay, there you go. Done. And now we tell Spring AI MCP to uh export that. So we've got to like wire that MCP server into the Spring web, right? Um tool, what is it called? Callback provider. Yeah, that's Oh, tool callback. There we go. Okay. And we'll say do that. And we're going to say uh we want method tool callback provider build. And then here we're going to give it the tool objects. And again, just as before, we're going to point it to our Spring Bean. Just inject that thing in. Yep. Okay. And then finally, we want this to run in a separate port since it's not we would don't want it to conflict with the existing port 8081. And now you've got yourself an MCP service. Let's just spin that up. Amazing. Amazing. Yeah. Not bad at all. Okay. So now we need to build uh the client side of the of the coin, right? We've built the service. We need now to tell the adoption service how to connect to our uh functionality being exposed by that MCP service. So we'll build an MCP syncclient mcpclient and we're going to say um MCPC client.sync passing in the transport and this is where the rubber meets the road so to speak. We have to tell it so with MCP there's two default transports that you can use. One is standard IO, which is great if you're using MCP with the AI code assistant and you can run your servers locally, but for this type of kind of agentic server, we're likely going to have the MCP server running somewhere else. And so, we're going to use the server sent events transport for this MCP server. Absolutely. So, we're going to do that. That's the uh the hard part right there is just making sure you wrote that one little bean that points it to localhost 8081. Okay. And then uh we'll just uh we'll plug that in just as we did before here. MCP sync client and um we're going to point the tools to that. We're going to say new uh sync MCP tool callback provider passing in the MCP sync client. Do you have any neurotic dogs? Let's see. Neurotic dogs. Prancer. There's Prancer. Good. Next question. Natural. Next question. When can I schedule an appointment to pick a prer at the London location? And so now they should be doing the remote MCP call. And sure enough, it worked. Yep. The appointment to pick up Prince Ranch at the London location has been scheduled for March 17th. We can confirm as much by going over to our service. And there it is. Great. Not bad at all, huh? Nice. We uh we've done it. We've actually integrated, you know, our MCP service. We built up a brand new AI application. Uh we've used the chat client. We've done system prompts, user prompts. We did rag. We did all sorts of cool stuff. The last thing I think the thing that people might have on their on the top of mind these days is how do I do agentic? Agentic just means you let the AI control its own uh outcomes or its own flow, its own control flow. And um that's a very natural thing to do. You can actually have one model talk to another to evaluate the results. You can actually have routing. You can say okay the request comes in, use one model to do very efficient routing and then send it to another for further processing based on a specific set of tools and rag. Okay. Or you could actually even have iterations where one model says, "Hey, I wrote some code." The other one says, "That's good, but keep you could fix this." And the other one says, "Okay, I'll fix it. What about that?" And you keep going. That's just a while loop, right? Yeah. That's the way I look at it is when you go agentic, you're just putting this kind of stuff into a while loop and letting it iterate until it gets to a result. Yeah. So, you're not defining a a static workflow. You're letting the LLM iterate into in this loop until it gets to the desired outcome. And there's two types of agentic systems these days. There's workflows and then there's like what with you know the the agentic uh sort of style which is you say here's the end goal and just let it go which is very hard to get right. Most people aren't doing that. And then there's workflows which is basically multi-step processes that are you know mediated by by models along the way. Great. Either way easy to get to that from where we we are here. Right. So great. So we've shown kind of end to end with Java and Spring AI how to build a AI powered service and done the tools done uh many different types of integrations with the vector store with the embeddings and this was awesome. Super easy. I think any Java developer should be able to do this. Absolutely. Thanks for watching. We hope you got something out of it. If you liked it then like and subscribe. I already did. You did? Yeah. Awesome. Big fan. Well, hopefully you will give Spring AI and Bedrock a try and let us know in the comments how it goes and enjoy your journey to production. [Music]