The AWS AI Advantage: Custom Models

Key Takeaways

Our next segment is with our presenting partner, Amazon Web Services. Models have become a big battlefield for AI companies as they try to oneup each other with new releases every few months. Amazon has been working on its Nova model and made some new releases recently. And today I want to look at how the company thinks about making its product suite competitive there. Joining me now is Sha Nandy, a director at AWS. Shawn, welcome back to the show. It's great to have you here. >> Akos, it's so good to see you again. I am just back from a week in Vegas and adjusting to all this cold in New York City. >> Well, and it is cold. I'll tell you that, Sean. It's it it certainly taking taking us by surprise here. I want to talk about some of the announcements that Amazon and AWS made at reinvent. You know, I was really excited about the Nova models that the company talked about. Uh tell us about what's new in that family of products. Yeah, look, let me pull back for a second. Last week was our annual conference reinvent. We had 60,000 plus attendees. It was uh really amazing. I think it was the 13th reinvent. I hope I have that right. Um and I've been to 10 of them. So, it was my 10th. I went as a customer uh half of them and now as an employee. And I'll tell you, there were a lot of announcements and that's something we're proud of. We love to innovate fast. Now you asked about models specifically and I'll tell you the announcements around models came in three sort of categories in my head at least this sh's analysis uh first off we announced the largest release of new models on bedrock which is our key platform and the reason that's relevant is we had new model providers like we had the open source models from Google the gemo models we had mistrol's new family of models we had um just just a broad set in different sit types and and and shapes and we can talk about why that's actually important. Second, of course, we launched our new Nova 2 models which you mentioned, you asked what was different. I will get to that. And third, we announced a great capability called Nova Forge around allowing customers to meld their data with our Nova models. Talk more about that, but just more specific on what's new. You know, we enhanced our speechtoech models. Super exciting. We launched the first reasoning model, Nova 2 Omni, that is multimodal. So it can take all kinds of input and have all kinds of output and that's really important for an advancement in in the case of uh multimodal models. >> Now one of the the the product announcements that I was interested in is is the the growth in the uh family of custom models and the tools that that uh AWS offers. You know why are custom models such a big focus for the company? >> Yeah, look um let me give a little history. I won't hopefully bore the audience too much with too much history, but it's all sort of recent, right? So, in 2022 and 2023, I think a lot of customers I talked to, enterprises, of course, startups, we're like, we got to build our own foundation model. Everyone's like, we got to be differentiated. We don't want to look like everyone else. And we all realize sort of the industry building your own foundation model in 2223 took massive expertise, lots of cost. It was only viable for the largest players. That's why you saw these frontier model companies being so successful, including us, right? In 23 and 24, we heard all about rag, retrieve augmented generation, other techniques to bring your data into models, but not really change them. And now in 25, we're seeing customers ask again, the frontier model advancement is slowing a bit. I don't mean slowing in terms of you see great stuff happening every you know couple months but in terms of relevancy if you're going a couple extra points of accuracy does that really >> and and I we've seen it too is that you know like the the advances are are starting to plateau a little bit. >> Yeah. I you know I I'm sure there'll be something that'll break that plateau but for most use cases customers like has something I can do fundamentally changed my business and so they're asking again about how am I different? What gives me an advantage? And it comes back to their data Akos right like data is a company's advantage and the companies with more data they are in such a driver seat for what they can do. So instead of saying what model we should use, we're starting to say how do we unlock our data? Now to answer your question that you started with, right? What we released at reinvent was a series of capabilities across our stack. And I'll focus on one for a second to let customers use your data more effectively. And Nova Forge and the rise of what we're calling open training models is a capability where customers can meld their data with a curated Amazon data set and effectively uh retrain the Nova model to be their own frontier model. That's sort of the mental [clears throat] model and you know we've done it so that you don't have to have a bunch of data scientists and engineers and large numbers of GPUs like starts at a much lower price point and we'll see how customers react to this. I I'm bullish that it'll become uh quite a big thing. Now we've also introduced capabilities in Bedrock to allow easier fine-tuning. We've done a lot in the space, >> but the net is we're making it cheaper and easier to customize, >> right? >> And customization we think is a massive competitive advantage. >> Let me ask you this. You know, if you look at the process for developing these models, we've we've written about this recently at the information. I mean if you look at the phases there's there's pre-training there's evaluation there's post-training there's the launch obviously but if you look at these phases of developing models I wonder where you think is the biggest uh competition right now what is the biggest battlefield what's the hardest thing that to innovate in and how are you seeing that >> yeah look I I think that we have a lot of large companies maybe they're not large but they're emerging companies that are really like evolving how to build the next structure model, bringing the right research, the right capabilities, our own AGI team at AWS or Amazon included. But the thing I talked about with customization, there's more and more interest in pre-training. And I'll give you an example of like what you probably have to write about all the time, which is benchmarking. >> And every time a new model comes out, the company announces all the benchmarks. We do too, right? Cuz people want some sort of quantifiable how different is this model. I personally think one thing you're going to see change is customers are going to start to say I don't want to see your benchmarking model provider or hyperscaler I want to run my own benchmarking with my own use cases and I want to see how it performs and that's part of where things like continuous pre-training using your own data are becoming more relevant and we'd like to see that pre-training be the providence of customers themselves versus us the hyperscaler or you know versus a frontier model provider and that's what we're trying with Nova Forge put pre-training into the hands of customers and we're going to see how how that progresses, right? I mean, we had great early feedback from Reddit. Uh they're one of the customers who helped us work on Nova Forge as a customer. They were trying to improve content moderation and really bring in their data and they were able to reduce a bunch of specialized ML workflows with just sort of one cohesive approach using Nova Forge. So, but it's just one example like we're we're first with it. I'm sure others will innovate in this space and we'll see how it changes things. >> Right. Last question for you. We are coming up on the end of the year in 2026. It's it's now time for people to start making predictions about how the narratives around AI might change, what might be the hottest topics. I wonder as it relates to the discussion around models, you know, this year if we reflect, I mean, you know, there was a lot of talk about margins, there was a lot of talk about pre-training, a lot of talk about the benchmarks. How do you think the discussion might change in 2026? What sorts of predictions do you have? >> So, you're asking for Sha's opinion, not AWS. Um, I'll give you my opinion and I'll pick a couple areas and hopefully they're not too uh uh they're not too boring. They're interesting. Uh, first off, you know, a couple years ago, we talked a lot about model choice and people customers were like great, but I just want to know the answer like which model's best, which is cheapest, which is fastest, which is safest. I think the rise of Agentic over the next year or two will bring that model choice discussion to the forefront. >> Customers are going to want and they're going to use AI to help them do it. Really curated model selection for their use cases. So you're going to see many more models that are like targeted individually. That's part A. Part B is this customization thing. I do believe customization is going to be a big thing because it's becoming easier. And when we talk about ROI, enterprises care about ROI, right? We've got lots of questions on why a year ago not enough stuff went from proof of concept to production. Customers didn't necessarily feel the ROI. The investment's going down. You can do custom models cheaper. You can run models cheaper. You and I talked about this a few weeks ago where I said I think the cost of inference will drop by 90%. As all these costs drop, >> as inference costs drop, as model customization costs radically drop, as you have more openway models that can be had very cheaply, you're going to see organizations say, I want to tailor for me because the ROI doesn't have to be as well, the ROI will be big, the investment is not as big. So the return to justify that personalization and that's what is every company becomes some type of AI company. By the way, I'm not saying everyone's going to become Nvidia or us or Meta. But as AI becomes endemic in every company, just like internet became in every company, I think that personalization layer for that company will become so important using their data. That's the future. >> Great. Well, Sean, it's great to have you on. I appreciate you making you making time and uh we'll talk to you again very soon.