Stop Building AI Chatbots (Do This Instead)

Key Takeaways

And and we spoke to a lot of the enterprise customers and we asked them, "Hey, listen, what are you doing with respect to AI? Do you really feel like you're ready for, you know, what's about to happen?" I mean, Cisco needs to do this because, I mean, we've got certain companies that are rushing ahead with AI development and they're releasing all these tools, but the security guys are like got warning bells going like crazy. Like, "How do we secure this stuff?" >> But what these models are not extremely, you know, familiar with is machine data. Yeah. And so, when you >> stuff like that, right? Correct. I love that cuz I was going to say, you know, vibe coding is all the rage for a lot of people these days and it's the worry is, okay, you get code, but it's full of these vulnerabilities. Everyone, David Bombal back with the amazing DJ. DJ, great to have you on the show again. >> Great to see you. Great to be here, David. Thanks for having me. So, I've got to ask you this. You talked to hundreds of customers. You've been working in the AI space for a long time. What's happening with actual adoption of AI? Oh, that's such a great question. And you're spot-on. I'm talking to customers. I'm talking to partners. We're here at Partner Summit. And and here's really what's going on, right? When you start to think about the wave of AI adoption that's going on. For hot second there, everybody was focused on chatbots. Yeah, I want to build a chatbot for this or that. But there's this massive shift that's happening where people are starting to recognize that it's not so much about the chatbots, but it's about building AI agents. Okay. And so, with this shift, what's really happening is there are three trends that we're seeing that is basically affecting like the architecture of what we think about the future of how we run these AI agents inside of our infrastructure, right? In talking to these customers, we're seeing three really interesting trends, right? There's a massive infrastructure constraint. There is a data gap that needs to be filled. And then the third one is a trust deficit. And I want to talk to you about every single one of these things, all right? Which are you going to start with cuz I know my favorite. >> I know. I'm going to start with um I'm going to start with the infrastructure constraint. >> Okay. Because if you think about what's really going on is with AI, you're going to need more compute. Yep. It's very obvious because you need more GPUs to be able to effectively do this. But with that, you end up needing more power. And then on top of that, you are you fundamentally need more networking to be able to connect all of these GPUs and all of these compute, you know, across each other, right? I'll give you a quick anecdote. When you when you think about these chatbots, right? Um a lot of times the interactions between a chatbot and a human is a sort of spiky nature. Yeah. >> You ask it a question, it comes back at you with the response. But with AI agents, it just keeps going on and on. It's constantly going back and forth between the GPU and the request that it that it it's it's basically, you know, getting a response back in in terms of the tokens. It reasons about it and continues to work night and day. The Codex agent, which is uh which came from OpenAI, Yep. has the ability to keep running for 30 hours straight. Wow. I have a personal experience where I have a Wi-Fi at home where um with in the wireless environment that I have, I have Meraki that's running it. I noticed I got an alert that said, "Hey, there's sudden spike in streaming media." For a second, I thought my kids are watching more Netflix. Yeah. Turns out it wasn't that. It's actually with the agents that I'm starting to spin up, they're constantly, you know, chatting back and forth and I and it takes, you know, a lot of tokens for it to be able to come back with the response, which means every single network that we're seeing, you know, has to be reimagined. Every single infrastructure that we're deploying has to now factor in this agent-based traffic that's going back and forth, right? And it's a large amount of traffic. And it's a large amount of traffic. Yeah. So, which means the capacity that you've planned for, if you're planning for, you know, uh a capacity for 8 billion people on the planet, it's going to start to feel like there are 80 billion at throughput capacity of 80 billion. And that's really what you're solving for when you start to think about what you what you really need. And that's that's infrastructure, right? Yeah. Now, when you start to think about the data gap. If you think about what's really happened with all of the models that are out there, right? The the publicly available large language models, they've been trained on a massive corpus of human data that's available, right? This is text, video, audio data that's available. But what these models are not extremely, you know, familiar with is machine data. Yeah. And so, when you >> stuff like that, right? Correct. A lot of the sensors are are emitting time series data, right? >> Yeah. These are varying over a period of time, which means, you know, you need different types of context windows. You need these models to fundamentally understand data on an axis of time to be able to start predicting trends. And so, so it starts to become really important to sort of factor that in in terms of reimagining what are the things that you're going to need to be able to bridge that data gap. And last but not least, trust deficit. >> My favorite. Yeah, cuz I I see it in the news often. Sorry, AI is hallucinating. Can't trust it. Sorry, go on. No, you're you're spot-on. I I I knew that would get you cuz yeah, I think this is one of our favorite topics to riff on, right? Now, with the with the trust gap, what we're seeing is, you know, fundamentally, you know, people are starting to recognize that there's AI safety that's important. That's your hallucinations and toxicity. Sometimes AI doesn't do the right types of things. So, a lot of the model labs are working on making sure that their models are aligned. They call it model alignment. >> Yeah. And and that's when the model sort of does the task that you're tasking it with, you know, you're asking a model to go find something, it understands that ask of yours, and goes out and executes on that stuff effectively. But you got to make sure that the model does that, right? So, you need the right type of guardrails for that. But beyond safety, there's also security. Yeah. This is where the attackers are recognizing that you're now running an AI system and are starting to attack it, you know, with prompt injection attacks or context window overloads and, you know, metaprompt extraction. They're constantly trying to come up with newer categories of attacks. And this is where it gets really really interesting, right? You have to make sure that your AI systems are trustworthy and they can actually predictably do the things that you want it to do, even though you're introducing non-determinism into that stack. So, at Cisco, we are systematically thinking about each and every single one of these things, right? With with the infrastructure gap, we just announced, you know, completely refreshed line of products here at Partner Summit. And uh we announced it at Cisco Live earlier and we doubled down here at Partner Summit where we talked about how we're launching the next generation of, you know, all the way from silicon. Like you you saw you know, G2 earlier today. Showed you the the new chip, the P200. And uh it's, you know, at this point in time from a price performance perspective, we're we're basically bringing to our customers and our partners the stack that is going to be ready for the future and solving that in infrastructure gap, infrastructure constraint. On the data gap, we announced it splunk.com. We talked about how we're reimagining the Cisco data fabric. And on top of it, we announced a uh an open source model, an open source open weight model called the Cisco time series model. I love open source. Go on. It's it's a time series model that can understand machine data really well. And then last but not the least, with the trust deficit, we we've been doing a tremendous amount of work with Cisco AI defense. Yeah. And and you know this, we you know, we've done demos, live demos. I'm sorry I don't have a live demo for you today. >> Shocking. Shocking. I'm kidding. [laughter] But what we have what I do have for you is this. We launched about 2 weeks ago, you know, something called the MCP scanner. Oh, great. It's an open source project again. I know you like it. >> Yep. And on top of it, as we see the shift on the on the trust side of the house from AI applications to agents, we recognize that the very first thing that everybody needs is to have a quick inventory of like, "What is my agent going to do?" And one of the interesting things that happened earlier this year was Anthropic had announced something called the model context protocol. Yeah. And that caught on fire. Everybody's using MCP servers to be able to talk to their AI applications. And so, what's happening is everybody's standing up these servers not knowing and not thinking whether they're doing this in a secure way or not. Yep. So, a lot of the security practitioners are basically saying, "Listen, I need a tool that can look through all of my my environments and detect if any of these MCP servers are compromised or they're using tools that they're not supposed to." And so, we put the entire tool in open source because we wanted folks to be able to >> immediately get out there and and start thinking through these things. And also improve with the with the community's doing with respect to this, right? Yeah. Um we also launched another project called Code Guard in the open source, which is everybody's now starting to code using um AI tools, right? Yep. They've got Codex from OpenAI. They're using Claude code from from Anthropic. And they're using these agents to be able to effectively write code for them. We recognize that if if you're going to be doing this work, and we did this in collaboration with Cisco security and trust organization, and and so, the way we we sort of came together with this was to be able to articulate if you if if agents are going to go write code, let us give the agents instructions to write safe code. Yeah. You know, making sure that there are no vulnerabilities in the code that they're writing. And so, we put together this Code Guard as a open source initiative to be able to give you this set of rules that you can easily take and plug into your agents and you get a certain level of code quality from a security perspective, which you could go out and deploy. I think that's really cool. I love that cuz I was going to say, you know, a vibe coding is all the rage for a lot of people these days and it's the worry is, okay, you get code, but it's full of these vulnerabilities. Exactly. And so, so I'm really excited about, you know, both of these projects because one of them is making sure that we're open sourcing the right types of like, you know, it basically tells you, "Hey, listen, I can scan every single, you know, MCP scanner that's out there." And the other one basically tells, "Hey, listen, I'm we're you can you can write code and you can secure it, you know, every single one of your agents are going to write better code as a result of this." And I think, you know, we're I feel like we're truly moving the needle from a from a trust deficit perspective. Yeah, I love this cuz I mean, there was a I can't remember, it was recently there was security researchers did a test of MCP servers on the internet and like 99% or some figure, I will put it on screen, were no authentication, just wide open. Exactly. The when you start to think about agents, I think the the big problem that still, you know, needs to be solved, and again, we're we're thinking about this really deeply and we're coming up with some really interesting prototypes on and reimagining what identity of agents is needs to look like, you know, Yeah. And and more importantly, you know, how do you qualify the behavior of these agents? How do you put the right types of guardrails around these agents to make sure that these agents don't do something completely, you know, bizarre. Um if you if you look at our big initiative on uh you know, called Agency, Yeah. uh which is our open source, you know, um project and an open source collective that we put together with 40 other companies, we have gone ahead and donated that project and it came out of our Outshift, you know, arm of of Cisco, which does like future-looking projects. They thought about agents deeply about the protocols that these agents are going to use to communicate with each other And since then we've gone out and donated the project to the Linux Foundation and you know, we're seeing some really interesting you know, feedback based off of that. We're seeing how people are using these agents, how they're thinking about identity, how they're thinking about security, how these agents communicate with each other. So yeah, we're we have a seat at the table for a lot of these things that's happening and and we're making sure that you know, we're bridging that trust deficit. >> I'm I'm so glad to hear this because it's a number one it's a lot of these this is open source, fantastic. And number two, I mean a lot of people the feedback I'm getting from the community is okay, AI is great and all agents are great, but do I really want to release this agent with privileges that you know, I have like admin privileges or whatever privileges without some kind of security on it. So really glad to hear that you're doing this. Yes, and we're doing this in close collaboration with our our identity intelligence team. As you know, we we've released a product in our earlier called identity intelligence, you know, they're duo side of the house and the team within duo is thinking really hard about how can we fundamentally reimagine and how do we plug this into that ecosystem so it's easy for folks to be able to go out and use their agents without having to worry about it. Now, you'll also notice that you know, about a few months ago we actually announced that we have our own duo IDP which allows you to integrate with your identity provider in a seamless fashion. So so we're we're just thinking through systematically what are the things that you're going to need to be able to you know, prepare yourself for an agent ready world. And so yeah, we're it's exciting times man. There's something new happening every single week. >> Yeah, it's just good news to do this because I mean we've got certain companies that are rushing ahead with AI development and they're releasing all these tools, but the security guys are like got warning bells going like crazy. Like how do we secure this stuff? We we ran this survey which is the AI readiness index. We talked to over you know, a sample of over 3,000 enterprise folks and we spoke to a lot of the enterprise customers and we asked them hey listen, what are you doing with respect to AI? Do you really feel like you're ready for you know, what's about to happen? And we got a very small number of just about 13% of them felt like you know, they had the right types of security controls in place or their production grade AI deployments. So there's a there's a massive gap right now in terms of folks that are moving their you know, proof of concepts into production and that's happening actively and we're making sure that you know, we are giving them the right sets of tools to be able to deploy AI without having to worry about it. DJ, I hate this. We're so short on time. I could keep you here for hours. DJ, thanks so much. David, thanks for having me.