Telemetry Pipelines for AI | Amazon Web Services

Key Takeaways

Hello there. I'm Nolan Chen, partner solutions architect at AWS. And I'm Ozan Unlu, founder at Edge Delta. AI is on the top of mind of everyone these days. But your AI is only as good as the data you feed into it. Ozon, can you tell us how telemetry pipelines can help ensure that you have the optimal data going into your AI applications? Yes, I think even for observability and security, even with the massive data volumes that we're seeing in this space, AI can be an incredibly valuable tool for a lot of these data sets. So, let's take a look. Imagine that you have your three main sources of data. Now, this can be a simple app. You might have EKS data sources. You might have lambda functions. And then, let's say for instance, you also have uh network data. So these might be three independent data feeds, but together they might be creating quite a bit of data. So let's say you're an average enterprise that's creating about 100 terabytes of data a day. There's no way you're actually going to feed 100 terabytes of data every single day into an LLM. And so in that case, where telemetry pipelines can really help is what we call continuous curation and inference. So if this 100 terabytes is going into the pipeline, one thing that the pipeline can do in real time is do continuous curation inference to identify the high value nuggets of data within that full terabyte. And that actually might only be a very small subset of data. At the end of the day, if you're taking a look at any of the statistical deviations or anomalies within your data sets and saying, "Okay, within a 100 terabytes of data, my anomalous data sets might only actually be a gigabyte." Then you can feed that gigabyte into the host of Amazon Bedrock models and out you can get a really nice inference for observability and security use cases. So this is just showing how critical it is to be able to use pipelines as a component to be able to take a massive data volume as we see in observability and security use cases and be able to glean insight out of it using AI and this is what at edge delta we call continuous creation inference. Thank you. Thanks Ozan. I'd like to dive a little bit deeper. Can you tell us more about how this 100 terabytes can be reduced to 1 GBTE? Is AI involved in this process as well? Yes, certainly. So there are various different components. Some of those can be AI, some of those can be distributed machine learning. So let's take distributed machine learning for example. As that 100 terabytes is coming in, we have a process that actually uses cluster detection to patternize that data. So imagine that it's a 100 terabytes of log messages. That actually might be only 20 or 25 different types of log messages repeated millions or billions or trillions of times. And so in that case for instance one thing that we're identifying is okay you had a specific message you had that happen billions of times we can start to track that and actually baseline that over a long period of time. And so uh it could be uh an hour over hour delta. It could be a day overday delta or a lot of our customers actually look to look at week overweek delta. So for instance, what are my error rates for this specific service this week compared to last week at exactly this time? And if we see a statistical deviation, we can identify what are the 1 gigabyte of data within that 100 terabytes. You can use time frames, you can use metadata to really scope it down. And you can grab that 1 gigabyte and send it into Amazon Bedrock models to be able to say, "Hey, I might be having an issue. my error rates are going up. Can you give me the first suggestion or recommendation for what my operations team should do? And this is a great use case for Amazon's AI bedrock models. Thanks, Ozan. So, if I understand correctly, out of this 100 terabytes, you're you tend to have a lot of redundancy or repetitive patterns, but with this pipeline, you can identify the anomalous data and send that only to the AI system. Yes, exactly. Uh, you know, the reality is is that 100 terabytes of data, a very very small fraction of it is actually very close to that anomaly or close to the anomalous uh uh things that we're seeing within your environment. And so what pipelines can do is identify those anomalous data sets, those high-v value data sets, send them into AI to inference and say, why are we having this change? Why are we having this anomaly? And get the recommendations back from the LLM. Now, you don't have to use those recommendations, but I know when I'm waking up at 3 in the morning and my eyes are adjusting, it's really nice that I already have AI that recommends how I'm supposed to fix the issue instead of me having to start at square one. So, you can think of it almost as a way digitally finding that needle in the haststack, right? Absolutely. And it sure beats a human trying to plow through all those terabytes of data trying to find out what went wrong. Oh, you know, I'm not sure if it quite beats a human these days, but it's a lot faster. Um, it can it can happen um in fractions of a second. And uh this whole process to be able to get that data uh package it up, send it into AI, uh that can be extremely quick. The the inference uh mechanism in AI itself, sure that might take a few more seconds, but certainly humans plus AI is going to be a lot more powerful than us humans on our own. Thank you. I think that's the key. humans plus AI with a tele tech telemetry pipeline makes it much easier now to perform inference and find help resolve issues in your systems using lots and lots of telemetry data. Absolutely. You couldn't be more right known. Thank you, Ozan. Thank you.