AI workload storage options

Key Takeaways

[Music] Running high performance AI and machine learning workloads can be incredibly resource inensive. We're talking about massive data sets and compute requirements that are constantly pushing boundaries. But all that compute is only as fast as the data you can feed it. If your storage is slow, your expensive accelerator cluster could be sitting idle waiting for a file. So, how do you choose the right storage for your specific AI workload? Let's take a look at the top choice for two common jobs and the powerful alternatives so you can make the right decision based on your specific needs for performance, cost, and ease of use. Let's start with the most demanding phase, training. For this, manage luster is a perfect place to start. Manage Luster is a parallel file system with high throughput that works great for AI workloads. It stripes data across many disks to deliver up to 1 tabyte a second of throughput for both reads and writes with submillisecond latency. This high performance is critical for workloads that write large frequent checkpoints or jobs that access millions of tiny files. Manage Luster provides the throughput and low latency needed to keep your accelerators fully saturated. Another option for training is Google Cloud Storage. GCS is different from the traditional file system of Luster. It's an object store for direct use in your AI jobs. GCS fuse lets your compute instances mount a bucket as a local file system. You can also enable GCS anywhere cache to create a zonal cache on SSDs closer to your jobs. This delivers file throughput of up to 2.5 terabytes per second and offers 70% lower latency compared to reading directly from the bucket. The trade-off with GCS is that you may need to adjust your job to fit the object storage model and manually tune the cache to get the best performance. This is in contrast to manage luster which we expect to have high performance for most training jobs out of the gate without manual tuning. Now let's switch to inference. Here the priorities change and so do our recommendations. For inference, our primary recommendation is GCS with anywhere cache. For inference, cost effectiveness and flexibility are key. You can store your model in a single multi-reion bucket and then use anywhere cache to create high performance read caches in any zone where you have inference servers, bringing the model closer to your users. The alternative for inference is manage luster. In some cases, it's higher performing and could be a logical choice in a few key scenarios. For instance, if you're already using managed luster for training in a single zone, it's simple and efficient to use it for serving there as well. It's also the top performer for workloads with the strictest latency requirements, like if you're relying on a KV cache. So to summarize our storage recommendations for AI training, choose manage luster when you need the highest performance with the least manual tuning. Or you can use GCS with anywhere cache for an alternative, especially if you're willing to write your job with an object store in mind. For AI inference, your primary choice is GCS with anywhere cache for its balance of cost and flexibility. Manage Luster is your secondary choice when you need the absolute lowest latency or are already using it for training. By choosing your storage based on your workload, you can build the right foundation for any AI application on Google Cloud. To learn more, check out the links in the description. Thanks for watching and we'll see you in the next one. [Music]