How to Use the Gemini Batch API for Processing Large Datasets

Key Takeaways

Hey there friends, today I'm going to show you how you can use the batch API to process large volumes of data with all of our Gemini models. And this can be anything from Gemini 3 flash, Gemini 3 Pro, Nano banana, the embeddings models, um to whatever your favorite flavor of Gemini model might be. Um batch mode is pretty special in the sense that for jobs that don't have um requirements for immediate turnaround time, you can process them on a a more extended time frame up to 50% cheaper than the standard cost. Um and the target turnaround time is around 24 hours, but just being very honest, pretty much every job that I've tried for a reasonably sized data set of less than 1,000 examples has run under a couple of hours. Um so it uh is a great way to save money um if you don't have uh kind of the need for immediate results. Um and is also relatively straightforward to get started. So today I'm going to show you a Colab notebook that I've created specifically for this purpose. Um I've defined uh my Gemini API key just in the secrets of this Colab notebook. Um I'm pulling in a data set from Hugging Face, so the the HumanEval data set, which is commonly used as a proxy for um for coding, but is actually just testing how well you can complete a Python function definition. Um it's used a lot for evals though, so it makes a pretty handy getting started example. I've converted them all to JSON lines. Um and then if I click this play, um you should see that uh the data set kind of gets downloaded, converted, um and we get a handy little uh job state pending um status for the uh the example uh seen here. Um if I want to keep kind of um pinging the job state, I can, so I'm going to to click again, and we can still see that it's pending. Um but as soon as the job uh you know, kind of toggles towards succeeded, um I can walk through each of the examples, so each one of the uh kind of the the uh the sort of requests that I've sent to Gemini and the responses that they've gotten back, um and do a kind of a score of the model um based on those data. Um I've already done this, um so as I mentioned, the I've run this job a couple of times, they've all come in at well under an hour in terms of completion. Um this is using the Gemini 2.5 flash light model, um, which you can see that I have defined here as part of creating my batch job. Um, it's under, you know, like five lines of code in order to, um, to be able to do this work, which is pretty awesome. Um, and then, uh, if we scroll down, we can see kind of for each one of these examples, um, whether the model passed or whether there was a failure, what kind of failure it was, um, and then also see, uh, kind of the the end results for the data itself, um, uh, visualized in kind of this nice, um, this nice, uh, sort of friendly map plot format. Um, so this, uh, I'll be adding this Colab notebook so you can kind of mix and match, experiment with other data sets, um, experiment with other models, see how well they perform. Um, but that really is just as simple, um, as a few, uh, a few cells in a notebook to get started with the batch mode in the Gemini API.