Google's New AGI Test Explained

So, Google has developed a new way to test for AGI. So, let's talk about it. On March 16th, 2026, Google DeepMind quietly dropped a paper that might end one of the biggest arguments in AI, and the title is measuring progress towards AGI, a cognitive framework. And what it proposes is essentially an IQ test, not for people, but for AI systems. A way to finally measure whether a machine is approaching human-level intelligence across 10 dimensions of cognition. Not vibes, not benchmarks you can game, a full cognitive profile compared directly against how real humans perform. Now, here's the actual problem. Now, I think most of you guys understand this by now, that every major AI lab, OpenAI, Google, and Anthropic, all of them say that they're building towards AGI, but none of them actually agree what that means. OpenAI, if you guys remember, they've defined it as a highly autonomous system that outperforms humans at most economically valuable work. Google DeepMind co-founder Shane Legg defines it as an artificial agent that can at least do the kinds of cognitive things that people can typically do. And François Chollet, the creator of the ARC benchmark, frames intelligence entirely around skill acquisition efficiency. How fast can you learn something new? So, everyone's racing towards the finish line, but nobody can agree where the finish line is. How do you measure progress towards something you can't even define? Google DeepMind's answer is stop trying to measure AGI with a single score. Instead, break intelligence into its components parts, the same way cognitive scientists have studied the human mind for decades, and test AI on each one, head-to-head against real people. So, I looked at the paper, and it essentially talks about something called a cognitive taxonomy. It identifies 10 cognitive faculties drawn from decades of research, psychology, neuroscience, and it identifies 10 cognitive faculties drawn from decades of research in psychology, neuroscience, and cognitive science. And these aren't invented categories, they actually map directly onto how researchers have studied the human brain. The first eight are what they call the building blocks of cognition. So, first is they have perception. Can the system see, hear, and read? Not just detect pixels, but actually understand the scene, recognize speech, and interpret text. Two, generation. Can it produce useful outputs, text, speech, and motor movements, computer actions? Then we got three, attention. Can it focus on what matters and ignore what doesn't? This is where things get interesting, because current AI models process everything at once, and they don't really pay attention the way you do. Four, learning. Can it pick up on new knowledge after deployment? Not just during training, but in real time, the way you learn a new card game, or you adjust to a new job. And some people may even call this continual learning, which we know that many AI labs are currently working on. Five is memory. Can it store and retrieve information over time? And equally important, can it forget outdated information? And six is reasoning. Can it draw valid conclusions through logic, deductive, inductive, analogical, mathematical, all these forms of reasoning? Number seven is one that's really interesting, we're starting to see signs of this, is meta-cognition. Does it know what it knows? Can it tell you when it's uncertain? This might be the biggest gap in today's AI. As you all know, most models are sycophants, they're going to confidently give you the wrong answer, because they have no awareness in their own limitations. But, I will argue that maybe we can look at Claude and say that it does have some form of meta-cognition. Then we've got eight, executive functions. Can it plan, inhibit impulses, and switch strategies? These are the abilities that let you set a goal and actually follow through on it. Number nine is we have problem-solving. Can it apply perception, reasoning, planning, and learning together to solve real, novel world problems? Number 10 is social cognition. Can it understand social cues, infer what other people are thinking, cooperate, negotiate, and respond appropriately in social situations? The paper makes an important distinction here. This taxonomy focuses on what a system can accomplish, not how it does. It doesn't really care if the system uses transformers, diffusion models, or something entirely new. It only cares about the results. So, we've got all the categories, which are 10, and how do we actually test an AI against them? So, the paper actually proposes a three-stage evaluation protocol. Stage one is cognitive assessment. You essentially run the AI through a broad suite of tasks, and one for each faculty. And these tasks need to be targeted, meaning that they isolate one specific ability, and also they need to be held out and kept private, so that the AI hasn't just memorized the answers during training. And they need to be independently verified by a third party. And of course, if you guys don't realize, this matters way more than you think. One of the biggest problems with current AI benchmarks is data contamination. If a model has already seen the test questions during training, the scores don't really tell you anything about the actual intelligence. It is just pure memorization dressed up as reasoning. Stage two is to collect the human baselines. This is where you give the exact same tasks to huge demographs of representative samples of adults with at least a high school-level education, same instructions, same format, same conditions, and this gives you a real distribution of human performance to compare against. Stage three is to build the cognitive profiles, and this is where it gets visual. You essentially plot AI's performance on each of the 10 faculties against the human distribution. The result is essentially a radar chart, which you can see here. You can actually see three of them, and that shows exactly where the AI system is strong and where it kind of falls short. Now, profile A here shows the system that is kind of below the human median on several areas. Profile B shows a system above the median across all 10, and pro- meaning that it can, you know, match at least 50% of humans in every category. Then profile C shows a system at 99th percentile across the board, essentially matching or beating almost every human in the sample at every cognitive task. The paper is careful to note that even profile C wouldn't definitively prove AGI. No sample captures the full scope of human capability, but it would be a remarkable milestone. And so, this is where we have to think about, okay, we've got all of these things, but what is actually missing and what comes next? You see, the paper is honest about the limitations, because of course with everything there are some. So, firstly, the taxonomy only covers cognitive capabilities. It doesn't measure the responsive speed, and speed matters enormously. For example, think about like a self-driving car that can identify a hazard, but if it takes 6 seconds to react, that is basically useless in the real world. A coding assistant that takes 6 hours to fix a bug isn't really practical. And second, it doesn't really measure what the paper calls system propensities. Not what the AI system can do, but what it tends to do. Is it risk-averse or reckless? Does it align with human values like Claude's constitution? These behavioral tendencies will be critical for deployment decisions and governance. And third, there's of course the question of creativity. The paper acknowledges that creativity is notoriously hard to define and measure objectively. Rather than trying to isolate it, they argue that the cognitive processes behind creativity, like cognitive flexibility, world knowledge, and problem-solving, are already captured within the taxonomy. And of course, there's the thorny problem, model versus system evaluation. You see, today's AI systems aren't just a model. They usually have system prompts, tool access, they can call other AI systems, and testing the model in isolation doesn't reflect how it actually performs when it's deployed. But if you let it use all of its tools during testing, are you measuring the intelligence or the ability to just use Google? The paper compares it to giving a human a calculator during an IQ test. The person hasn't really gotten smarter, they just have better tools. And Google's conclusion here is that you need to evaluate the entire system, tools and all, but design cognitive tests carefully so that the tools don't change the results. Now, what's crazy about this is that this framework isn't just theory, it's not just some academic paper that they dropped. Google are actually putting money behind this. So, alongside the paper, they launched a $200,000 Kaggle hackathon asking the global research community to build those actual evaluations that we just talked about. [music] The hackathon targets the five areas where the evaluation gap is the largest, learning, meta-cognition, attention, executive functions, and social cognition. The prize pool includes $10,000 for the top two submissions in each track, and four $25,000 grand prizes for the best overall submissions. And it's open until April 16th, so if you want to try, I'd get started. And the results are going to be announced June 1st. I mean, when we look at the entire conversation right now, it's pretty broken. Shane Legg, who co-founded Google DeepMind, coined the term AGI, said in December 2025 that minimal AGI could arrive as soon as 2027 or 2028, which is only a year or two away. Not that long in terms of the time frame when we look at how quickly AI moves. Meanwhile, we look at the ARC prize, and they just released ARC AGI 3 this week, and it's an even harder test of novel reasoning, but the best systems with some tool usage still score around 24%, but most systems score around, I think, 0.6%, which is pretty crazy. You can see here, François Chollet says, "At the moment, ARC AGI 3 is the only unsaturated agented AI benchmark. Sub-1% scores from the front end models on the private test set." And if you want to be the first to know when an AGI-level breakthrough happens, monitor the ARC AGI 3 leaderboard, and any sudden jump in score will mean something important has changed about AI capabilities. Apparently, ARC AGI 3 is going to be your early warning signal. But of course, everyone has an opinion on whether AI is, you know, 2 years away or 20 years away. Without a shared framework for measurement, those claims are just vibes, and Google is essentially trying to turn those vibes into a real science. And so, when we think about everything here, Google is essentially saying that we spent decades trying to learn how the human mind works. We map the components, perception, memory, reasoning, attention, all of them. We already know how to test humans on these abilities, so we might as well just test AI the same way. Not with a single leaderboard score, not with a cherry-picked demo, but a full, real cognitive profile, and a radar chart that shows exactly where a system is brilliant, and exactly where it falls apart. Because the truth about today's AI is that it is jagged. They call it the jagged frontier. It can speak 150 languages, and it knows obscure facts about a small town in New Zealand, and in chain legs words, it still can't count the spokes on a graph the way a child can. And until we can see that full picture clearly, we're flying blind to want we're flying blind towards one of the most consequential milestones in human history. The question was never just can we build AGI, it was always how