NVIDIA Nemotron 3: 1M Context, Hybrid MoE Architecture, and Open Source AI Agents

Key Takeaways

Okay, let's unpack this. >> Let's do it. >> Today we are diving deep into a strategic model release that I mean it doesn't just promise better performance, it feels like it's fundamentally rewriting the blueprint for open-source AI architecture. >> Yeah, >> we are talking about Nvidia's Nimatron 3 family of large language models. And this is uh this is a really pivotal moment. Our source materials and we've got the highly technical white paper, the detailed Nemo user guides, even developer community discussions, they all confirm it. >> It's not just an incremental update. >> Not at all. These aren't just faster models. They represent a fundamental architectural pivot toward what we're calling agentic AI. >> And for anyone new to that term, what exactly does agentic AI mean here? >> It means AI systems that can perform complex multi-step autonomous tasks. So think beyond just simple chat. The architecture here, especially in the new Nimatron 3 Nano, is customtailored for extreme efficiency and reliability in those uh mission critical workflows. >> The sources we've gathered feel less like an announcement and more like I don't know a full engineering guide to the future. We're dissecting these novel mixture of experts designs, seeing how Mamba layers are being used for maximum speed and examining these aggressive new low precision training formats. It's a gold mine. >> It really is a gold mine for anyone looking to understand not just what the next generation of AI looks like, but how it's built and you know critically how you can optimize it for commercial use. >> So our mission today is pretty clear. We need to quickly but thoroughly distill why Neotron 3 and especially this new nano model is being positioned as probably the most efficient and open model family released to date. >> And we really have to focus on the specifics, >> right? The often hidden architectural innovations. We're going to be talking about things like hybrid MO, latent MO, and the training regimens that allow these models to be I mean simultaneously massive in knowledge, minimal in compute cost, and this is the crazy part capable of handling an unprecedented 1 million token context length. >> And the strategic context here is just impossible to ignore. I mean, before we even get into the silicon level details, you have to understand this. >> Yeah, >> Nvidia isn't just releasing the finished product. They are giving away the entire recipe, >> the whole playbook, >> the data sets, the weights, the complete training guides, the environment simulators. This level of openness, it's a calculated strategic move. >> Absolutely. It's designed to embed their hardware optimized architecture as the universal standard for open agentic AI development. It ensures that anyone building the next great AI agent, well, they're probably going to build it on NVIDIA's platform. >> It's simple, really. Yeah. The better the open- source blueprints, the more chips they sell. >> So to start, I think we need to frame Nematron 3 not as a single model, but as a complete product line. >> Okay, >> it's structured into three distinct tiers. And these are designed to meet, you know, varying enterprise needs from high throughput task automation all the way up to state-of-the-art complex reasoning. This tiered approach, it shows a really serious commitment to the commercial market. >> It's a classic infrastructure strategy, isn't it? You provide an entry model to capture volume, a mid-tier for more collaborative work, and then a flagship model for maximum performance. >> Precisely. And the initial and I think most immediately disruptive release is the Natron 3 Nano. >> The small but mighty one. >> That's a good way to put it. Its size is key. It has 31.6 billion total parameters. But, and this is the important part, because it uses a sparse architecture, it only activates 3.2 two billion parameters or maybe 3.6 if you include embeddings for any given calculation. >> And that ratio is absolutely critical for the listener's bottom line, right? Because you're only paying the compute cost for 3.2 billion parameters even though the model has the knowledge breadth of something 10 times larger. >> Absolutely. The Nano's role is defined by efficiency. It's designed to be the most cost-effective solution built for high throughput inference where speed and repeatability matter most. >> So you can think of it as the tireless worker be of an AI system. Exactly. Handling rapid, repeatable tasks without incurring the massive operational expense you'd get from a dense model. >> Okay. So, moving up the ladder, where does the complexity start to increase? >> That takes us to the mid tier, the Neatron 3 super model. This is where the models are optimized for more complex collaborative agent systems. >> Like what kind of systems? >> Systems that might automate a high volume of structured but you know varied workloads. A good example is sophisticated IT ticket resolution or maybe interacting with multiple internal systems at once >> on the scale. >> This model is expected to use around 10 billion active parameters out of a total of roughly 100 billion. And our sources indicate that both super and the largest model Ultra are slated for the first half of 2026. And then of course we reached the absolute peak of the platform. The model that's designed to showcase what this architecture can truly achieve. The Neatron 3 Ultra. >> Ultra is the flagship. It's focused entirely on state-of-the-art accuracy, deep reasoning, and maximizing performance for these highly complex multi-dommain applications. >> We're looking at a huge scale here. >> A huge scale. Approximately 50 billion active parameters. And those are drawn from a colossal 500 billion total parameters. >> Half a trillion. >> Half a trillion. And at that scale, Nvidia is aiming to dominate fields that require, you know, graduate level scientific reasoning, novel problem solving, and applications that simply cannot tolerate errors. >> Okay. Now, let's circle back to the openness because, as you mentioned, this is a strategic move, not just a goodwill gesture. Why is the sheer volume of material they're releasing, the data sets, and the recipes, so much more important than just a standard openweight dump? It's about reproducibility and trust. When developers receive model weights, they often face a blackbox problem, >> right? You know it's good, but you don't know why it's good. >> Exactly. And more importantly, you don't know how to reliably fine-tune it without breaking it. Nvidia is just circumventing that entirely. They are providing the model weights plus the precise training recipes all neatly packaged in the Nemo developer repository. So it ensures that when you optimize their model, you are doing so on a path that they guarantee will work best on their hardware. >> You got it. >> And the data releases, I mean, it's truly shocking in its scale. Over 10 trillion tokens of data sets. That's an almost unimaginable amount of highquality curated training material. >> It's massive. And it includes proprietary efforts like Neimotron CCV 2.1 which alone contributes 2.5 trillion new English tokens that have been meticulously cleaned from common crawl. >> Wow. >> And that's alongside huge amounts of specialized synthetic data for coding and STEM fields. By providing this data, they're essentially shortcircuiting the need for competitors or developers to spend years and millions of dollars on data curation. They're just saying here here is the best data trained on the best architecture ready for you to customize. >> That's the message >> and that level of transparency. It directly enables the Neotron purpose which you said is agentic AI. How specifically do the sources define this agentic focus. >> It's defined by capability. The white paper focuses less on say conversational fluency and more on verifiable performance in structured multi-step environments. So less how are you today and more solve this complex problem >> precisely they highlight mastery in advanced math complex coding and here are the two keywords for agents robust instruction following and sophisticated tool calling. >> What does that mean in practice? >> It means the model is optimized for taking a highle goal breaking it into subtasks interacting with external APIs or databases and managing all those intermediate results. >> Okay, give me an example. If you are building an AI that needs to say resolve a complex customer issue by checking three different databases, then generating a piece of code and then filing a report, Neatron 3 is the intended engine for that workflow. >> So if the ultimate goal is complex, reliable and cost-effective agents, the architecture just has to deliver speed without sacrificing that reasoning depth. >> It has to. >> And this brings us to the core engineering breakthrough of the nano model. Here's where the architectural genius I think truly shines. To deliver that speed and cost efficiency for agentic workflows, Nvidia had to solve the problem of inference scaling. And they did this by fusing three kind of disperate ideas into one. The hybrid mixture of experts mamba transformer design. It's a mouthful, but the concept is revolutionary. >> It really is. I mean your standard large language model your LLM is pure transformer and its main bottleneck is the self attention mechanism >> which is computationally very intense >> very intense and crucially it creates this memory burden known as the KV cache >> that cache stores the keys and values of every single token that's been processed and it grows linearly with the sequence length >> so longer reasoning traces or massive context windows translate directly into slow expensive inference. Exactly. Now, MOI, mixture of experts, it's solved the model size problem by sparsely activating only a fraction of the parameters like we discussed with Nano's 3.2 billion active parameters. >> But that still leaves the linear scaling problem of the self attention layers when you're dealing with long sequences. >> And that is the crucial leap that Neatron 3 takes. Instead of interle layers primarily with those expensive self-attention layers, they use much cheaper Mamba 2 layers predominantly. >> And Mamba is a selective state space model, right? which is just much more efficient for sequence processing. >> That's it. So the architecture becomes this kind of three-part harmony. You have MOI for the knowledge breadth, Mamba for sequential speed, and then the transformer elements for global coherence. >> So if you actually looked at a diagram of the layers, >> you'd see it's clear. Neatron 3 is mostly Mamba 2 and MO layers. And the core benefit of Mamba 2 for you, the user, is speed and generation. Mamba layers process the sequence by maintaining only a constant state. They don't need to look back at the entire history of tokens. >> Okay, let's use an analogy here for clarity. If a standard transformer is like a courts stenographer who has to reread every single line of testimony before typing the next word, >> then Mamba is like a seasoned jazz musician who only needs to keep the rhythm and key in their short-term memory to improvise the next note. >> It's constant time complexity, not linear. >> That's a great way to put it. By minimizing those resource-hungry self-attention layers, they directly tackle the inference bottleneck for workloads that demand generating long outputs >> like an agent generating a detailed multi-step plan or writing thousands of lines of code. >> Precisely. >> But, you know, historical attempts to move away from self attention or even hybridize it have sometimes led to models losing that ability to grasp global relationships within the text. So how does Neatron 3 maintain the high fidelity needed for complex reasoning if it's minimizing attention? >> That's the critical design choice. They don't eliminate attention. They use it surgically. >> Surgically. >> They include a select few full self attention layers that are interspersed throughout the network. These are the layers that provide that necessary alltoall information routing. [snorts] And that is absolutely crucial for tasks like recognizing a dependency between a token at the very start of a 100,000 token document and one at the very end. >> So it's a pragmatic compromise. >> It is gain efficiency 90% of the time with Mamba, but pay the cost for attention that 10% of the time when global context is paramount for reasoning. >> This all sounds great in theory, but what's the payoff? I mean, what do the numbers say about this efficiency gain for the listener's cloud bill? The efficiency payoff is the aha moment. It really is. The sources give a direct comparison on a common agentic reasoning workload. >> What was the task? >> Specifically, processing an 8,000 token input and generating a 16,000 token output. This is a task that heavily penalizes linear scaling architectures. And the result >> in this test, the Neatron 3 Nano30B A3B achieved 3.3 times higher inference throughput compared to a competitive similarly sized transformer MOI model Quen 330B A3B >> 3 times faster. Okay, let's translate that into economics. If you run a high volume agentic workflow, say processing a 100,000 support tickets a day, each requiring a 16,000 token output trace, >> you are now paying roughly one-third of the compute cost for the same performance. or you can run three times the volume for the same price. That's a generational cost disruption. >> It radically changes the total cost of ownership for any large-scale AI deployment. And what's more, the nano model is reported to hit output speeds of approximately 380 tokens per second on serverless endpoints. >> And that's not just a lab result. >> No, this is realworld speed that ensures low latency for agents that need to operate in real time. This combination of speed and low active parameter count, which is also noted to be highly competitive on benchmarks like the ruler score for long context utilization, it just confirms that the hybrid MO architecture isn't a technical curiosity, it's an economic weapon. >> So, the nano model sets the standard for speed. Now, we need to look at the larger models, super and ultra, where Nvidia focuses on scaling up quality and accuracy without losing that speed advantage. >> Right? Speed is one half of the equation. Accuracy and deep reasoning are the other. And as the models grow to the super and ultra size approaching that 500 billion total parameter mark, the challenges shift. They move from compute cost to communication bottlenecks between hardware components. >> Okay. >> And this is where latent mo comes in as the first major innovation in the larger neatron tiers. >> This term latent mo, it suggests some kind of compression or hiding information. >> How does it tackle the hardware communication problem? It's a brilliantly hardware piece of design. In a standard massive MOI model, when a token needs to consult specialized experts, the system has to send this huge chunk of data, the to in embedding and routing information from the GPU memory to the specific expert compute unit, >> right? >> And when you have hundreds of billions of parameters, this all communication of large data packets between components becomes the primary bottleneck. It just slows down inference. So if the data packets are too large, the system spends all its time waiting for the data transfer, not actually doing the calculation. >> Exactly. Think of the data packet as a massive highresolution TIFF file that you need to email to a dozen colleagues. >> Okay. >> Leon M is the equivalent of zipping that file first. The input token embedding starts in the model's large hidden dimension. let's say a dimension of D which is maybe 40 and 96 dimensions deep and it's immediately projected into a much smaller latent dimension uh which could be say 124 dimensions >> so you're shrinking the communication payload by a factor of four right before it even hits the network interface right >> what immediate capacity does that unlock >> it unlocks massive capacity first this compression reduces the communication payload and the per expert weight loads by that factor of DO that's an immediate speed up and reduced latency. >> But there's a second benefit. >> The strategic genius is the second benefit. The computational capacity saved by reducing that communication bottleneck is instantly reinvested. They use the safe capacity to increase both the total number of experts and more importantly the number of active experts that are consulted per token. >> So you shrink the message which allows you to send more messages and that lets you consult more specialized parts of the model's brain all at once leading to a higher quality answer. >> Precisely. The goal isn't just speed. It's getting higher model quality for the same computational budget. And we have hard data showing how effective this is. >> What does it show? >> A standard MOE bus line might use, say, six active experts out of 128 total. The Neatron 3 latent MO architecture because of its communication compression can afford to use 22 active experts out of a total of 512 experts. >> Wait, so that's four times the computational diversity consulted for every single token. That's a massive nonlinear increase in processing power. >> And the results clearly demonstrate this focus on accuracy at scale. Implementing Lemo resulted in significant accuracy jumps on hard reasoning tasks. For example, MMLU Pro accuracy, which is a very tough benchmark, leaped from 48.3% in the standard B line to 52.87% with a latent MO system. And that confirms that for the largest models, how you manage hardware communication is just as important as the parameter count itself. >> It's optimization engineering at its finest. >> Okay, so that innovation handles scaling quality. But the second critical innovation for agentic speed is multi-token prediction, MTP, which focuses on pre-planning and generation efficiency. >> MTP is all about predicting the future, quite literally. While traditional LLMs only predict the immediate next token, MTP involves training the model to predict multiple future tokens simultaneously. >> That sounds incredibly difficult to train. The model needs to maintain coherence over a greater span. What does this offer during the training phase? >> During training, it provides a much richer multi-step training signal. It forces the model to move beyond just shortsighted next word prediction and to develop an internal planning horizon >> which is absolutely vital for agentic behavior. >> It is agentic behavior requires multi-step look ahead and planning. The ablation studies confirm the payoff. MTP provides an average performance improvement of approximately 2.4% across crucial reasoning benchmarks including MMLU Pro and GSM AK >> and a 2.4% gain on highly competitive reasoning tasks. from looking ahead is substantial. >> It really is. >> So MTP makes the model smarter during training. But how does it deliver that speed boost during inference? >> This is where the magic of speculative decoding comes in. The predictions made by the MTP module serve as highquality draft tokens. >> Draft tokens. >> Yeah. So when generating text, a lightweight MTP module quickly spits out a sequence of these draft tokens. The main heavy Nematron 3 model then only has to validate that entire batch of draft tokens at once instead of calculating and confirming each token one by one. >> And if the main model agrees with the draft, >> you accept the whole batch and you jump ahead significantly in the generation process. >> That sounds like a massive latency improvement. But the efficiency depends entirely on the reliability of those draft tokens. If the lightweight module is constantly wrong, the heavy model spends all its time rejecting and recalculating everything. That's the crux of it >> and the reliability is exceptional. The sources note that the lightweight MTP module achieves a very high acceptance rate. We're talking around 97% acceptance on the first two predicted tokens. >> Wow. >> When you have draft tokens that are that reliable, speculative decoding becomes a powerful practical accelerator. It enables the Neimotron models to deliver text generation with much lower latency, which is absolutely essential for responsive agents. Okay, so we've established speed and quality. Now we need to turn to the sheer scale of the information these agents have to handle. I mean, enterprise agents need to sift through gigantic code repositories, long legal documents, or years of corporate communication, >> right? >> And that brings us to Nematron 3's industry-leading 1 million token context window. >> Handling a million tokens reliably is a monumental task. And the architectural choice that enables this in Neatron 3 is the use of no PE or no positional embeddings within the attention layers. >> Wait, that sounds counterintuitive to everything we know about transformers. How does the model know where a token is located if it has no explicit positional encoding? >> So, because the Nimatron 3 architecture is a Mamba transformer hybrid, the Mamba layers inherently handle the sequence modeling. They provide implicit positional information through their state space mechanisms. And this allows the attention layers to completely avoid techniques like rope or rotary position embeddings. Now ropey is ubiquitous, but it suffers from a well-known flaw. The extrapolation cliff. >> The extrapolation cliff. What is that in simple terms? >> Think of it like a highly detailed road map that's designed for a specific city limit. If you try to use that map 10 miles outside the boundary it was trained for, the grid lines and the orientation become meaningless. The map just collapses. With rope east models, if you try to input a context window that's significantly longer than the length they were trained on, say you push a 32k context model to 100k, the performance tends to just drop off abruptly, making that long context unreliable. >> So the nope architecture allows Neatron 3 to avoid that cliff entirely, meaning their performance degrades gracefully rather than just catastrophically when pushed to the limits. >> Exactly. The source data provides really compelling evidence of this robustness to extrapolation. When they tested the Neotron 3 Nanobase model beyond its standard training length all the way up to the 1 million token limit, it showed a graceful degradation. >> What were the number? >> It maintained a ruler score which is a metric for long context utilization of 54.19 at 1 million tokens. Now, compare that to the previous dense hybrid model, Neatron 2, which suffered an abrupt drop off, plummeting to a ruler score of just 23.43 at that same length. >> That difference, 54 versus 23, is the difference between a functional, reliable agent and one that just suddenly has an information processing breakdown when it's faced with a large file. >> It means developers can trust the model to handle unexpected large inputs without an immediate failure. >> But do we have proof that the model is actively using this massive context? It's not just keeping the window open. >> We do. They measured the negative log likelihood or NLL on huge code sequences up to 1 million tokens. And the NLLL, which basically measures the model's uncertainty, continuously decreased as the sequence length grew. >> So, it's getting more confident as it reads more. >> Exactly. It confirms that the model is actively integrating information from the far reaches of that 1 million token input to make better predictions. It is truly using the context which is paramount for tasks like understanding legacy code bases. >> Okay. Moving from architectural scale to hardware precision. The training of the larger super and ultra models introduce a really aggressive optimization. NVFP4 training. This sounds terrifying from a data integrity perspective. >> It is aggressive, but it's a necessary step toward the future of hardware optimized AI. The purpose is all about maximizing hardware utilization. By training these massive models using the NVFP4 4-bit floatingpoint format, they enable the use of native NVF4 matrix multiplication operations on the newest NVIDIA silicon like the GB300. >> And the benefit, >> this provides a peak throughput advantage that is three times higher than using FP8 precision. It's an enormous speed gain during the training process, which leads to quicker iteration and lower energy costs. But training at 4-bit precision means you are quantizing your model parameters down to a point where you risk losing critical information. How on earth do they avoid collapsing the model's accuracy? >> They employed a meticulously engineered hybrid precision recipe. The principle is simple. Use NVFT4 wherever you can to get that 3x speed boost, but be extremely conservative and keep sensitive components in higher precision like BF-16 or MXFP8 to preserve information. >> Okay. So what parts of the network did they identify as being too sensitive for 4-bit quantization? >> This is where the deep engineering knowledge really comes in. They found several critical areas. First, the Mamba output projections. They kept these in MXFPA because the nano model when fully quantized showed that up to 40% of its outputs were just flushing to zero. >> That's an unacceptable loss of fidelity >> completely. Second, the fundamental layers of the transformer components, the QKV layers and attention projections were maintained at BF-16. And third, the unique neatron 3 layers, the latent projections for latent mo and the MTP layers were also kept at BF-16 precision. >> So that's a very sophisticated surgical approach. It's high speed everywhere possible, but highfidelity is maintained only where the network is making mission critical decisions >> and the results really validate the strategy. They successfully minimize the loss gap between the aggressive NVFP4 trained models and the full BF-16 baseline. On the larger models, the loss gaps are reported to be less than a 6%. >> Which is tiny. >> It is. This stability ensures that while the training is massively faster and cheaper due to the 4-bit format, the resulting models maintain competitive downstream accuracy. It proves they successfully manage the risk of aggressive quantization. Finally, we need to talk about the post-training strategy because this is really what molds the model into a specialized agent and that's the multi-environment reinforcement learning RL. >> Right? This moves the model past simple pre-training and fine-tuning. Neatron 3 models are trained simultaneously on a very wide range of RL environments that are contained within the Nemo RL and the Nemo Gym collection. >> And this collection simulates all kinds of different environments. Yes, diverse environments including advanced math problems, complex coding challenges, tool use scenarios, and long context retrieval tasks. >> So why is training them simultaneously on all these environments better than say focusing on math first, then coding, then tool use? >> It's necessary to prevent what the sources call reasoning drift. >> Reasoning drift. >> Yeah. If you optimize a model intensely on one domain, it often forgets or degrades its skills in another. For example, focusing purely on coding might degrade its complex instruction following. >> I see. >> So the simultaneous training approach is significantly more stable. It ensures that performance gains are steady, uniform, and generalized across the entire spectrum of required agent capabilities. And the charts in the paper illustrate this beautifully. You see steady, concurrent gains on everything from Amy 25 and GPQA to live codebench and mmlu pro. It demonstrates a general increase in agency, not just a specialized skill. All right, so now that we've established the architectural foundation, let's pivot to the practical side. How can the listener, whether they're a professional developer or an advanced enthusiast, actually utilize and customize these open source blueprints using the Nemo ecosystem? >> The openness of the blueprint suggests that customization should be straightforward. But let's be honest, with massive models, fine-tuning is still a huge hurdle for a lot of people. It remains complex for sure, but Nvidia has lowered the barriers significantly by releasing these predefined fine-tuning recipes. And these recipes, they cover a wide spectrum from the smaller 8B model size all the way up to the planned 253B versions. >> And these are all configured within the Nemo 2.0 framework. >> Correct. And developers have crucial choices here. You can opt for lore a low rank adaptation which is applied by default to all linear layers for maximum efficiency during fine-tuning. >> So that's the cheap fast path. >> That's a cheap fast path. Or if you have the resources and you want maximum performance, you can choose full fine-tuning by setting the PEF scheme to none. >> And for those running their own clusters, how do the recipes help maximize hardware utilization during fine-tuning itself? >> So the recipes include some really important efficiency tweaks. For instance, developers can use the packed sequence true setting for sequence packing. This is a powerful technique that optimizes the use of GPU memory and processing time by combining multiple short sequences into a single maximally dense training batch. >> And that just increases throughput. >> It dramatically increases throughput during fine-tuning, allowing you to iterate on your customized agent much, much faster. >> That's great for the enterprise user, but what about accessibility for local development? We're talking about models with tens of billions of total parameters. Can the average enthusiast actually play with the nano model on their desktop? >> Yes. And this is a huge point for democratizing agent development. The Neatron 3 Nano GGUF variant, which is the highly optimized format for local inference, can be run on devices with just 24GB of RAM or VRAM >> or unified memory if you're on a certain platform, >> right? And this makes the nano model which again is a high performance 30 billion parameter class model with a 1 million token context capacity accessible to a vast swath of the consumer and proumer market >> that completely resets the benchmark for local model capability. You can now build sophisticated agents on accessible hardware. >> It drastically lowers the financial and logistical barrier to entry. And moving beyond local accessibility, let's talk about the feature that speaks most directly to corporate users who are obsessed with cloud costs. granular reasoning, budget control. >> You mentioned this is like an accounting feature. Why is controlling the thinking process so important for commercial agents >> because internal reasoning that process where the agent works out the steps to solve a problem is computationally the most expensive part of any query. If an agent automatically launches into a 10,000 token internal thought trace for simple task, you've just wasted money and introduced unacceptable latency. This feature gives developers crucial financial predictability. >> Exactly. >> So how is this budget control mechanism implemented at the token level? >> The models are specifically trained to respond to control tokens. They use the think token which is ID12 to start an internal reasoning trace and a think token ID13 to end it. >> And the user or the developer can specify a budget, >> right? A maximum token count allowed for that thinking trace. So if the model is reasoning and it hits say a 500 token limit, what happens then? >> The system forces the model to append the closing think token and immediately generate the final concise response and it does this regardless of whether the internal reasoning was fully complete. >> So it allows the user to dynamically manage that accuracy efficiency trade-off. >> Precisely. For a high stakes complex financial query, you might set a budget of 5,000 tokens to ensure maximum accuracy. For a standard internal request, you might cap it at a 100 tokens to ensure speed and low cost, >> complete control over the expensive part of the process. And I understand you can toggle this behavior really easily. >> Yeah, the sources confirm that in environments like the Llama server context, you can simply use the system prompt to switch this behavior on or off. You can input detailed thinking on for a full trace or detailed thinking off to receive only the concise final response >> which makes the modus far more predictable and financially manageable in production environments. It does. Okay. Finally, let's talk about the support system. It really sounds like Nvidia didn't just release the model, they released the entire laboratory ecosystem they used to build it. >> That's the strategic move in full effect. They released the Nemo Gym as part of the open- source package. This is the exact environment library they used to build and scale those multi-environment RL training systems we talked about. >> So now any developer can access the same environments that Nvidia used. >> Any developer the same competitive coding, advanced math and tool use environments. It fundamentally democratizes the process of designing, fine-tuning and testing reliable high-erforming agents. >> And for those who are ready to deploy, what are the options and what's the real world pricing? Deployment options are comprehensive. They cover their own NVIDIA NIM micros service, hugging face, public clouds like Amazon Bedrock and third party inference providers like Deep Infra. And the commercial pricing for the costefficient nano model is clearly laid out making that efficiency tangible. >> What are we talking >> approximately 06 per $1 million input tokens and 0.2410 per $1 million output tokens. >> That is aggressive pricing. And when you couple that with the Nano's demonstrated high throughput, it's explicitly designed to undercut and disrupt models in the competitive 70Bclass market. >> Exactly. It's designed to establish Neatron as the default lowcost high performance agent platform. >> All right, let's synthesize the core findings of this deep dive. Neatron 3 is an architectural blueprint for the future of AI infrastructure. >> It really delivers a triple threat. >> It does. First, you have this unparalleled architectural efficiency achieved through the hybrid MO mamba transformer design which provides up to 3.3 times higher throughput. Second, scaling quality without scaling cost thanks to latent mo and multi-token prediction. >> MTP >> and third massive capacity and reliability with a tested graceful 1 million token context window enabled by the NOP architecture. These innovations combined to create what seems to be the most robust cost-effective platform yet for agentic AI. >> So what does this all mean for the bigger picture? I mean this release is not merely about better models. It is a strategic infrastructure played by Nvidia to define the standards of the open source world >> by opening the entire architecture. >> By opening everything, the weights, the training recipes, the highquality data and the crucial Nemo gym environment. They ensure that the best most efficient agentic AI models are perfectly optimized for their hardware. >> So these models, especially the larger super and ultra tiers trained with hardware specific NBFP4, they're essentially the ultimate advertisement for their Blackwell and Hopper silicon. >> That's it. By mastering the open source ecosystem, Nvidia guarantees that demand for the chips that run it best will skyrocket. >> The ultimate closl strategy, leveraging maximum openness. >> Indeed. So here's the final provocative thought for you to chew on. The integration of multi-token prediction means Neotron 3 models have this internal clock and planning mechanism providing the highquality draft tokens that are necessary for costcontrolled speculative decoding. >> Right? >> This ability to internally plan and to allow developers to control that specific thinking budget. It moves us closer to commercially viable, affordable, and predictable AI agents than any release we've seen before. >> I think that's true. So if the nanom moner at just 0.24 per million output tokens is already disrupting the economics of the 70 billion parameter market, what implications will the 500 billion parameter ultra model trained using NVFP4 precision and latent MOE have for the long-term economics of complex general purpose AI? What is the tipping point where superior efficiency and throughput definitively outweigh raw parameter count in the open model