Nemotron 3 Super Tutorial: Multi-Token Prediction, Latent MoE, Perplexity and OpenCode Integration

Key Takeaways

Hey, what's up everybody? My name is Chris and I'm really excited to talk to you today about our brand new Neotron 3 super model, the latest entrant in our Neotron 3 family of models. Once again, this is a hybrid Mamba transformer architecture. Uh we've added a few bells and whistles though, so it has multi-token prediction as well as latente. You can read more about those things in the technical report, but today we're going to get hands-on and see how we can actually use the model. It's a 120B total, 12B active parameter model. Uh, which means that it can still rip pretty fast. So, what we're going to do is look at how we can access it through build.vidia.com, look a little bit about the integration that we have with Perplexity, uh, look through some code in a notebook, and then showcase how you can use the model with something like Open Code as your driver for your agent harness. All right, let's dive right in, and we'll see what this model can do. All right, the first thing we can do is check out the model on uh build.unvideo.com. So you can see here we have our Neotron 3 Super 12B- A12B. Uh we can ask it questions like, "Hey Super, how's it going?" We're going to get a reasoning trace, of course, and then the final answer. We can do things like ask it a riddle. And this is basically the standard interface for using the model, right? Uh what we can get from here though is actually our API key by clicking this generate API key button. So what we're going to do is generate that key and then head to the notebook. All right. So now that we have our API key, we can work through this simple walkthrough. Uh we're going to look at how we can enable thinking, how we can leverage reasoning budget, and how we can use the brand new loweffort reasoning mode that's unique to Neotron 3 Super. We've already got our API key, so we can skip these instructions just fine. We're going to set up our base URL as integrate.api.invidia.com/b1 just like the instructions said. And we're going to have our model be exactly what we just looked at at build.envidia.com. First things first, let's look at the model in its default mode, which is reasoning on. You'll notice that I explicitly passed in enable thinking true here. But even if I didn't pass this parameter, our model would actually operate the same as we saw before. So let's go ahead and try that like this. And you can see that we get the uh default behavior which is a little bit of thinking and then the response. Now we can use a reasoning budget in order to tell the model how long to think for. What this is going to do is using enable thinking true allows us to you know set a maximum token budget. So if our model needed to think for more than 8,000 tokens, we would truncate its thinking and start returning the answer. This is great if you want a really granular control of the model's reasoning. Next, we have loweffort mode. Loweffort mode is what it sounds like. The model just tries to think as little as possible. Uh this is something that we can set just by setting loweffort equal to true. Uh the recommendation would be to start with the normal behavior of the model. If it's a little bit too ver verbose, you can move to using uh the loweffort flag and then if you need something in between at that point experiment with different potential reasoning budgets. We also have an integration with perplexity which actually allows us to use this model along with their web search tool through perplexity. So we can ask a question like what is the new architecture for neotron 3 family and get this awesome response as found by the search tool through perplexity. Not only can we use it in the notebook like this, but we also have access to it through the model dropdown in perplexity.ai. So we can actually select super as our model and ask it to look up something. So we can use it to look up something like what is the newest architecture for the Neotron 3 family of models and then send this request off. And same thing as before, it's going to think, look up, review, and then provide us with a final response. Very cool. You'll notice in the notebook that we also have this open code blob. This open code blob is actually going to let us use the model with open code. So let's look at how we would do that. All right. So we can head to our terminal. And what we're going to do is go ahead and open up our open.json. I have a demo version just to hide my API key. But the idea is that we can set our schema model and our provider. For our provider, we're going to call it NVIDIA NIM. We're going to include our base URL, which is the same URL that we saw in the notebook, as well as our model, which is the same model we saw at build.vidia.com. Once we have this set up, we're good to go ahead and actually run open code, which we can do with the command open code. And then we're going to find our model in the models section. We can search it up with super and see that we have our old versions of super as well as the latest Neatron 3 Super. Now that we've picked our model, we can go ahead and ask it to do stuff for us, like perhaps, you know, build a cool snake game in HTML CSS. It's going to start cooking. And that's it. And finally, we look at some of the things that our open code agent was able to build, like this sweet landing page with animations, cool things when we hover a slide. Uh, so this is for a slide deck. And of course, you know, we got to add the snake game. So, we have this snake game that we can play, which keeps track of our high score, as well as, of course, shows us a fun little animation whenever we hit a fruit. Thanks so much for watching this tutorial video. I hope you enjoyed it. We have some resources prepared that you can check out down below, including a tech blog, a tech report, as well as a number of examples on how you can actually customize this model if you wanted to do that. Thanks so much for watching and have a great