Beyond the Algorithm with NVIDIA: Generating Reasoning Enhanced Podcasts with Open Source AI Agents

Key Takeaways

are we live I think we should be we are welcome everybody back for another developer live stream we've uh We've altered our name a little bit called ourselves the beyond the algorithm with Nvidia this this month we're talking about generating reasoning enhanced podcasts with open source AI agents and I'm joined by one familiar face and one that's maybe not so but familiar we're joined by Neil who is the uh a tech marketing engineer he'll give an intro to Nim including an overview of what Nim is and trying to get them in the Nvidia uh aipi catalog and we also have Chris joining us who is a deep learning developer Advocate at Nvidia Chris will be walking us through how to build a PDF to podcast agent using reasoning models um but before before we get to all that some of you might or may not know that we have a little conference coming up next month uh called GTC GPU technology conference um we have our mods in the background hopefully can share a link to a discount code but um the registration code is up uh on this on this slide it's from March 17th through the 21st happening in in San Jose California so if you can stop by we be great to see uh people showing up for the event and participating you know Jensen's keynote is always a highlight so it would be great to uh to have as many many folks there participating as possible all right so I will just kick it off uh for Neil to go ahead and speak his peace all right thanks Zach really appreciate it yeah so um we're going to talk a little bit about Nvidia Nim today and some of the things that you can do with Nims or to you know create really cool applications use them in sort of agentic workflows and and really take advantage of you know all of the um all the really incredible stuff that you can do with with large language models and with sort of generative AI more broadly um so uh last time uh I guess two two live streams ago I gave a little bit of an introduction to um what Nim is in general um and I'm going to do a little Refresh on that just in case anybody wasn't here last time um but if you're more Curious and you want to get more details we do have the recording of that previous live stream available on our YouTube channel um and we also have a couple of other videos about Nims um sort of introducing the product and sort of uh you know understanding how you can take them and deploy them and things like that um but yeah for those you guys who aren't familiar uh Nim is nvidia's product for basically taking um both large language models and other kinds of AI models and kind of packaging them into um a container that has all of the dependencies that you need and has all of sort of nvidia's goodness for accelerating these models and making them run really really quickly and really efficiently and really cheaply um and what you can do uh with those Mims is basically take them download them from our website um deploy them on you know any gpus that you have locally or in the cloud or you know um and some sort of data Center um really all sorts of different kinds of gpus um and you don't have to worry about you know dependencies and um you know downloading and installing a bunch of different um uh you know libraries and things like that so it's it's a it's super great um and a really convenient way to be able to deploy uh to be able to deploy AI models um and so if you're interested in trying out uh Nvidia Nim uh you can go to uh the Nvidia API catalog and here we basically have those Nims that we're talking about um you know hosted ourselves so that you can check them out play around with the models sort of see what it's like to interact with them and what of the what the um you know the the output of those models are um and I I showed a couple of of examples last time um and so uh one of the things that I wanted to talk about today is that we have a new model available um uh it's been around for for a little while but it's it's new since the last live stream which is deep seek R1 um this is a really cool model if you guys haven't heard about it before um it incorporates uh thinking into its responses and so basically what it does is it kind of before giving you your final response generates this really long chain of uh you know chain of text that kind of you know reasons through and tries to sort of um you know sort of think about what the what the final response should be um and these uh models like deep SE 1 and sort of other uh reasoning models are are really interesting for a couple of reasons they um they're they're much more effective um at sort of uh you know at doing tasks that require you know that require thinking that require planning ahead that require you know taking multiple steps um which is really which is really key for a lot of agentic use cases use cases where you actually want your large language model to be able to go out into the world and do something rather than just um you know answering questions or uh giving you sort of um you know uh really quick responses or summarizing things and things like that um but the other thing uh to think about with uh these these reasoning models is that they are pretty slow um they output a lot of text um before giving you your final response and so you have to wait for all of that text to be generated before you actually get your final response so I'm going to just try this out here on our web website um and I'm going to ask um how about I'm planning to go to GTC next month what are some of the ways I can take advantage of the conference and we can see it takes a little while it's starting to generate the output um and once it sort of processes things and kind of gets scheduled we'll be able to see uh it outputting these thinking tokens right so before giving me an answer it's going through all of this process of you know thinking about what I said and reasoning about its uh you know it's its background information and things like that um and and basically the the implication of this is that it becomes much more important to have an accelerated runtime to be able to have something that can run your large language models much more quickly than a sort of you know a standard runtime running them in pie torch or you know in in eager mode or things like that and and and so that's one of the things that um Nvidia Nim can really help with because we have all of that you know Nvidia goodness all those accelerated libraries and all of the dependencies sort of figured out and bundled into these containers it makes it able to um you know to respond much more quickly than what you would see um you know without without Nim and also to be able to handle much uh much larger amount of concurrent users um than than what you would see with with other programs so oh and here's here's our response um so there you go if you're interested in going to GTC you can uh check out um uh pre-conference preparation and Keynotes and sessions and networking and make sure you wear comfortable shoes all really great advice for GTC um so especially I think wearing comfortable shoes is is the key part yeah yeah definitely bring your bring your notebook and water um yeah but if you're going to GTC it's a really fun time and I do hope that um that some of the people here um watching this are are able to attend if you're not able to attend all of the um or not all but a lot of the sessions are recorded and you'll be able to watch them um afterwards uh and like Zach said uh we do have a lot of nim related GTC uh GTC sessions so um definitely recommend checking those out there's also virtual registration too so for people who can't show up in person um you know you can't register and attend virtually if it's a little too far for you to come and also uh to add to that the U you know us having this live stream is really for uh the audience you folks attending so we encourage you to ask questions you know we can we might bring one or two of them up and and have Neil or Chris answer them um in this Stream So or maybe three or four so please you know ask away um you know also things like you know if if you have experience using our API catalog in general or perhaps what you know some other Nvidia technology that's your favorite you know leave us a comment what it is yeah definitely would would love to hear from you guys please please ask questions we'd love to to answer questions we eat questions for breakfast yeah absolutely um okay one other thing that I wanted to show on this API catalog which you may not have seen before is this blueprints Tab and what These Blueprints are is basically you know if you just have a single um model it can be you know it can be fun to play around with and you can you know sort of ask it questions and things like that but where where things really start to where these models really start to shine is when you start incorporating them into little more advanced workflows you know things that um are are built for you know a certain uh specific um specific output and and that's basically what These Blueprints are we've sort of taken all of our Nims and built out these workflows and you know given uh examples of what you can do with um with Nims for various different uh you know for various different things so for example one of our featured blueprints is this PDF to podcast blueprint which Chris is going to be telling you all about um in in a little bit but there's also plenty of other ones so we have this PDF data extraction you know container security building a virtual assistant um building a digital human this one's super cool um and so definitely also encourage you to check out all of these um on uh on build. nvidia.com um if we click into uh which one should I click on let's let's do the multimodal PDF extraction we have um you know this uh information here about which Nims basically Which models are included in this uh blueprint um you're able to see the source code on GitHub which basically explains how you can you know deploy it in your own uh environment we also have this cool deploy launchable button which will basically spin up a environment for you in the cloud that you can use to um to to deploy all of the the components of the blueprint and you know just really quickly get started using the um using the NIMS um if you saw one of our previous live streams we we we actually went into you know launchable and the launchable platform um and sort of how you can how you can use that so um another really cool uh feature of um of our API catalog and something that I really really hope that you um that you take advantage of okay um I think that covers a lot of uh what I wanted to talk about sort of introductory that's a word um it can yeah uh let's see should we should we answer a couple questions I've seen a few coming in the chat um okay here PDF data extraction that assumes the PDF document has set form Fields with or without text it's it's actually a little bit more um more robust than that so basically it's able to handle all sorts of different PDF layouts um so whether it's got form fields or whether it's you know got like diagrams or just free text or you know all all sorts of different things and and that's why it's got all these different uh models associated with it because basically all these different models are handling different kinds of PDFs that have different kinds of data um and so it it sort of takes those PDFs and converts them into into free text which is a great format for doing like future like further processing or using it in a retrieval augmented generation system or you know incorporating them into llms and you for training and things like that so it's B it's basically able to handle all sorts of different kinds of of of PDFs it's really robust to to all sorts of different formats which I think is super cool um and and very useful so yeah okay is it in the cloud do we have an opportunity to use it with a smartphone uh so I would say not directly but the the launchable does allow you to spin up an instance in the cloud which you can then interact with through through potentially a smartphone uh it's not like optimized there's no like uh beautiful you know responsive UI that you're you're going to be able to leverage but but certainly you can host it in the cloud and then you can access it as you would any other Cloud resource yeah and and one of the great things about these These Blueprints I mean we um we don't necessarily intend for These Blueprints to be like a final product that um you know you sort of uh use in your day-to-day life that's that's why we we open source them and share the code and sort of give you all sorts of examples for how to deploy it in your own environment the idea is um for you to sort of uh to look at the components that are part of these that are part of These Blueprints and kind of build off of them and and you know customize it for your own um for your own applications and sort of see see what's possible with um with large language models and generative AI um so if you wanted to build something for a smartphone um I think it would be really great to you know sort of look at what's going on in the blueprint you know open up the the source code and see um all the you know the documentations and the and the layout and everything um and and and see how you can convert that into you know models that you can use on your smartphone or at least like a UI that you can use on your smartphone so um yeah it's a it's a great question but um this this particular example is is in the cloud and and not on a smartphone okay awesome okay another question do you have distilled versions of deep seek to be used in a desktop or laptop with an Nvidia GPU we totally do what a great great question um let me see if they are up here on the API catalog it looks like the API catalog itself doesn't um include uh the distilled versions but we do have um we do have uh Nims uh for some of the distilled um models that that the deeps um put out the the Llama models and I think maybe one of the quen models too um you can find those on Ng see doing some live searching I think um we have one out right now and there's going to be more coming in the future um if I'm not mistaken so let me see yeah here we go so we have the Deep seek R1 distill llama 8B which is an 8 billion parameter model let me zoom in a little um which is obviously much smaller than the full deep seek which is like 600 billion parameters um and definitely will not fit on any toop um whereas this 8B model you know can fit on laptops that have a decent amount of of GPU memory so yeah you can you can take this um and download it and run it on your laptop you you know it's great yeah for for deep seek you you need a lot more than laptop GPU just a bit just AIT just a bit yeah um yeah okay I think uh I think we're good so Chris do you wanna do you want to take over and start talking about that podcast oh yeah yes okay so we're uh we heard from Neil about uh kind of what what is going to get us to power the thing uh to to act as a Reasoner to produce you know our final product uh now we're going to go ahead and look at what we're going to be building today uh so if you want to throw my screen up there Zach uh we can uh we can rock into this so uh we're going to be looking at the PDF to podcast uh blueprints so this is a blueprint just like Neil described the idea of the blueprint is to give you a blueprint right to build something on top of uh so it does work out of the box does cool stuff but it's meant to be tinkered with it's meant to be uh you know adapted to your use case uh very very exciting stuff okay uh so once again we have kind of this Blueprint Card that Neils showed us it goes over all the licensing the mod everything you need it's got a sweet diagram it's also got the view source code button that's we're going to be looking at today and of course you can deploy it as a launchable so let's look at the diagram for the blueprint we're going to be using today so this is the uh diagram for the PDF to podcast application uh the basic idea here is that we're going to ingest some pod or some PDF as well as some optional PDF context uh we're going to use docking to parse that PDF into a more suitable or consumable format and then we're going to use this kind of uh you know flow in order to construct a transcript for a podcast so what we're going to do is uh create an outline so just zoom in a little bit there so we can see it better so we're going to create an outline we're going to structure it we're going to segment it into different transcript sections do some deep Dives uh use some transcript optimization podcast dialogue this is like the back and forth classic stuff then we're going to start fusing these things together so they make sense in between chunks do some revising and then once again go back to our evaluation uh of the total transcript and potentially do some iteration here you'll notice that we see llama 3.1 405b in a number of these uh Nim boxes that are powering some of these steps uh these models are totally flexible so you can use whichever models uh you you uh you wish today we're going to be replacing that 405b with the build. nvidia.com endpoint that Neil showed us which is a deep seek R1 endpoint okay so uh once we have a finished uh podcast all we need to do is send this to 11 Labs 11 Labs is going to generate the actual audio for us and then we can listen to our newly constructed podcasts uh again this is using dock Ling on the back the back end to parse those PDFs into a more consumable format uh for the llm this is a integral step right uh but again that this is a blueprint you can change this with whatever uh pipeline you have for a PDF extraction or file extraction or what have you uh again these are designed to be to be played with and modified uh so let's look at the actual repo with the source code so you see it's got uh everything that we'll hope for in a repo it's got uh you know all of the resources we need exactly what we're going to use out of the box uh what services we're going to use what kind of Hardware we need uh you know if we're if we're running this locally then we need to support all these models in our environment as well uh but we don't have to run them locally we can point to build. nvidia.com for that uh and then it shows us a quick start guide so we've uh we've done some of this ahead of time just to uh to be able to save you guys a little bit of time watching loading bars the idea is that we have to have Docker once we have Docker we have to get some API Keys uh once we have our API keys we're good to just clone this repo set those API Keys up and then we're going to uh install our dependencies using UV and then we're going to move to the next section together with make all services uh excellent uh question in the chat is is it open source yeah you you can use it you can contribute to it uh we'd love you to uh to to leverage this as best you can uh so what is actually powering this thing right so we see that so yes uh it is a pile of code right but what is the pile of code uh the this is again totally visible to you right so it's all released you can play with it you can do whatever you'd like uh but you'll see that we have a number of different services that we're going to cons uh consume uh so we've got our API service our agent service our pdf service and our Texas speak service the agent service is probably the most interesting because it's uh it's the that gets us to the podcast right so that agent service is everything that's happening in this box here right that's going to actually generate our our podcast so what we can do is look into exactly what's happening in this agent service like let's look for instance at the uh you know podcast prompts to see what kind of prompts are are leading to this uh to this final output that we're we're going to listen to together uh and you'll see that it's just they're V totally visible prompts right you can play with these you can change them you can make them whatever you want uh every prompt that is used is going to be provided here and you'll see that they you know they're quite they're quite verbose they're well set up uh but the idea is that these are completely flexible right so the podcast that you hear today uh is going to be generated using these defaults but uh feel free to to change the props change the characters how they should interact uh you know change the amount of time you want to spend listening to the podcast Etc all totally under your control through these uh through these Services uh we'll also look at the actual like thing that makes it go right so you'll notice that this is basically just using uh a uh a simple kind of laying graph application so what we're going to do is we're going to have our different uh you know nodes that do different jobs and then we're going to build a lang graph application that connects all of these nodes together so the the uh the prompts are are open right and consumable for you everything is open and consumable for you you're able to uh change any of this right that's happening under the hood you're able to change the way that it's stitched together uh you could do that all uh very very fun the podcast that we're going to generate today just to be a little bit meta and have some fun is going to be a podcast that talks about the Deep seek R1 paper so we're going to use that as our source PDF uh right uh this is just to to use the Deep seek model to make a podcast about deep seek it's just fun uh okay so what I'm gonna do now is I'm going to switch to my vs code screen for you guys and we're going to go ahead and we're gonna do that here and then I'm GNA ask Zach to share that for us to see perfect thank you Zach and uh you'll notice that we're already in the uh PDF to podcast repo so I've already cloned it I've already got docker installed and ready to rock I've already got things set up on that end uh what we're going to do now is just take a little you know again peruse to the repo you'll see we've got a number of different uh folders number of different uh things that we can do uh the important one for the test case is the samples so this samples uh uh you know directory is going to contain the PDFs that we care about that we wish to use as context for our PDF to podcast application so I've uploaded the or not uploaded but I've put the Deep seek R1 PDF in this samples repo you'll also notice that we have this uh this tests so tests is where we're going to kind of save all of our uh save all of our podcasts that we generate you'll also notice that we're going to use this test.py so this test.py is the thing that's actually going to give us our uh podcast so how we're going to generate the podcast uh again you can use uh you know you can use this service however you'd like but this is definitely the the first thing you should try right uh you'll notice that we assign random speaker names so you can change these speaker names uh as you'll see in the in the example we use Bob and Alice because it's it's classic Alice and Bob right all this is doing is it's consuming the actual services that are going to be built when we do the make all files that we're about to uh to do okay so uh we've got our test kind of uh you know py that we're going to use we've got our uh Services everything kind of finished to this step what we're going to do next is we're going to use make all services this is going to actually get things up and running so I'm just going to clear the screen and then we'll do a make all services now the first time that you run this it's going to take a little bit longer this is because uh we have to download the docking model right uh which is uh which takes some number of of gigabytes uh but you know you can see pretty straightforwardly that once you have that done the service is up and running ready to be consumed extremely quickly uh all what this is doing behind the scenes right is it's loading all of these different services in concert through Docker compose so that we can consume them all uh in a way that uh results in us having a podcast so the next thing that we're going to do is uh and I just I I'll talk about this uh briefly we have kind of like two things that we can play with right so we have our end uh file which is where we put our API Keys remember because you can customize this you can put kind of whatever things that you want in these uh through these apis right so if you want to use like open AI or or anthropic or whatever have you together it doesn't matter you know uh you can you can put these in and then of course for 11 Labs we're going to provide our our key as well and then we we have our models. Json now this is like the most important part quote unquote because this is the part where we choose uh Which models we want uh so you'll see here that we've got our deep seek R1 uh model which is going to be uh through this uh through this API that you can find from build. nvidia.com this is this is like the only change you need to make is this and then it's using deep SE right so if that's all you did then you're generating a podcast deep seek you don't have to do any other steps the rest is just like uh what we've already shown right uh but you can also change our Json file or model uh it might it might make sense to you but the idea is that this should be a model that supports Json mode right so you can use any model you want so long as it supports Json mode and then we have our iteration model this model should also support tool calling uh in order to to make the flow work this reasoning model though it can be whatever you want choose your favorite reasoning model and uh and pop it in okay before before you before you move on from this just the the nvy dev thing that's a that's an internal thing for for us at Nvidia if you are following this blueprint just leave off that Envy Dev kind of prefix for the for the model name so it should just be yeah like meta llama 3.1 um but everything else you should be able to do exactly the same from wherever you are yes and and great great Point Neil you'll just take these right from uh build. nvidia.com so exactly as you see those addresses on build. nvidia.com is what you should put in this uh in this models. Json excellent point yes okay and if you have if you have your own Nims deployed um locally for example like if you're deploying them in a data center or if you have them on your own um you know your your your your local machine like on a you know laptop or something this would be where you would modify that as well so that you can point them to uh yeah like your local machine Yeah so basically uh like uh depending on where you've hosted obviously uh but you just need to change your API base so where the uh where the endpoint is are hosted and then you need to make sure that that model is available on that endpoint uh so that you've deployed it just as just as Neil said uh the idea being that this is this models. Json is really all you need to touch to CH change any of those things right so as Neil mentioned if you've hosted NS uh if you've got uh if you want to use different models from build. nvidia.com all we're doing is we're editing this model . Json we don't have to do anything else right there's no additional steps for us uh in order to make that work uh which is which is fantastic so you can get you can get experimenting and and and playing around very quickly uh okay so we're going to use deep seek uh AIS deep seek R1 to create the uh the the initial kind of transcript to pars through the document that's going to be fun uh and then we're going to use our just our classic you know llama llama models to do the rest of the work Workhorse model so they're going to do the work it's going to be great okay uh so the next thing we need to do after we have made all services is we're going to go ahead and we're going to actually generate the podcast so what we have to do is head to our uh uh our our terminal here and what we're going to do is we're going to use the uh First Command to activate our UV generated VM make sure that that's active and then we can go ahead and we can use our python you know tests test.py our Target is going to be the Deep seek R1 PDF there's a bunch of additional flags that you can add here so we can use the monologue flag we can use the context flag if we look in the actual uh test script there there's there's a bunch more options that we have access to where we can actually change uh you know the the number of uh people in the podcast uh it's all all available in this kind of uh you know test.py file so you can choose the names right from the command line you can choose the duration the the target duration of the uh podcast it's all available to you in the test.py file uh and uh and can be leveraged to to great effect so I'm curious about one thing have you have you tried out um with like multiple different speakers or with one speaker like what is what do worked the best best uh in your opinion for for those yeah uh I like so so works the best is is an interesting question so they they all work suitably well right so depend especially depending on the model composition so like R1 plus the the 8B and 70b is a great combo it generates coherent podcasts every time I'm not a big fan of the monologue because I think the the back and forth like the the but that's a subjective thing right I mean the quality as in does the does the generated podcast stick to the context well does it cover uh you know the the key p uh parts of the PDF for example like that that's not really changing a lot or at least in the in the testing that I've done it doesn't really change a lot between you know monologue mode multiple speakers uh all these other configurations like it it's all good the just for for like listening fun though I enjoy having the back the back and forth uh especially if you choose like 11 Labs voices that you like so you can actually set the The Voice IDs right so if you have like favorite 11 Labs voices you can choose those directly as well uh through through that test.py uh or or the command line and that is extremely fun uh but yeah I like EXA again they're all high quality according to the testing right they they stick they stick to the script they do a great job they don't hallucinate a lot uh but uh I just like the back and forth I love it yeah got it yeah I feel like I feel like if you're really interesting to have like like 10 speakers and just like a huge cacophony of people talking to each other I I don't know might be fun it I I do believe it would be fun it might lose my mind a little bit but uh but I do believe it would be fun uh so this is ianu only uh so you are required to have iunu 20.04 22.4 with sedo uh you you can run this on Windows as evidenced by the fact that I'm currently on Windows through wsl2 uh so it it is possible to run this uh without problem on wsl2 uh which means it's open to Windows I have not tested it on a Mac but the system requirements do say that it's for Ubuntu uh so I would I would stick to those to to try it first uh but again wsl2 is is doing a good job as we can see by it uh by motor in here okay uh finally we will uh click the button that makes it go so we submit the job we have our test test.py our Target's going to be deep seek r1. PDF and you're going to see it's going to start chugging you're going to see that we have this uh uh you know this uh kind of log that we can view through the the uh the docker container so it's going to tell us what's happening when uh it it doesn't work the same as open a chat gbt which is a foundation model any particular ml is trained on so this is leveraging a number of models so it's not a specific model uh so if we go back to our models. Json basically this is leveraging these three models so what that means is that it's it's not using like a specific trained model for this task it's just using a a combination of models that can handle this task uh which is which is what we what we would hope for uh we also have a question can you use open source voice model clones instead of 11 Labs how much oh yeah if you want to for sure right so this is the beauty of the of the blueprint right this is like the this is the thing so you see in our services we have our agent service we have our API we also have our TTS service so as long as you're able to adapt to the uh inputs expected inputs and outputs that are present for the 11 lab service you can just straight shove whatever you want like in the place of that 11 lab service so long as the API is the same right it's going to work relatively out of the box if the API is different it's going to be a bigger lift on your end right you're GNA have to do more coding basically uh but the idea is that this is uh totally adaptable to whatever you're using uh all this code is editable for you again you know if the API is line up it'll be easier but it's it's certainly possible to use any uh voice service you want any voice model that you want so long as you can adapt it to the the expected inputs and outputs from uh from this blueprint yes you can you can change all of it if you want uh you you can change the agent pipeline right you can change the way we ingest documents you can change anything uh totally totally up to you and and in fact please do right we we encourage you to we we providing this as a base for you to make cool stuff with uh right uh not not like uh isn't prescriptive it's not like this is the way you should do this it's like here's a way you can do this please please Tinker with it play with it and uh and make it suit your your needs the best okay so this is we're going to do a cooking show movie Magic right here so uh the the the service takes a little while to complete because the end points that we're leveraging today are uh are not the fastest in the world uh so we're gonna we're gon to uh do some Again movie Magic so what I'm going to do is I'm going to stop sharing my screen that was it by the way just to be clear this is the this is once we once we've done make all services and it's up and running we just have to submit this pointing at the PDF we want to podcast up that's it uh that's the demo good job okay so now we'll fast forward using movie Magic to a completed podcast and uh I'll g go ahead and share my screen again uh Zach if you wouldn't mind loading this up we're we're gonna listen to like 10 15 seconds of this podcast not the whole thing but just to hear kind of what the final results are are going to be so this is uh generated with deep sea car 1 with uh our Json model being llama 8B and then our uh our uh uh revision model or edit model being the 70b okay so we're listen to it Welcome to our show today I'm your host Bob and I'm excited to explore the cuttingedge deeps car1 model and its Advanced reasoning capabilities I'm joined by Alice an expert in the field welcome Alice thanks Bob I'm excited to share my insights on this powerful model so let's dive right in there you go it makes a podcast right that's that's the thing it says it would do and it did it uh the length is is variable so you're typically going to get between like a like a four and a half Minute Podcast like a Nish Minute Podcast there is some tinkering you can do in the test.py file along with the services files to try to force the uh the uh the length to be more specific i' I found that it's you you got to give yourself like and half minutes leeway on either side right it's going to get close but it's certainly not going to not going to get like exactly five minutes uh but they do the whole podcast thing including like a sign off right so we'll just listen to the very end just to hear it Nvidia AI Foundry and Nemo software to learn more about customizing all the time we have for today thanks for tuning in everyone if you want to learn more about t in right there you go it's a real podcast uh yeah and that's that's the thing that's what it does PDF the podcast it makes podcast for PDF there you go all right fantastic thanks Chris that was uh that was really exciting does it work with multiple podcasts or can only use one at a time uh so you so you can or not multiple podcast sorry multiple PDFs multiple PDFs yeah totally totally understood you uh so you can add additional PDFs as context for the initial PDF so basically like uh there is going to be a main PDF but you can supplement that with additional context uh that uh like say for instance you wanted to to do deep seek plus news around deep seek Etc you can provide that in PDF form yeah absolutely um and uh yeah can they beyond the scope delve in a different topics or least yes so yes to the second part they they they definitely try to be funny from time to time uh how successful are they at trying to be funny I would say like as successful as normal podcaster yeah okay pretty good uh but uh they don't usually go off script like uh they they might do like a small tangent uh especially in the close they tend to do like a tangent about the actual podcast that they're they're they're they're hosting but uh for the most part it stays extremely on script which is which is desirable right like we want them to stay on script you can introduce noise in your context uh and uh that that will cause them to you know go more Off Script so if I do deep seek as my PDF and then I add like a PDF about nachos or something like they'll they will talk about nachos right getting hungry yes okay you can string all the PDFs together upload them with one document sure but uh remember that the model has to consume all of this so there like there's an upper limit right to how long your your PDFs can be uh and also remember that the you know a lot of that context is going to be uh it's going to slow the process down let's say right uh and uh and most importantly uh you're you're going to lose some information so it's always going to be compressing information it's always going to be you know going from PDF to a like five minute podcast so if you have like 70 documents in a chain uh and you and you haven't exceeded the context limit you're you're likely to lose a bunch of information and context just because the podcast is only like five minutes or whatever yeah great can you can you do something with computer vision uh not like immediately um but there's uh examples on uh on the API catalog with a couple different models that that use um that use computer vision most of them I think are like multimodal llms I don't I don't know if we have like like a dedicated like resinet or something up on there umbe can I can check that out but yeah there's like a bunch of there's a bunch of models that can like analyze images and like tell you what's going on in the image and you know sort of um do like visual question answering and things like that even the um the PDF ingestion um model that we talked about uses computer vision to understand like diagrams and um you know like the the layout of the page and things like that so um yeah definitely do things with computer vision um it's great and there's a blue blint as well uh which is like for video search right so there's blueprints like Neil already mentioned one but there there's a bunch that that cover a wide variety of of modalities can it talk in different languages so not out of the box uh so it's English out of the box straight up uh it points at English voices it uses English transcripts but again uh if you substitute in models or you change the prop or you you know you take it with the system you can certainly uh you can you can make it talk in whatever language is supported by text to audio uh right so text to speech models uh and as long as your core right your core uh implementation uh is is adapted to make them produce transcripts in in whatever language you desire yeah you can make it you can make whatever language you want out of the box though it is English yes ah emotion Performance training with yes okay so uh so again a lot of these questions are going to be answered the same way which is that you can inject this information in your in your if in your flow if you want uh a lot of services like 11 Labs actually uh uh they they provide a an Avenue for you to like choose the emotion right uh so you'll notice that like the podcast voices we listen to they s they sound excited at times or maybe maybe like uh what's going on incredulous you know but uh this is something that you'd have to you'd have to change by default we're going to generate those as part of generating the transcripts and then 11 Labs is going to handle that but you can inject the motion as you desire uh based on the API you use 11 lives is one of the apis that allows you to do that yes you can make a longer than five minutes podcast I would I would though I would though remind you that you're paying 11 lab credits for the default experience and going much longer you're going to uh you're going to run into credit issues so uh but you can you can make them as long as you as you feel are necessary up to like you know 10 minutes and that might be that might be a great use case for you know using your own open source text to speech models right um which uh I think we have some available on uh the the to build as Nims on on the platform as Nims but yeah the 11s are also really really good so yes uh can we also train your model on custom data sets uh I'm the answer broadly to this question is yes uh in terms of this there's no specific model training or data set that you're that you're going to get out of the box but you can certainly use like a fine tune model uh to to power the if you've if you've got that set up yeah great questions great questions more questions oh like new Learners of python or like new Learners of like uh like deep learning because I feel like those are very different questions um I'm guessing it's probably the the new Learners of python but it could be wrong um I don't know that's a that's a that's a great question um I'm not sure that I have any advice on this one Chris what about you what's your favorite if you're gonna start with python I I know it's the book's got to be like 40 years old or something like that I don't that's really true but then automate the boring stuff is always a fun way to start with python uh it kind of gets you in the mindset of what programming is is useful for it keeps you uh on track if you're going more from the deep learning side then like Sebastian rajka has a book called building LMS from scratch which includes some python that I think is uh is useful and fun yes that is a great one and yeah he just responded he meant deep learning or they meant deep learning so yeah uh yeah I think Sebastian rashka book is really good um if you want like more like Theory stuff um elements of statistical learning is a great book for like learning a little bit more about the the underlying principles of deep learning um so yeah two great options there I think I think elements of statistical learning used to be it used to use R but they put out like a new version recently um that uses python I I haven't looked at the the Pyon version I only use the R1 but I've heard it's good so check it out can you combine the PDF extraction model and the podcast model together so you extract data from the PDF then create a podcast so so that is what's so I think if what you're asking is can we go straight from extracted PDF to podcast then uh this system does not do that you could you could probably build a model that would do that uh my might take you some time but uh in this case we we we do rely on these models being Specialists basically having a model that's good at creating these long form plans and then models that are good at executing on those those plans so that that The Ensemble here really does add value yeah yeah great questions oh and there's more oh yeah uh so so it dumps everything into the logs in the in the containers so you can get a bunch of raw you can get the raw transcripts you can get the raw voice uh you can edit it however you want uh yes again this is going to require you to to uh move Beyond like test.py right uh but you can you can certainly edit the transcripts uh there's a full when you watch that when you do make all services uh there's a full API like a docs page classic like uh uh Swagger docs page that will give you the different endpoints you can that you can use and consume but yes you can you can hack into any part of the flow all right great question s oh more questions okay how long does it take to generate the podcast and could you optimize how long it takes to generate so it takes so dep it's all depending on your end point right so the fast your end point is the easier it is to generate podcasts right reasoning models are going to take longer because they love to do this thing where they uh spin out tons of tokens right but uh optimizing the podcast is by doing things like using dims right like uh Nim is a is an excellent tool it produces a lot of uh tokens very quickly uh but basically you're looking at model selection and uh inference Solutions being the levers you can play with to get podcasts faster uh for the testing we did it took anywhere between uh for me it took anywhere between 5 minutes to 14 minutes to generate a podcast great well we have actually a question from one of the other platforms um can't really bring it up on screen on this end but it's it's pretty straightforward it's it's asking or they're asking can I deploy Nim in public cloud like Google Cloud AWS yeah absolutely you can you can deploy it anywhere um well anywhere that there's a GPU which is I think one of the really great things about Nim um so you know you canpl it in Google Cloud you can deploy it AWS you can deploy it locally on your own machine you can deploy it um with uh our our launchables that um we have some examples of on the uh on the API catalog um there's all sorts of places that you can deploy Nims um and so yeah just uh go go forth you know go forth and deploy yeah exactly how much what's the cost um for for the Nim itself uh you can download it for free through the developer program so you just have to join the Nvidia developer program and you'll get like free access to download the NIMS in terms of like the like the hardware cost it depends very um like a lot on on which model you're going to use for the smaller models you can deploy them on small gpus and so it's not very expensive for like the really big models like the deeps R1 you need a lot of gpus and so it can be a little bit pricey um but uh yeah just you know you can find the models that that work the best for your budget and for your use case great um we actually have a our developer Discord that um you know people here can continue the conversation after the stream and going forth in the future if appreciate if our mods can throw that into the comments or if you have already thanks um should direct people and uh you know you can keep chatting away uh at any time of the day or night all righty well great well so we put up this last slide again um magically threw in a GTC discount promo code that we just picked out of the air um just for this audience so if you want to register for GTC come join us it's happening in two or three weeks it's it's coming up pretty quick um I know hotel rooms are are pretty hard to come by but uh I'm sure you could find find something to uh to shack up somewhere um but we'd really appreciate seeing you there you know come find us uh any sessions that might interest you from you know any one of our various technology verticals uh join us virtually if you can't make it um in in person and you know we look forward to seeing you on a future developer live stream yes and we will we will also be at GTC we have a connect with the expert session at GTC you can come by and ask us your questions in person um so yeah please uh please come through we we would love to see you guys in person yep yeah thanks Chris thanks Neil for joining us uh on the stream hopefully we'll get you back I'm sure we will thank you everyone here for joining us and we'll see you on the next one all right thanks Zach all right thanks guys thanks everyone for joining for