Llama 4 - Christian Keller, Meta

Key Takeaways

Thanks for the intro. Hi everyone. Um, all right. You think somebody working in tech might be prepared, right? Yeah. Product manager. There you have it. So, um, my name is Christian. I've been working at Beta for the last um, six years. I was based in the US previously, but I'm in Paris now. And I was four years as the product manager on PyTorch. So, you know, this is just full circle for me. It's quite great to see uh, see the community growing. Um and with PyTorch I worked on uh on device and privacy and then took over PyTorch and led the 2.0 launch. After that I moved to uh Paris to work on Llama um in Jagger AI research at Meta and today I'm a product manager at which is our foundational AI research lab working on world models what we think is going to come next. So but today it's about lava. So I won't tell anyone anything new here that LLMs are taking over the world. Everybody's talking about it. Meta itself is integrating in a lot of its products from you know creative tools to ads whether it's for image generation or um or text and engagement. And um AI isn't just about generative uh models. It's also about um initially uh you know classification of images um recommended system. It was used in many many different ways. And some of my friends are like oh I'm starting to use AI. I'm like oh this is great. Have you not been using Google for the last 20 years? I don't know. But um so you have it now gener. What's coming next beyond these LMs and what we're going to try to do with Llama here is uh get these models to think better and go towards artificial general intelligence. Now Llama 4 is not that but it's a step towards it. Let's hope so. So the evolution of llama, one thing a lot of people don't know and you're here in Paris is that llama one and llama 2 were built here in Paris like at the offices near opera uh by a set of researchers that were here working at fair and um I work personally lama 3 and so the launch of lama 3 was our our way to catch up to say look there's these great models out there GPT there's Geminis uh we want to play again, we want to be serious about it. And so when we launched Llama 3 was our way to do that with Llama 3.1 in particular. That was a 45 billion parameter model. So pretty big, hard to use um every day in most cases. But late last year, we launched Lama 3.3. And with Llama 3.3, we're able to compress pretty much a lot of the quality of the Llama 3.1 model into a semi-billion parameter model that's used and deployed in a lot of our products at Meta. And so one thing about innovation I think in this space is that the way I frame it is that it comes in two different flavors. One is around improving the capabilities a given model has to be able to do more things, unlock new types of applications. But the other one is around efficiency and around making uh you know more bang for buck in the models that you have so that they've got lower cost and then new use cases can also be deployed because they're suddenly um they make sense from an ROI perspective. And in April we just launched Llama 4 at least the first two models of our part the Lama for Scout and Maverick. And I'll tell you a little bit more about those but first at a high level thank you for the hugging face people here. that I see um llama has been downloaded over a billion time on a hiking face. Like when I think of it personally, I don't quite grasp that because there's maybe I don't know 50 70,000 researchers or engineers working in LLMs like in the world, but they've downloaded a billion times. Like did they just fail on these downloads? No. But because each time we launched a new model, they um we're able to kind of test new things. There's different versions of the model that they try out. Um and it just shows that I think um that people are really engaged I think in this community. There's a community around Llama that's being built and that's shown with the derivative models that are built on top of it. So there's over 200,000 models that have been built on top of Llama that people have been able to take and customize and and make their own for their own applications. That's one of the powers I think of of open source. So um Llama isn't just a model. It's not just also the community. It's it's a group of things that involve um the stack that's how you're going to deploy it. I'll tell you more about this. Uh an ability to run it in many different places whether it's on your premises or cloud service providers and customize it. And then uh you have a set of tools around it to make uh the model safer and I'll have a whole section on responsible AI and how to build it with long. So I think everybody knows what you can do with LLMs here but very quickly uh it's used for translation, it's used for content creation. Um it's used for education and my thesis on this that I don't think we yet have the hero use cases that are going to be deployed that are going to make these things uh big. They're already big. Like I I use it to uh you know edit documents. Uh my spouse is learning French right now. I wrote a little program for her to correct her essays. um and you know with some pre-prompting on it. So it would be rated like the del B1 level that you're trying to take. Um and so you can do a lot of these things yourself. These models are very powerful. They're very um flexible in what you can do with them. Llama can be deployed anywhere. So whether it's on your, you know, premises or on the cloud, that allows you also for the second point to have more control around your data and around privacy because you're not sending your data into API calls. You're not connecting your own tools elsewhere. You're able to control like how you choose to deploy these things yourself. There's a whole ecosystem not just derivative model I talked about but a lot of people build around llama tools and ways to connect and improve like um agent tic tac behavior or tools that can easily be connected to um you can customize it uh I've tried myself to fine-tune a model turns out even product managers can do it uh so so it's not so hard and I'll have a little slide also on pietorrch and torch tune you can use uh to do so Um, and then finally takes allows you to take control of your future because you're able to build a whole stack around it and evolve with the model as it's changing as we're updating it. So the open approach image always gets me. Um, so open AI open source AI is the is the path forward like I'm on the open. So why do we do open AI meta you might ask? You know we're got federary responsibilities. We try to make money and uh we try to make our users happy too. But overall I think it aligns very much with what we want to do um at Meta to do open source work. One is it's good for developers. Good for developers because it gives them tools that they can use that are top-notch that allows them to innovate outside. These developers we recruit a lot of them. So when they come to work for us at Meta, they already know these tools. There's a benefit to do that. Uh it also helps accelerate innovation in the space. And for us like we're not making like our main revenue stream isn't like API calls on our models. It's like advertising. It's these other things. So what we want is people to have the best tool possible to build create on our platforms create the best ads possible whether it's the tools we built or the tools others have built. And so if the space advances fast, if it improves fast, it benefits us in the long term. And we think it will um help the world. I have a few examples of use cases where llama has been used also in outside of meta and positive words. So the llama stack I'm just keeping track of. So the llama stack think of it as an abstraction to make life easier as a developer. Uh it connects with different ways you can deploy uh your model whether it's different hardware backends or different cloud service providers your own and different interfaces in how you connect different tools on top of lama to make it a useful product for you. There's ways you can use it to uh help with inference to build agents to manage the memory. There's also ways uh where you can use it to uh customize the model whether it's creating your own evals do fine-tuning on those and try to improve it with reinforcement learning and the likes and our community has adopted it. So whether you're deploying it on um Amazon or phase that you're deploying on face whether you have Nvidia hardware like they're all um adopted lama stack so you can use it this is a couple of pietorrch slides here. So torch if you haven't tried it out this is a way this is an example here with llama 3.1 where you can use it to um to fine-tune um a llama model. So I haven't tried it with llama 4. for I'll be honest. Uh but with 3.1 it was working quite well and there's um uh ways and and tutorials where it shows you walks you through it. The second one I'll call out and uh this was uh my pet project I'd say where Raziel and I former HighTorch engineer worked on that and executes a way you can deploy llama on device and deploy any model in effect on device but we have optimized versions of llama that work on exeutor that allow you with a very low overhead to deploy on any type of device that uh that you want to target. So, llama models, I'm only going to talk today about llama 4, but if you have questions, I I worked on llama 3. So, I can I can tell you more about this one. So, so the two first models we launched for llama 4, llama 4 scout, lama for neighbor. So, main a few of the innovations I think we're bringing in here is that compared to the previous llamas, they're a mixture of experts. They've got 17 billion active parameters being used at any given time. For this scout, it's 16 experts and for the MA it's 128 um with the number of parameters growing for one and the other. There are differences though in how you might want to use it. So for example, Lama for scout has a lot more context length 10 million context length. So this is one of the feedback we got in Llama free which is people wanted more context length. There were a lot of use cases people wanted to to leverage. um lama for that they were not able to because they lack context like if you're working with like a code base you want it all loaded you know for example in your in your model you couldn't do that before now you can you can have your entire code base loaded and work on it and ask questions about it and then um they're inatively multimodal so you're able to ask them um uh any questions about you know images and other modalities that we're going to keep adding on over time um that makes it different from Lava 3. So, one point I'll say in this one is that Lama 4 Scout fits on the one Nvidia H100 GPU. So, you can run on just one GPU, which you know, they're not cheap yet, but they're getting there hopefully. And so, you don't need like crazy infrastructure to manage it. like one GPU will handle your workload and you can handle like a very long contact sign just for that. So this is just a quick summary of um of the models that I already said most of the points in this formula. So when you think of what model to use, it's not a one-sizefits-all. You got to think like most use cases and that I pretty much deploy and play with models with right now is Java 3.370. It's a dense model. It's easy to deploy. I've already done it a bunch. Um and um it's been great. But now Lama 4 brings these additional um options for you and additional, you know, better uh benchmark numbers. I'll talk about all right. This is just the summary slide if people some people take pictures of this one in general because they they have pretty much the best use cases for for each of them that that I tend to uh to abide by. All right. So, Lama for Scout and Maverick, I'll repeat, are natively multimodal, extended context window. They've got image grounding where they understand images and text, and they were trained like that at the same time. Multilingual, which is 12 languages now, a lot better than what we had better, and superior performance at lower costs. All right. So on the eval front um I don't know how you pick your model in general but for people that deploy models what I tend to recommend is like don't look at every benchmark try to understand what benchmark works best for you and what you want to do initially is like find what's your proxy benchmark for uh for any deployment that you're looking at and then from there with llama you can fine-tune and improve on these benchmarks for example uh for this one like dog vqa 94.5 really high on that life code bench 32.8. I know there's a lot of other models are better at this, but Llama can be fine to tune and improve on all of these. I'm sorry with question. It's with questions or Okay. So, I'll go to the responsible AI element. So, one thing that's difficult for us at Meta is because we do open source, the models release go in the open. We don't control at the API call level any type of of way to protect people like people use it they deploy them and so what we want to do is we want to think like an entire development flow like how safety um impacts the model what we do also is we provide lag guard and so I'll talk a little bit here about how lamagard works um lamard is is a llama model that we've turned into a classifier so a lot of these lms can be used as classifiers where you teach them you know to understand whether this type of content is you know risk about X Y and Z or if it's safe and so the where systems are built even at beta is that for every input that comes in we filter this input through a lamag guard we classify to see if there's any risk then depending on how that's classified we send it to the LM itself then there's a whole backend system to access to tools where Lagard can also be used to figure out if the model should or not be able to request things from tools like databases or the likes and then the same there's what the model comes out with. We make sure that it's content that's appropriate to show and then we only then show it to the user. So in terms of deployment, it's not just giving access to your model. It's building this entire system around safety and we provide the tools to do that with Lana stack and lang guard in particular. So there's different versions of it depending on your use case whether it's for vision models or purely text or focus on multilingual. So we've built AI responsibly for years over 10 years uh building open research at fair. We've started the AI alliance with companies like IBM um that look at um understanding and putting frameworks around what safety is about. And for every model we have we put system cards out there on how to use it. So if you want learn more about building air responsibly, we've also got education online on how um on how to do that. And if you want more to learn more about Lama, you can scan this QR code. And that's it for me. Thank you.