No Priors Ep. 46 | Best of 2023 with Sarah Guo and Elad Gil

Key Takeaways

[Applause] hi no prior listeners happy 2024 this week we're taking a look back on 2023 by bringing you clips from a few of our favorite conversations of the year we had so many insightful guests and these are really just scratching the surface we'll list all of the episodes featured so you can go back and relisten to the whole conversation up first we have a clip from our conversation with ilas sser the co-founder of open AI we talked with him before all of the drama with the board asking Sam Altman to step down and then his return so we don't touch on any of that but in this clip we talk about open ai's nonprofit roots and their evolution into the cap profit so the goal of open AI from the very beginning has been to make sure that artificial general intelligence by which we mean autonomous systems AI that can actually do most of the jobs and activities and tasks that people do benefits all of humanity that was the goal from the beginning the initial thinking has been that maybe the best way to do it is by just open sourcing a lot of Technology we later and we also attempted to do it as a nonprofit seemed very sensible this is the goal nonprofit is the way to do it what changed some point at open AI we realized and we were perhaps among among the the earlier the early is to realize that to make progress in AI for real you need a lot of compute now what does a lot mean the appetite for comput is truly endless as as now as as now clearly seen but we realize that we will need a lot and a nonprofit was wouldn't wouldn't be the way to to to get there wouldn't be able to build a large cluster with a nonprofit that's why we became we converted into this unusual structure called CAP profit and to my not knowledge we are the only cap profit company in the world but the idea is that investors put in some money but even if the company does incredibly well they don't get more than some multiplier on top of their original investment and the reason to do this the reason why that makes sense you know there are arguments one could make arguments against it as well but the argument for it is that if you believe that the technology that we are building AGI could potentially be so capable as to do every single task that people do does it mean that it might unemploy everyone well I don't know but it's not impossible and if that's the case it makes sense it will make a lot of sense if the company that built such a technology would not be able to make U infinite would not be incentivized rather to make infinite profits I don't know if it will literally play out this way because of competition in AI so there will be multiple companies and I think that will have some unforeseen implications on the argument which I'm making but that was the thinking up next we have a clip from our conversation with Alyssa Henry the former Square CEO we talked about how AI can help small business owners with all the complexity of the parts of the business they don't love what's so exciting to me about um kind of really how the landscape has changed and the the technology advantes in the last year are how much better the tools have gotten and how much more broadly applicable they are in terms of bringing kind of expert assistance to much larger audience right but it effectively unlocked the consumer and started to then show what this technology could do um when then you know further integrated into domain specific areas you know you go talk to small business owners most of them will tell you gosh I know I should be doing marketing right like I I know if I was more effective in doing that and reaching out to my customers you know I could drive more business but I got to tell you you know I work all day and then I come home at night and I've got to take care you know take care of my family and then it's 800m and I'm starting to think about gosh you know do I just need chill for a minute or you know am I G to spend the next three hours trying to you know create an image and write text for the campaign and everything like that and what they tell you is like I know I should be doing this stuff but it's just too hard and it Tak too much time and I'm not an expert like I got into doing this because I love cupcakes not because I like writing email marketing right um and so what's exciting about all this technology you that's one example but there's so many of these kind of different things where um just the the ease of use and the accessibility opens up what previously was effectively just massive white space right it was customers or people that if it was easy enough to use if it was accessible enough if it was cheap enough they go yeah that would be that would be huge for me but it was wasn't accessible it was too too expensive it was too hard to go find and hire a marketing consultant to do it for me and the ROI wasn't there and blah blah blah so I think this this know the evolution that's occurring right now is is exciting in part just because of really the you know previously unaddressed demand that it's unlocking we also talked to Mustafa sullan the co-founder of Deep Mind and now co-founder and CEO of inflection AI about how his team worked to Define intelligence and emotional intelligence and give themselves measurable benchmarks to move toward when building their models spent a lot of time with Shane leg as well and Shane was really the core driver of the ideas and the language around artificial general intelligence I mean he had worked on that for his PhD um uh with Marcus hutter um on definitions of intelligence I found that super inspiring I think that was actually the turning point for me that it was pretty clear that we at least had a thesis around how we could distill the sort of essence of human intelligence into an algorithmic construct and it was it was his work in I think he I think for his PhD thesis he put together like 80 definitions of intelligence and aggregated those into a single formulation which was how do we um you know the intelligence is the ability to perform well across a wide range of problems and and he basically you know gave gave us a measurement an engineering kind of measurement that allowed us to constantly measure progress towards you know whether we were actually producing an algorithm which was inherently General I it could do many things well at the same time is that the working definition you use for intelligence today um actually no I've changed um I I think that there's a more nuanced version of that I think that's a good definition of intelligence but I think in a weird way it's over rotated the entire field on one aspect of of intelligence which is generality you know and I think um open Ai and um then subsequently anthropic and others have taken up this default sort of Mantra that like it all that matters is can a single agent do everything you know can it be multimodal can it do translation and speech generation recognition etc etc I think there's another definition which is valuable which is the ability to direct attention or processing power to the Salient features of a of uh an environment given some context right so um actually what you want is to be able to take your raw processing horsepower and direct it in the right way at the right time because it may be that a certain tone or style is more appropriate given a context it may be that a certain expert model is more suitable or it may be that you actually need to go and use a tool right and obviously we're starting to see this emerge um and in fact I think the key and we can get into this obviously in a moment but I I think the key element that is going to really unlock this field is actually going to be the router in the middle of a series of different systems which are specialized some of which don't even look like AI at all they might just be traditional pieces of software databases tools and other sorts of things but it's the router uh or or the kind of central brain um which is going to need to be the key decision maker and that doesn't necessarily need to be the largest language model that we have up next is a snippet from a recent conversation we had with Reed Hoffman he's talking here about how we should think about the risk of Labor replacement and how people can make a plan to best work with AI I mean the obvious thing that AI that everyone probably listening to this podcast already agrees with is that it's somewhere between the largest you know Tech transformation of our lifetime and perhaps the largest tech transformation of of of human history one of the things I used to describe it is like steam engine of the mind so just like the steam engine gave us physical Powers you know kind of superpowers of you know construction and transport and Manufacturing and a bunch of other things this will give us a whole bunch of mental superpowers it's both the amplification of humanity um which is part of what the impromptu book was gesturing towards and also there will be some places where we will create you know kind of um uh substitution uh replacement of work in various ways and obviously we'll get into some depth on that but I think that's the the the macro picture and then with that of course there's tons of things that are current status and current needs and you know I think everyone tends to a little bit overpredict like how quickly things like Everything Will Change next year and that's not going to happen um but then they tend to underpredict you know 10 20 years um in some ways in terms of how the transitions although you know obviously because just like all Technologies the doomsayers come out first um whether it's the printing press electricity everything else is like this is the end of the world you can go back and you can find that this is the end of the world in each of these things you know the printing press was described as as degrading human capabilities through cognition and spreading misinformation um as as an example and um but you know what I'd say that probably as an arc the thing that I would want to see more of in the and that's part of the reason why I did impromptu the way I did in the creation theorization and the design of what we're doing in artificial intelligence is more in the kind of um symbiotic uh amplification Loop we tend to as technologist say well I'm going to have autonomous vehicles and they're going to drive separately which I think is a good thing in that case uh because I think you know you don't need an amplification Loop you just need uh effective Logistics you know safety uh you know save the 40,000 deaths that we currently have in in human D driven vehicles and so forth can go into depth and that if that's useful but like like the fact is there's going to be a whole bunch of things that are actually going to be better with people Plus um AI that plus is a thing to focus on and I think we haven't nearly as much and that's of course part of the reason I wrote impromptu our conversations on no priors can range from the philosophical to the extremely practical our conversation with Dean ker from initro was a look into how AI can improve the economics of Biotech Discovery in this clip she's talking about probabilistic graphical models as a precursor to current architectures so I think that um just like like in most fields there is a swing of a pendulum a lot of uh the early work in probalistic graphical models was hugely influential in bringing um artificial intelligence more into the world of machine learning and uh and working with numerical data rather than just symbolic AI um and then I think the Advent of um deep learning pushed that to the side a little bit because there was so much power that could be gained from basically the kind of pattern recognition um from raw inputs um raw images text and so on without having to worry very much about interpretable representations what I think we're starting to see right now is a uh pendulum starting to swing back in the sense that there is a greater understanding that you really need a bit of both you need that uh hugely powerful pattern recognition that we get from Deep learning but you also need the ability to reason about things like cality and you also need some interpretability of your deep learning models so that you can potentially convey to a clinician why you made the decision that you did and so what we're ending up with as a really powerful Paradigm is some kind of synthesis of the ideas from both of these disciplines coming together next we have a clip from our episode with n shazir the celebrated Google engineer and now the co-founder and CEO of character AI where he talks about why he's a text nerd and the possibilities of language models I've just had my head down in uh in language like that like here you have like something that like a problem that like can do like anything like I want this thing to be good enough so I just ask it like how do you cure cancer and it like invents a solution um and you know like so so I've been totally ignoring like what everybody's been doing in uh in all these other modalities where like I think a lot of the early successes in in deep learning have been like in images and people are like all excited about images and I kind of like completely ignored it cuz like you know an image is worth a thousand words but it's like a million pixels so like the text is like a thousand times as dense so like kind of big uh big uh text uh text nerd here but um you know very exciting to see it uh it take off in you know in all these other modalities as well and you know th those things are going to be great it's like super useful for uh Building Products that people want to use uh but I think that a lot of the core intelligence is going to come from from these text models to wrap up our favorite moments from 2023 we have part of our conversation with Arthur men the co-founder and CEO of Mistral talking about the evolution of collaboration in the AI space and why mrr's mission is to keep AI open models can output any kind of of text uh and in many cases you don't want it to Output any kind of text so when you build an application you need to think on the guardrails you need you want to put on the model output and potentially also on the input so you do need to have a system that filters input that are not valid that you deem ilal and output that are not valid or that to Illegal so the way you do it in our mind is that you do create the modular architecture that the application maker can use which means you provide the RO model so the model that hasn't been altered to ban some of its output space and then you propose new filters on top of that that can detect uh out the output that that we don't want so it can be I don't know pornography it can be hateful speech these things you want to ban when you have a chatbot for instance but these things you don't want to ban from the RO model because if you want to use the RO model to do moderation for instance you want your model to know about this stuff so really assuming that the model should be well behaved is I think a wrong assumption you need to make the assumption that the model should know everything and then on top of that have some modules that moderate and and guard the model so that's the way we approach it and it's a way of empowering the application maker in making a well gued application and it's we think that it's our responsibility to make very good modules that allow Guard railing the model correctly it's part of the platform and we think it's it's the way of uh we there should be some uh health healthy competition on that domain of different startups working on Guard railing the models and the way you make this healthy competition is not by trusting a couple of companies to do their own safety it's rather for it's rather the way you do it is to ask application makers to comply with some rules so chatbots should not output hateful speech and so that means that now the application makers need to find a good Guard railing solution and now you have a competition where you have the where there's some economic interest in providing providing the best Gathering solution and so that's the that's the way we think the the ecosystem should work and that's the way we position ourselves that's the way we build the platform with modular filters and modular mechanisms to control the model well we of course have to mention our chat with the amazing Jensen hang co-founder and CEO of Nvidia here he talks about how Nvidia decides what use cases to support what applications of AI he's most excited about personally there are a couple things that that our company is shaped um and structured to do there's one part a very large part of our company is designed to uh build very very complicated computers perfectly and so that's that is um uh one of its missions okay and and that kind of architecture that kind of organization uh is a is a um invention and refinement organization and then we have we have um a a a whole bunch of of of um uh Skunk Works if you will and the reason for that is because we're trying to invent things 10 years out that we're not exactly sure whether it's going to work or not and and there's a lot of adaptation a lot of pivoting and um and so so you know our company actually has has two different ways of working one of them is rather organic shape-shifting all the time if a particular investment is not working out we give up on it move the resources somewhere else and so that's the agile part the company and then there's a part of the company that's not rigid but it's really refined MH and so these two these two systems have to work side by side thank you all so much for listening last year if you want to dive more deeply into any of the conversations you've heard today we've linked the full episodes in our description we'll be back next week with more interviews with the leading Builders and thinkers in Ai and Technology find us on Twitter at no prior pod subscribe to our YouTube channel if you want to see our faces follow the show on Apple podcast Spotify or wherever you listen that way you get a new episode every week and sign up for emails or find transcripts for every episode at no- pri.com