No Priors Ep. 27 | With Sarah Guo & Elad Gil

Key Takeaways

hey everyone welcome to no priors I'm Sarah Gua I'm a lot go this week on no priors we're back with another episode where we answer your questions about tech Ai and everything in between I think we have a lot of different questions that people are brought up this week that they were hoping we could cover and some topics that we thought were would be kind of interesting I want to go to one of our listener questions and I think a topic that's really popular with many of the companies that you and I work with in terms of access to Computing for a much smaller scale experiments what uh what's going on with the GPU crunch yeah the companies that you and I work with many of them are companies that you know they need to use very specific infrastructure to train and serve large models right these work on gpus and the structure of the industry is like um it's just not very robust right so you have a very small number of producers Nvidia and amb AMD generally and then Nvidia is very far ahead on the high-end processors that are most efficient for large scale training in inference then you have the pandemic Supply disruption which we haven't fully covered for if you actually look at the supply chain you go from the actual designers to you know the Reliance on a few major foundries like tsmc um you know expansion of this capacity is not easy right new Fabs are billions of dollars yield is a very complicated thing you can think of it as a massive Precision manufacturing problem where temperature pressure chemical concentration tool imperfections new processes materials issues like anything can make production have lower yield or lower quality right and and so like if you think about the speed with which the industry driven by both large and small players has decided that they want to do AI uh like the physical processes cannot keep up with that demand it's as if you know half the companies in the world over a year-long period decided like yeah we need super computers not superconductors but gigantic networked gpus so like what is the actual Gap so you're to your plane it sounds like much of the AI world is dependent on gpus in order to train and then do inference on these big AI models and the big suppliers are basically Nvidia AMD and then there's like a long tail of smaller folks what is the Delta between the amount of capacity that exists today and that's needed are we off by 2x 10x some other number it's hard to say because uh right now there's no way to explore like the price elasticity of these things right um so you know just very specifically like the industry is kind of looking at deliveries in small quantity in September larger quantities in December January most of the large Cloud providers are sold out for any scale for at least through April of next year and so you have like really interesting Dynamics like large Cloud players who you know are the biggest consumers of of these gpus already like a Microsoft going and buying from other providers for near-term near-term Supply right so I think one question that I ask you is like do you think this is a long-term thing do you think it's a very short-term thing but I I think it just goes back to like the the fundamental Dynamics are do you expect the demand for these chips to continue increasing at a pace that outcreases the ability to scale a very physical like real world process right just to even be more specific one of the challenges like I was talking to Jensen about this and a bonder like not part of the GPU itself but like a critical tool in the manufacturing and assembly of gpus is very specialized and so the ability to build any of these tools as well to enable these processes is is a blocker if you look at the demand from large Labs today to continue increasing model scale and training Time by magnitudes I think it's hard to see that Dynamic going away what do you think I feel like there's a couple different sort of second order implications of the fact that we're seeing this giant GPU bottleneck I think the first one is that we're seeing new sort of models that are dependent on GPU access or ownership is ways to create all sorts of really interesting monetization and potentially eventually cloud services so that's things like Coral weave or Foundry ml or other companies that are basically providing now gpus in different ways in some cases through aggregation or federating different sources of gpus in some cases it's just having these large GPU clouds and being able to use them in really interesting ways and one of the interesting I think side notes is that gpus used to be very heavily used for crypto mining and while crypto is down it may actually be more economic to just use them for to rent out for AI training purposes or inference purposes so I think that's one really interesting almost like sectoral shift in terms of existing GPU capacity the second is that a lot of the different players that are startups who've built their own semiconductors specifically for AI training I think are starting to see a lot of really strong pull so for example cerebras and I think we're going to have Andrew from cerebrasan um our podcast in a couple weeks they just signed a 100 million dollar deal with UAE for building nine super computers using their chips which are optimized for AI amazing and so I think they and grok and other sort of semiconductor providers are going to find really strong pull during this period where people are desperate for any solution and they're willing to do take the extra steps to really be able to utilize other forms of silicon and so I think it creates a bit of an opening for other players in the market and so it does seem like it's going to have these really interesting sort of cascading effects on members of the startup ecosystem and you know new players that are working against all this two um sort of second order things are like what do you do when scaling is blocked on capacity like you try to be more efficient it's not been an area of massive Focus to date because people have been chasing the state of the art following chinchilla scaling as the simplest path forward but there are really interesting lines of research that are undervalued today unless the hardware supply crunch continues including in dynamically figuring out or routing to efficient models so think of like The Frugal GPT work or generally like distillation or even just a more intelligent choice of data for your pre-training or your fine-tuning training mix so you can use less compute right for for the same or for for improved quality and I I think like everybody's been on this one path and an interesting second order effect is like does it spread people out into lots of different directions in terms of chasing performance I personally don't think the um Supply crunch goes away immediately and like a part of the dynamic is just you know how much more people want to scale and another part is like you know if this stuff is actually useful then inference like inference already dominates open AI compute usage right and so that demand will continue to go up yeah I do think demand will only um rock it from here at least in the short run and so the real question is the degree to which the semiconductor industry adjusts to that and the reality is that people really view nvidia's chips as the most advanced on the market right now and so that means that a lot of it is just a bottleneck and how much can Nvidia scale up manufacturing and there's other players like AMD there's the startups we mentioned cerebrask and others but a lot of the capacity is just going to be how much can it can Nvidia and maybe AMD scale up in the short run at least and so that may just cause some ongoing bottlenecks assuming again that we continue to see this very rapid growth and an AI and AI applications I'm working on a blog post right now actually about this because it feels to me that we're still in the very very early innings of this wave of AI adoption right it's not a Continuum where we had cnns and rnns and that's something we have Transformers Transformers created a whole new capability set and we're only you know eight months since chat GPT in a few months since five months I think since uh gpt4 and so the only people who've really adopted this technology yet are the AI native companies like open Ai and the journey and a few other folks and then we had the first wave of startups come the perplexities and Harvey's and characters of the world as well as the first wave of incumbents adopting it notion and zapier and sort of very very early founder-driven adopters and so we've had zero real Enterprise adoption in terms of real products at scale or close to zero and you know most Enterprises big businesses take six months nine months is to do their planning Cycles and then they'll spend a year prototyping and then finally they'll launch these AI apps and so we're probably a year or two years before we really start to see large-scale AI applications by existing incumbent Enterprises in real products live everywhere so from a ramp perspective one can imagine that a lot of the future ramp and AI is coming in about two years or you know one to two years something like that so there's still a lot of um room I think for the hype cycle for increasing ongoing excitement sometimes irrationally so and then also for sort of adoption of semiconductors and other underlying infrastructure so there's still a lot to come it feels like I agree with you and I still think we're really early in let's say like the collective exploration of applications and constraints right like you had the people who were bleeding edge of just personal interest uh like I think Chachi BT was is looked at correctly as the starting gun for people to begin developing these AI applications generally but if you think about how long it takes to ship actual interesting products to Market and then the build up of some collective understanding of like how to make these models more useful in different applications and then you know turn them into workflows and then Advance the state of the art given a particular workflow if you have a hypothesis on value like that'll that all takes time so I think we're in inning one yeah it's all been demos so far yeah so I guess related to that a lot of the interest and excitement right now is around to agents you know I spoke recently there's a group um called the AGI house which you know Jose's different hackathons in the Bay Area and stuff like that and they had me come and help kick off like a agent hackathon they had and things like that what do you think happens in the ancient world like what form does that take and is it a handful of very broad Asians is a highly specialized ones like what do you think has come in there yeah uh it's such a um like powerful broad idea that I think both will happen right um and and so like the the overall idea is you know you you don't just talk to a chat bot or or query an interface you have some sort of planning mechanism that is model driven that allows you to take asks autonomously take actions autonomously um and like complete a more sophisticated task often using other tools and then return that result or report back on your work to an end user right and so you know I think that is going to range from um the pure consumer applications so things like inflection which is going to you know have personalized that do more for you minion which is working on like web agents uh and and then you know I think like there's been very recently more attention or just more understanding of how powerful it is to have agents that in some way write executable code right um because you can programmatically use many more tools you can call apis and I think if that is uh do a task that is not a single query but requires multiple steps in uh in analytics or an Enterprise automation um or even you know within like you know companies that we work with uh like Harvey like a single legal task is actually a composition of thoughts planning attempts of research like writing that a an associate might do and so I think it's going to be a pretty dominant paradigm yeah it's kind of interesting because if you look at past technology waves and you ask about specialization versus sort of broadness you know are you building a broad-based platform that you can use for anything or a vertical application that really helps you with one or two things well most of the things that really work are these vertical applications that help you really well now some of them broaden and grow into the broad-based platform for everything right even in consumer that's true like Facebook started off as a college network and in fact it started with like five colleges and they added all colleges and then later they added the ability to add your work email as a way to register and then they open it up to everybody and then they start building the platforms on top of it in gaming and other things right but it kind of happened sequentially and there's counter examples to that you know Google would be a very broad-based thing from day one it helped you discover information on the web right you needed a tool for that but it feels like in the agent World a lot of the people that I hear talking about ideas have these very Broad sort of abstract ideas and so an idea would be um I'm going to build an agent that is going to be your assistant and you're like okay well what is it going to help me with and they say everything it's going to make you happy and you say well I'm you know I'd love to be happy but at the same time you know starting with a very targeted focused initial use case tends to be the best way to build product a because you know who you're building it for B you can really nail the use case and there's the old sort of ycism which I think which is really good which is better to Delight a small number of people than to have a very large number of people indifferent to your product and so I think my my bias for the agent world is if you're building an agent start with something really targeted if it's a assistant to help you what exactly does the assistant do does it do background information searches on all the meetings you have that day does it specifically help with certain forms of scheduling does it help with other aspects of your day planning or synthesis of what you've done or follow-up action items or whatever it may be about choose one or two things and do them very well versus do everything and then eventually you may build a thing that you start off that does one thing very well but then broadens into everything but usually starting with everything means you're not really doing anything deeply or well and so I think that's that to me is one of the main patterns at least in terms of Prior waves of Technology development I very much feel like this is like a very classic tension between um what I consider to be like uh I don't know the like infrastructure platform engineering like even research agenda driven approach that is like oh you don't understand like the technology is General we don't want to be taken off the research path that pollutes our uh our data mix in a way that it is not a general purpose technology anymore right um or you know it can do anything why limit it um or even getting feedback from users because you release this stuff it is broadly capable that they're doing everything with it something's much more successfully than others and I think more of a like a product engineering like traditional like startup mindset that is like actually complete the task right and I I definitely think um uh the overall exploration has been skewed to one side not as productively today um and one of the like even if you think from the research agenda one of the reasons it is interesting to think about the like uh the you know have more Focus everybody's thinking about but have more focus on accomplishing the specific task is like you want to be happy a lot all I want to do is like never write boilerplate code again right and so if you think about that's how I define happiness okay great then we're still the same um but uh like if you think about like okay let's like complete one task if I uh want to ask um you know an agent to just like fix all the bugs in my software um then my uh ability to like successfully complete that task includes a lot of like bug fixing specific techniques right like you could do test time search and then see if all of the different things that you uh generated actually execute as one very simplistic example right and so like I think there are a lot of ways to advance in um the research in very specific tasks that are much more attractable but maybe I'm not thinking big enough that makes sense I think um I would add one-third piece to that framework you have which is the research driven versus product driven I think there's a third approach which is infrastructure tooling driven and that's where you're like I'm not going to build that agents but I'm going to build the infrastructure that allows anybody else to build them rapidly now sometimes those types of businesses or approaches work really well and sometimes those things are solely an outgrowth of a vertical product that works really well that you then open up the infrastructure for everybody else to use and it's very Case by case dependent it's the difference between stripe where it's just like we need to build payments for everybody everybody keeps building it over and over again and the Facebook all the platform which only existed because you got to hundreds of millions of users you could open up office like a third party service and so I think as people think through that third angle of building an infrastructure for others they need to understand whether that infrastructure will be an outgrowth of an existing product area and benefit from the characteristics of the the market liquidity of that product or whether it's just a piece of infrastructure everybody keeps building over and over and therefore it's a really good thing that just provide to the world so I think it's kind of an interesting future topic we are on a you know couple month bull run at this point 2024 Tech markets what's coming like will people be able to fundraise will funds be able to fundraise our customers purchasing you know I think there's going to be basically um four markets next year in some sense one market is just Ai and I think AI will continue to run in different ways and it'll look very expensive at the time and a handful of companies will look really cheap in hindsight just like with every other technology wave and I think that's separable from the rest of tech that existed prior to the AI wave for companies that fundraised in 2021 prior to being like AI companies a subset of them I think if I were to sort of divvy up that pie of those companies sort of mid to late stage private tech companies not an AI and what's going to happen to them next year and in 2025 I think a third of them are just going to go under or a third of I should say unicorns we'll eventually just go under be fire sales whatever they won't be able to ever raise money again a third will be at the highest valuation they'll ever be at ever in the lifetime of the company they'll reach their terminal value and those examples from 2014 of companies that went through that same wave they've you know raised in 2014 they went public a few years later and then they never surpassed their their market cap again and then I think lastly there'll be a third of companies that grow past it and so I do think there's going to be a lot of Carnage next year and a lot of companies going under and as those companies go under three things will happen number one it'll be much easier to hire people and people are already seeing that at startups it's easier to hire again second it should have follow-on effects and ramifications for commercial real estate and we'll see a second shoe drop there and then third The Venture Capital Community will be impacted because a lot of the things that they've been using to fundraise new funds or do other things with Will suddenly go to zero they're big unicorn success will go from a multi-billion dollar or billion dollar company to basically a company that isn't worth anything and so I think that's going to have knock-on effects to the Venture ecosystem but I think that'll take like two three years to play out because all these things are a bit time delayed um but yeah I think that other shoe still hasn't dropped in private Tech markets and a lot of it is just companies raise so much money in 2021 they still have lots of money so everything still feels like it's continuing to go but at some point that money's going to run out so I think it's going to be a pretty bumpy 2024 and 2025 potentially yeah my advice to companies that you know raised a very healthy evaluations during that period of time and then are actually building businesses is to try to completely disassociate from that valuation um because people will put themselves into all sorts of contortions to do a flat or up round to evaluation that makes no sense right yeah and if you don't have the historical context of that making no sense it's an extremely painful um sort of realization to have but if you look at there's this um one analysis of uh actually the very best technology companies and the ones that endured from the internet bubble and how long it took those companies to reach the evaluations they were at before the bubble burst and it's a decade right and it's like startups don't have a decade to try to you know get to at par evaluations yeah I'm actually less worried about valuation I think valuation is ephemeral right effectively every or roughly every tech company in public markets did a Down Round over the last year and a half right they all lost or many many companies lost 30 to 90 percent of their value right effectively they just did it down around in public markets because every day you're repricing a public stock I'm more worried about the people who burn tons of cash and they don't have a lot of Revenue to show for it and then when they're going to go out to raise more money people say well you burnt 50 million dollars you burn 100 million dollars you generate five or ten million dollars of Revenue and so the issue isn't that your valuation is off we can always reset valuation it's the fact that you burned all this money and you don't have anything much to show for it right and that's where I think the real issues will happen because you can always reprice things and people will be forced to and you know it'll just happen but I think it's the underlying business case and business model that's going to be the real issue yeah I guess like the the unforced error there for companies who actually have the time to make the decision is um the thing you want to avoid is like not adjusting your cost profile or you know holding on to that valuation until it's too late yeah or just deciding it's the wrong business and it's not working and you know the most important precious thing for you as a Founder is your time and I think people forget that you have this golden period in your life where you don't have hopefully a lot of other complications in terms of sick family members or School related issues or whatever it is and you can take risk and you have a low-cost basis and you can do all these things and that's the moment when you can best take risks to start a company for many people not for all and you know you're really giving up the best years of your life working on things that potentially Network thanks for the discussion it's a lot of fun yeah super fun thanks to everyone who sent us your questions find us on Twitter at no Pryor's pod subscribe to our YouTube channel if you want to see our faces follow the show on Apple podcasts Spotify or wherever you listen that way you get a new episode every week and sign up for emails or find transcripts for every episode at no dashpriars.com