No Priors Ep. 48 | With Covariant CEO Peter Chen

Key Takeaways

hi listeners welcome to another episode of no priors this week I'm joined by Peter Chen the co-founder and CEO of covariant a robotic startup that is developing AI robots before he started covariant Peter was a research scientist at open aai and a researcher at the Berkeley AI research lab where he focused on reinforcement learning meta learning and unsupervised learning he is a prolific publisher and now a Founder I'm so excited to have you on today to talk about what's uh going on in robotics welcome Peter thanks Sarah it's great to be here um there is there many exciting reasons to be here one is I have been a frequent listeners um of the podcast and the second one is just because of the name like I just have to be on this show so it's great to be here right let's go establish uh some some priors for everybody uh in a very unknown landscape right can we start with just uh why you were drawn into Robotics and the beginning of your research Journey yeah when I was working on Research at both UC Berkeley as part of my PhD uh and at open AI there were two topics that were particularly exciting to me one topic is like as you have introduced UNS supervised learning like how can we build models that learn from vast amount of data and we now more colloquially known this as generative AI because like we train this large models on large amount of text images videos uh you learn from them in an unsupervised manner that topic has always been very interesting to me because if you want to train very capable AIS you want to have a lot of data uh and where you can get a lot of data is through this kind of unsupervised data set and then the second topic that was really interesting to me was reinforcement learning like it's not just building models that understand but building models that can make decisions um and reinforcement learning teach these models to make decisions by having them make trials and errors and learn from the better decisions and do less of the worst decisions and Robotics is just a such a great combination of these fields like in order to build really capable robots they need to really understand the world in a very very robust way and they are not just passive agents that just understand text or what's in an image they actually need to take actions in the real world and the consequences do matter and so we found robotics to be such a great way to both utilize the advances in AI but also we think of it as a way to also Propel AI forward like this is where you get the grounded data this is where you get that embodied data of not just AI that is trained on browsing the internet but AI that is trained with physical interactions with the world and so we also believe robotics would be a key way to advance AI that makes sense you were at places that are great places to do research why did did you decide to start a commercial company it's a really good question um I mean there were a lot of companies that are founded by prior phds that kind of the classic journey of there's a technology that was built in a lab environment and it got to enough a level of maturity that oh we should start to commercialize it in the real world that was kind of not the Journey of karian like when we started karian there was not AI that was good enough to make robots do useful things commercially uh and so it was not a classic journey of technology developed in Academia and then transition to a commercial landscape the key Insight that we had at that time when we left open AI in 2017 to start covariant was the future of AI is going to be the future of foundation models these models that are truly multitask learn from large amount of data and are as such be more generalizable they can solve new tasks more easily and are also more capable at at every single one of the tasks because of the transfer that you get across tasks we just had early conviction that that was the path to build Ai and that is also going to be true for the physical world for robotics but there's one big problem which is you have no data set to build robotics Foundation model like there's no data set that you can build this AI that understands the physical world and take actions in the physical world um and so in order to build this Foundation models for robotics you really have to build a company that can collect data to do it and the only way to collect enough data is to build fleets of robots that are actually creating value for customers that so that you can collect those data in production because even if you try to scale up data collection in a lab environment there's a limit on how much you can do that in that perspective like we strongly believe in the Tesla approach like where they have the most self driving card data because they ship a great cars that people want to drive and a good enough entry-level autopilot that people are willing to use it and they're creating value for their customers like customer use their products and those data that they collect can allow them to build much more capable models and Ai and so um why we left open Ai and um Academia to start cing is very much this belief that in order to build Foundation models for robots you have to have a lot of data and in order to have a lot of data you have to build autonomously working systems for customers and the only way to do that is to build a company to serve those customers yeah there's a really interesting tension if you're trying to build a let's say AI capability that doesn't exist yet because the you know there's no model that is good enough of how much you invest in that upfront versus deliver the product that already exists in the world right like you could just go build a bunch of robots and deploy them on mass or you know if we draw a analogy to um the prior generation or current current existing generation of autonomy companies like we were you know I involved early in my prior role in Aurora and nuro and then I was personal investor in Kodiak right like a lot of these companies you you were trying to build a a brain is is an alternative to the Tesla approach and I think the the economics of collect as you go is getting very very compelling just in terms of how expensive it is to try to sequence it the other way yeah like this definitely needs to be a um incremental approach like you have to just find like the right sequence of what is the technology advanc that I want to build now that enable enough of a products that I can deliver which then in turn allow you to build more capable models that then in turn like a larger service um of area and this is like I mean we we have seen this play out in the non robotics world as well right like if if we think about open Ai and thropic cohere a lot of these big language models um players um like the models that they have are not fully General language models yet right but they are good enough that can solve a large section of problems that it's worth productionizing them um getting commercial value out of it which then in turn allow you to build the next incrementally um better system and I think of it as the same kind of um Road mapping exercise that you have to do in autonomy like you you cannot just go straight to the like full General physical AGI um at the beginning like you have to build something that that like represents like a justifiable INB spend as well as timeline that you can justify but that allows you to build something that is valuable that you can ship to customers and from that process you get more data you get more learning that then in turn allow you to build the Next Generation model so we think of is very much an interative approach uh and having real products and having real customers like allow you to ground that approach as opposed to um just be in a philosophical debate of like how we build this super super General thing that is very far in the future then I think the right way to start actually be to ground the conversation and kind of like the application landscape can you walk us through the sort of limitations of Robotics in warehousing and Manufacturing that are common place right now and how much intelligence these robots have robot are extremely common um nowadays like so what we typically work on are robotic arms so think of these as six axes 7 axes um robotic arms that can do very flexible movements they are super precise they're super fast and super doable and very cheap lots of factories around the world have robots um but the challenge is like 99 plus% of the robots that deploy in the world are dumb robots like these robots are pre-programmed to do the same thing again and and again and they don't really have any kinds of intelligence that can adapt to new circumstances communicate with people and change what they do on the flight and so think of Robotics that exist today are extremely rigid and so really the problem that we are solving is we're not trying to make the existing dumb robot use cases better right like we're not trying to say oh instead of uh manually programming this robot you could just have an AI that that programmed robot we're not talking about that like we're really talking about like opening up a couple orders of magnitude more use cases where the robots actually need to be smart like they need to adapt what they do based on the scenario that is presented to them right so like the a good way to visualize this is on one hand like think about a robot for example in a Tesla Factory that is handling a car body right okay this is this has this is a very incredible feat of engineering that can move like multi-ton object um very fast very precisely but it's just doing the same thing again and again like and then imagine another robot in a e-commerce Warehouse that has hundreds of thousands of unique items that it has to distinguish pick up and pack carefully into a box that gets shipped to you that's a very different kinds of diversity um um that we're talking about and so when we think about building AI for robots when we think about building Foundation models for robots we're thinking about really lifting robotics as a category from this former category of just being able to do repeated things to this category of really being able to handle diversity um of environments changes in the environments and being able to understand what's around it and make intelligent decisions and actions um to handle a diverse set of circumstances and we think like this would enable really a whole different way W of Robotics that is not how robotics is used um today and for cian specifically we are starting from um Logistics and warehouses as an industry that we focus on um so this is think of it as the explosive demand that is driven by the growth of e-commerce there's a lot of complexities that's been injected into the logistics and supply chain um and at the same time coupling that with demographics change changing immigration landscape makes few and few people want to do this kind of warehouse jobs like drive an hour and a half to the suburb and then have to work through the midnight like these are not the kind of jobs that people want to do and our customers have extremely high turnover rate like an average Warehouse that we serve have typically more than 100% uh year-over-year turnover rate and so like these are the type of places that we have an extreme shortage of people that want to do those kind of jobs and yet at the same time there are no prior robots that can solve Peck pack ship in warehouses because like traditional robots are just machines that do the Motions that you program it to do repeatedly and but here you actually need systems that's actually adaptive and do it at a very high level of reliability can you um describe like like how we should imagine the physical like you obviously have Cent brain but then you have the physical instantiation like what's a what's a put wall just for our listeners yeah so a um common use case that we have for our customers is what we typically call a put wall use case a putw is a uh ter that is used in e-commerce fulfillment um and which is like when you click a button to buy something online and and then the Box show up to your door and you might wonder like how is that done well there's a complex sets of operations that's happening in the background uh and the put W is one step of that like and this step is typical used to sort a mix of customer orders to different customers right like let's say both you and I have order a new generation of iPhone right and then like a robot would be sitting there and picking up one iPhone and say oh this one should go to Sarah and this one should go to Peter if you think about like what that robot needs to do like the robot needs to have an incredibly great ability to grasp items without damaging it uh and have the accurate ability to identify what is the item and then route them to the appropriate customer like in this case like either you or me uh and so put wall you can think of it as some sortation mechanism you can think of it as a physical router that exist um in the world like so instead of thinking about um Network router that sends digital packets around like you can think about putall as a physical router that sends Goods to different places is it fair to say that you know identification and routing are more solved problems than grasp in I would say identification and routings are typically more considered more solved problem than grasping like because if you there are other like more um mechanical way to solve those problems like you can design a piece of conveyor that like if you always put an item to the same place then you can route it to a design location and so like that becomes mostly a mechanical problem and anything that is a mechanical problem is typically more solved and so that is very much true like I would say out of this grasping identification and routing like definitely the grasping Park um involves more AI but as we build more advanced Ai and bring it into a more traditional Fields like robotics like what we actually find is that even in the identification step even in the routing steps there are a lot of ways that AI can make more traditional mechanical systems smarter right like for example like a classic way to do identification is through scanning the ble but um where's the ble like how do you scan the ble well that's actually something that AI can inform it right and like often times like human can identify an item without even scanning the ble because you can read the packaging like you can infer like what is in there uh and that is also something that uh AI can help and so like while it is true that there are some steps of the problems that can be solved by more traditional mechanical and robotic systems uh what we have found is that like once you have a very flexible AI you can actually rethink a lot of the processes like you would make something that was previously impossible possible like grasping and then you can also improve a lot of the other steps of the processes that were previously possible but now you can do them in a more intelligent way is the um next step of expansion that you are excited about for covariant still within um Pick and Pack or are there other tasks within um warehousing and Logistics that you think are really interesting to expand into or you know there's um other phrase into different robotic applications like you know humanoid robots like the Tesla Optimist or other industrial applications yeah um a couple like starting at a very highest level right when we think about the covering brain this Foundation model that we building like we are not building it just for warehouse appli applications we are not just building it for pick employees uh applications within warehouses um so definitely like everything that you're talking about it's very exciting to us like so both applications outside of warehouses as well as applications to newer Hardware form factors like humanoid um um robots and so like that definitely is the long-term path um for us I would say like in the very immediate future uh as a company we are focused in the manipulation space of warehouses just because there are so much demand and there's so many different kinds of use cases that exist um in the warehouse domain already because a warehouse for a apparel company is very different from a warehouse for a Cosmetics company which is very different from a warehouse for a me prep um company and across all of these you actually have very different manipulation skills that you need and very different kinds of data that you can collect to train the foundation model and also very different large markets um that we can tap into um but we are very intentional in how we build the models in a way that make sure it's generalizable and so you can actually extend into new domain uh and one more comment on the humanoid um question like I think that would be one of the most exciting events in robotics is to make humanoid as a form factor possible right because our world is designed around human bodies like so humanoid is the universal Hardware form factor that can be dropped into any place uh in our world and so like we really like we cannot um um we really cannot wait for the human noise to be like commercially and also technologically available like because when that platform is available that is really the best mechanism for us to deploy covering brain this Foundation models to go to more places more quickly fortunately we are not relying on it like even by using the existing industrial robots Hardwares um we can build a scaling business we can continue to boost strap and build incrementally more capable models but if when it comes like that would be a really big acceleration for us mhm mhm one more um question on the sort of uh application or um maybe just the covariant side before I would love to talk a little bit more about the research is um can you give our listeners a sense of you're five years into covariant like how big is the team you have robots in the production any like what are your types of customers yeah so cian is about um 200 people company uh and we are extremely International um I would say roughly half of our customers are in Europe half of our customers in North America um and uh we have robust deploy across three continents at this point and more than 10 countries um and what is really remarkable uh all of these customers all of these different robots are network together like it's one single Foundation model and everything that they learn come back and make this Central model um better and our customers are typically large retailers large e-commerce Brands uh and essentially anyone that runs a um large distribution centers or a network of distribution centers like would likely choose karian um as their model that power the physical world amazing can we talk a little bit uh just a about the research and I I think the first uh thing I'll ask you to explain as just a very high level concept is is what the concept of grounding in understanding of the real world or you know Foundation models that understand physics and objects interaction like what that means or you know how that's missing today yeah um so grounding is this interesting idea of um like if you just read the text on the internet like you learn a lot about abstract Concepts right but but they could be like purely symbolic like you might read apple is delicious okay I I I I have this Association that okay like something that is Apple could be delicious and if I ask for a delicious thing you can say apple is a delicious thing but that is very symbolic like that has like no actual grounding in our physical world like what does an apple look like if I give you an image of an apple can you recognize it uh and can you recognize like the different other physical properties of an apple uh and so like the first thing that you want to do is like grounding is to to ground all of these symbolic abstract Concepts into something that is real that is physical um and there are actually a lot of advances of this like even outside of Robotics um that's happening already like we have a lot of multimodel model that exist um um in the world like if you go to gbt 4V like you actually could give it an image and then it can um answer something for you intelligently about what's in the image like so like GPD 4V has grounded like this type of multimodel language models like already have an understanding um of um those Grounded Concepts so where does where does it get those grounding from like it gets those grounding from um essentially the image and tex pairs that happen in on the internet right like if you look at uh an Instagram image like it might have a set of captions um along with it so we can train this kind of multimodel models with a combination of those data right like after you have seen enough of the uh Instagram image of an apple and enough of people tag them as Apple then after you have trained on a large amount of such data you start to get that grounding you start to pick up that um associations um so that's like I would say outside of Robotics like how typically grounding happens and how you typically get this kind of multimodel um understanding that understands Beyond just pure symbolic Concepts but actually has an understand of how it gets associated with the real physical world uh typically manifested through an image of the real world and uh if I think about just the concept of an apple is in many videos on YouTube um they are kind of round they are affected by gravity they have some Mass like what's missing from those captioned images and videos when you talk about like the data that's missing that you need to go collect for robotics to improve yeah so um there are a couple aspects of it right so um like obviously this kind of Internet scale data is very useful like you can already pick up a lot of Association and grounding with the physical world um but there are still a lot of things that's that's missing right so for example like when you think about this kind of um naturally occurring text and image pair data they're typically about high level Concepts like they're typically not about something that is very precise like so for example like when I present it an apple to you like you don't typically describe like the precise shape of the apple right like is is is this like a very round shaped apple is this like a very full Apple like you might use some high level concept to describe it but there's really nothing that describe it say down to submillimeter level Precision which is kind of like the level of like precise understanding that you need to interact with the wheel you you don't just say oh there's kind of an apple there but there might be like up to a 2 cm like difference in understanding of where the boundary of that apple is and how should I do it and so like here's like the first dimension of like things that is missing which is like there's really no very no precise grounding um um there's no precise understanding of the physical world that's naturally occurring um on the internet um so that's like one of the first thing that you find kind of the departure of Robotics Foundation models from like other General uh multimodel Foundation models like is this idea of precision like you now actually need to understand things to much higher level um of precision um that don't otherwise exist uh in this kind of data set um and so that's like one big thing and then another really big thing is um like this ability to um understand effects of your own actions uh and a large part of this is just because there are not a lot of robots that are doing interesting things uh in the world and so like there are not a lot of data sets that uh are in the format of robot does something and you know the outcome of it like is this a good way to pick up something like if I move an item too quickly like would it damage it if I press like for example a tomato like what is the force that is appropriate that that is possible like you don't have a lot of these kind of um action and outcome pairs um that exist in the world like the closest thing to that is probably on the YouTube you have human doing those things but then there's a research question of like well can you have a robot that learns from just watching a human does it and you don't actually fully know like how hard does a human press on the tomato or like how you precisely they size something so you're still lacking a good amount of the data that like completes this feedback loop do you have some sense of like how or if scaling laws apply for you like do you know how many robots you need to deploy or how much data you need to go collect to get to certain levels of improvement or can you try to predict it now so I would say the most technical C definition of scaling law um does apply and we have seen it apply uh in this domain and it's somewhat not surprising because like like if you think about like the scaling law in the most technical sense which is if you scale up data and you scale up your model capacity and you scale up with the compute that you throw at it you get lower loss function like training loss function um out of it and we have seen this play out across so many different domains like more than just language model that do not surprising um I think the question that you're asking is probably the more um not the most technical definition of scaling law but it's the general definition of scaling law which is as you scale those up would you get emerging capabilities out of it like would you kind of like get something that's is like modeled as orders of Mag smarter in some loose um definition of it like which is kind of the thing that we see from the large language model world like when you go from GPT 3 to G 4 when you go from cloud one to cloud two like you kind of like see this step change Improvement in reliability in generalization um that you get from it so I assume that's like probably what you're what you're asking yes do you believe in some emergent so I would say we see some element of it but it is something that we rely Less on and here's like where I think there was a really interesting crucial distinction between a um quite full General model that is designed to solve everything in the world to what I think of as a domain specific um Foundation model like in our case like solving robotic uh manipulations so in a full General model like for example like GPT 5 that you wanted to solve everything in the world then you have this problem of essentially out of domain generalization like when we say like like as you scale it up like do you get something that is much smarter out of it like we are not seeing like whether gp5 would fit the training data better like we are saying like if you give a scenario that is completely outside of training data like how well does it work and that is where you kind of like need to rely on this strong form of scaling law um but you kind of don't need that um when you are in a more restricted domain like robotics um because like you actually could have so much data coverage that your test scenarios are just part of your training scenario um so to some degree like we actually don't need to rely on this strong form of scaling law to hold um for us to build really valuable technology out of it um and so I expect like something similar like that would happen like would follow the similar Trend that you see in the language world but at the same time like we don't we don't require it like we know that like as you get more customers as you get more data like these systems would get better and especially if you have targeted data coverage for specific domains for specific customers like they would be guaranteed to get better like so to some degree like we um whether you believe like robotics can scale or not it's it's a simpler bet like it's just like whether you can get data of that domain and if you can get it like then you can for sure that you can fit it last question in this research area is there a specific scientific Insight that or bet that Co covariant has made or should we think of this as no not at all trivial but a full stack play with the right people very well prepared um engineers and scientists doing the relevant data collection that doesn't exist today that will support increased robotic intelligence versus let's say like a architectural bed or whatever it is yeah it's like the architecture has changed like maybe five times already like like it has gone through like significant transformation like every year like I don't I don't think you can be married reach to any single specific architecture in a field that is moving so quickly but there is one unique bet that we are placing right so um that one unique bet is we believe the future of Robotics would be built by whoever that has most robotics data right and and essentially the whole company is built around that thesis um and like you can say like an what is an alternative belief like an alternative belief would be can we just solely rely on simulation like we actually don't need much we will data like that would be a different philosophical bet like uh on it like we also use simulation but we think of simulation as more of a way to augment the data not as the way to replace everything there are lots of smart Tesla and ex Tesla people where Tesla has been a I guess big proponent of high quality simulation including for um you know training data generation right where are the gaps or why do you believe that's that's insufficient so when we think about simulation it's actually somewhat different for different kinds of autonomy domain like so when you think about simulation in self-driving car like we are really mostly thinking about systems that hopefully don't physically interact with each other right like if two cars get in contact with each other that's a really terrible thing right and so the simulation there is more about simulation of avoidance multi-agent behaviors like like avoidance of contact but if you think about like like manipulation like if you never contact something that's also a big problem like because like then you actually don't do any work um and whenever you involve contact simulation of those things become very very difficult like items that can deform like like the contact Dynamics is incredibly challenging and so those are where simulation becomes very difficult like it's when it involves contact complex Dynamics and then there's the second thing that makes simulation difficult is like I I mentioned earlier that it typical customers that we serve like may have 100,000 distinct objects in a warehouse like so like if you want to fully recreate that in your simulation like that is actually more work than just learning A system that can deal with um um the real W like so the vacation problem like in order to specify the real W in your simulation like that actually might require more data or more work uh uh or whatnot and that being said like we believe in learn role model like we believe in Foundation models that can learn from The Real World and you can simulate new scenarios of what would happen if you do things differently but that I think of that as like different from the classical simulation that I referred to earlier which is program program based and you are just um hardcoding the rules of reality and then building agents that learn from the mechanical interpretations ofu the rule of realities um that you encoded in your simulator so for our last couple minutes should we zoom out and talk a little bit about about the future yeah so you have said we're sort of pre-chat GPT for the robotics industry what is the chat GPT moment for robots what what do you imagine the chat gity moment for robots you want AI that is as general as chat gity like so you would be able to throw a robots into any arit new scenarios and they'll be able to learn how to deal with it very quickly um but in addition to that like which is kind of like what chbt allow people to experience this you can ask it OB problems like and then it can solve to some degree um um to you so you want the same kind of generality um but in addition to that what you also need is really high reliability um because like you really don't want robots that only suceed in like the task that you asked it to do 70% of the time and then there's like there might be 30% like really catastrophic outcomes um that come out come with it so I would say like the bar for the charity moment for robotics is higher like you you need to to solve the generality like which is the same kind of problem but you need to solve it with high level of reliability and this is like where like um one of this concept that we talk about earlier comes in like you really need large amount of high quality data to densely cover um um like this robotic Fields um um that you want and so that would be what I think about is the model side um of the chbd moment for Robotics and then you also need to think about the hardware portion of it right like even if you have a robot AI that is very smart unless you're just interacting with this robot AI in some metaverse digital 3D world uh you still need some Hardware body for robots and before humanoids are fully widespread uh I think we would see that the chaity of uh moment for robotics being articulated in the industrial settings earlier than in the commercial settings like because those are the places that can actually justify the hardware Investments because the hardware is being used 24/7 as opposed to like a home robots that might only be used 2 hours a week like that's a very different Roi from the hardware piece that you need to put in it what does the uh like Warehouse or factory or um logistic center of the future look like like lights out no humans I don't think it would be fully lights house and no human at least in the near future but I think of it as would be very robotics augmented like so um think of one person would be able to oversee 10 20 30 robots like so like like instead of like one person have to manually do all those work like you actually work with a fleet of robots like so think of a kind of as a physical co-pilot type of setup like you just get this like large amplification of like what a one person um can do but most likely it wouldn't be completely lights out like you would still have people there I think this form of um expression of AI like would probably be true not just for robotics but many other fields of AI as well I I realized you just said industrial applications first from an Roi perspective that makes sense but do you have a guess or a hope for what the first form or use case for intelligent robot that your average human like your consumer interacts with if I have to guess it probably would be a home robots that don't involve much manipulation so think of it as like a home robots that might be like a Roomba it can follow you around like you can talk to it so like it has that navigation of movement aspects of it but not necessarily the manipulation aspects of it like not actually manipulating the physical world around it I think that would be the most technologically feasible um um version so think of it as similar to Amazon's Astro robot like this kind like killed robot that has two wheels that can follow you around and someone calls it it can it can go there and so like I think that type of form factor uh would probably be like when we would see it earlier robotics AI work it triggers a lot of concern around safety in both like the short-term practical sense and in sort of the AGI breaking into the real world sense how do you think about safety at covariant uh we have a simple caral to this question like because we focus on Industrial applications uh and well all industrial robots like have a uh uh set of safety rules that they need to conform to like because it's not just AI can be dangerous like manual programming can be dangerous like you could make a you could program a robot to do dangerous things already and so there's a really robot sets of rules around you have to put safety cages around robots uh and if you have you don't have safety cages you need to have certain kinds of certified controller that make sure robot doesn't do anything that's dangerous um to the surrounding equipments people and so from that sense like because we're just following the same um r rules like any kinds of robots that we build and deploy are by definition safe um or by construction safe but that is very different from like when you say well what if we hook up like an arbitrarily expressive agent into a home robot that also has um like how do you limit that to be safe it's much harder like just similar to like if you just hook up a language agent to give it arbitrary python code execution capability and arbitrary ability to access the internet it just becomes very difficult to say well how can you make sure like it doesn't do anything dangerous and and that's where the alignment problem comes in and that that's where there's a lot of this good Safety Research um comes in but we have a simpler C like at least for the near term in this kind of industrial applications Peter what advancement in AI research or application outside of Robotics are you most personally interested in Looking Backward or looking forward looking forward I can only look forward I think the same kind of events that we have seen in in last year like we would see at least the same more um a of magnitude of them in the coming year like it's just if you look really behind like all these advances in large language models image Generations they are still using relatively primitive um technology like so like if you especially large language models like they are mostly still trained just on next token prediction like which in for people that study reinforcement learning we call it Behavior which means you're just asking the AI to clone the behavior of another agent and that is like one of the most primitive way possible to train this type of systems like because if you're just mimicking something like there's a natural ceiling on how good you can get on that and then there was just so many other proven two boxes that we have not deployed yet that like I would say like progress is guarantee like in everything that we have seen um so far and I'm so excited about that uh and I'm also super excited about the uh open source movement um continuing in the AI world like where a lot of these events make uh available to a broad set of communities that can continue to build on it and experiment with it um and so I think it will continue to be a very exciting year um of AI progress okay then looking looking backward and forward at the same time last question is your favorite sci-fi book with robots in it realistic or not it's not a book but I really like Westworld okay great Westworld the future comes Peter thank you so much for joining us on no priors until next time thanks find us on Twitter at no priors pod subscribe to our YouTube channel if you want to see our faces follow the show on Apple podcast Spotify or wherever you listen that way you get a new episode every week and sign up for emails or find transcripts for every episode at no- pri.com