Compositional ML and the Future of Software Development with Dillon Erb - #520

Key Takeaways

[Music] all right everyone i am here with my good friend dylan erb dylan is the ceo of paper space dylan welcome back to the twiml ai podcast awesome sam thanks for having me hey i am really looking forward to digging into our conversation it is just about actually just over a year since the last time we spoke we had a really good conversation on machine learning as a software engineering discipline and maybe we'll reflect a little bit on on that but before we do i'd love to have you kind of reintroduce yourself to our audience and maybe share a bit of an update on paper space and what you've been up to in the past year awesome thanks sam uh yeah so my name is dylan erd i'm the ceo and co-founder of paper space we are a cloud computing company that builds a suite of tools for machine learning developers that simplifies the process of training and deploying machine learning models we're based in new york uh and yeah i guess it's been a it's been a fun year uh since we last uh last chatted yeah so i i mentioned that conversation and it was uh one that we got a lot of great feedback on we we talked about this idea of machine learning this i guess was what was a a point in time where it was becoming very clear to folks that uh there was a shift for for many in thinking about machine learning as this experimental process or an exploratory process to one that required engineering rigor and discipline and we had a really good conversation about uh that idea but uh i wonder if you would share maybe your big takeaways or or recollections from that conversation what were the the key points for you yeah definitely um i you know i think that it's probably true in any space that's moving very quickly where you know the kind of underlying technology is is changing seemingly every week um but you know as as you know in the machine learning space in particular uh there's been a big conversation about you know questions such as does machine learning require its own special set of tools or can we reuse maybe existing tools from the software engineering world uh you know are there you know kind of special considerations for the users of these applications so is a data scientist a you know traditional software engineer or something different how do we bridge the gap between the kind of more standard machine learning programming languages like python or julia and the more traditional kind of software you know web web tools and programming languages like go and javascript and so you know i think it's moved very quickly and i would say even today it's it's shifting but i think it's undeniable that you know machine learning is very quickly making its way into um you know into the software engineering kind of discipline and and we're you know more importantly into a practice of like delivering machine learning models um so i saw a tweet this morning actually from uh srk and i'm paraphrasing it a little bit here but the idea was 2015 to 2016 image and vision uh someone added 2017 2018 transformers 2019 2020 nlp 2021 2022 ml ops and the the big question was 2023 2024 question mark um and uh you know they were soliciting thoughts on kind of what's next uh an idea that that we've been talking about uh that we'll kind of discuss more here you know could be uh the could be the thing that fills in that blank and this this idea of compositional machine learning uh we picked it around a couple times in prior conversations and uh you know maybe this is kind of a good entree to have you share a little bit about you know when you you think of this idea of compositional machine learning you know what is it where did it come from what were some of the inspirations uh that you've seen recently that uh started you started you thinking down this line yeah um i really like that uh that framing and it's funny how quickly all of those changes happened um you know i think there's there's also this whole question around foundational machine learning or foundational models um you know uh where are we in sort of the adoption curve i think you know an idea that we've been kicking around i know that you and i have talked about in the past is this you know or more recently around compositional ai and so for us you know we've we are um we're really at i would say um the beginning of a lot of folks's journey into machine learning so gradient our machine learning stack is um is used by you know at this point hundreds of thousands of data scientists and machine learning engineers for primarily a jupiter notebook product um uh similar to like a google colab or or you know kind of a web-based ide and so you know we've been really close to seeing folks you know kind of begin their journey um into machine learning and very rapidly we've started to see some some kind of interesting breakout cases of of how this of how machine learning has become sort of um you know composed or or remixed in a way that i think is just fascinating um you know a couple that come to mind that have really kind of sparked a an internal you know kind of dialogue for for us at paper space have been uh you know one is as you know this model uh first order motion model came out of nurips in in 2019 um and then earlier this year we had uh uh someone on our platform build a uh kind of viral funny lip-syncing app um that went from you know sort of academic paper a couple years ago to you know number one uh uh app in the app store you know in lots of countries worldwide um and so you know it's kind of interesting where you see you know maybe an app developer taking a machine learning model and applying it to something you know odd or interesting um the other one that i think has been really inspiring is you know we talk a lot about you know who who's the audience of this and is it software engineers is it data scientists is it mathematicians statisticians um and and i think it's actually you know gone expanded more quickly than we could have imagined so today you know one of the biggest audiences in the in the twitter sphere is uh you know artists and creators you know folks doing generative art and that's been precipitated largely by open ai's clip model which is a contrastive language image pre-training which basically was a model that was introduced that when you kind of remixed it with a couple of other generative models gives you the ability to kind of you know use a text-based input and generate you know really fantastical amazing art projects you know now this is making its way into nfts so so i think what's really interesting today is um you know whether whether or not some of these big models are foundational or essential or or whatever i think what we're seeing is um you know they're getting composed in interesting ways so the api is not necessarily a cloud-based api that people are consuming but really like you know taking these building blocks and reapplying them um so so i think this idea of compositional ai is something that we're you know it's it's kind of a framework that we're understanding how machine learning is moving past kind of this academic phase into you know kind of unexpected and interesting real world applications now do you do you draw inspiration for that from kind of the first wave of apis around the web but certainly especially when you use the term remix that was a term that we like to throw around i was trying to kind of mentally pin that in time and i don't really have it the i don't you know i have to research that but like there was this transition from kind of this old school way of thinking about integrating different applications like you know soa and web server xml web services that no one thinks about anymore uh you know to kind of like web 2.0 and rest apis when the like the essentially the bar for integration and remixing different services got dramatically lower to the point that i think you know it's almost not a specific thing anymore because it's just such an integrated part of the way we think about building new applications and services especially given the rise of cloud um you know i'm i'm i wonder you know what that experience and that context tells you about the way composition will evolve on the machine learning side yeah i mean i think um it's a it's a big question i think a lot of like you know a lot of smart folks are thinking about sort of what that looks like you know one of the you know i think the question for us is kind of what's the form factor there um you know if you think about sort of composable um portable encapsulated building blocks of any kind of software architecture um you know immediately the question becomes sort of how big are they you know uh what is the the kind of interface between them um you know machine learning has rapidly gone through a number of phases um and you know we've been around uh you know for i guess six six years now uh and sort of seen a number of these kind of rise and fall so the first was this kind of idea everyone's going to consume machine learning models through apis because you know that's how web developers are used to consuming things like you know clear bit scores or something like that you know a lot of companies kind of rose rose on that model companies like clarify doing really interesting vision work and making that available as apis then the big cloud providers followed with vision apis and the idea was that we would just kind of layer these into applications um i you know i i kind of even when that was really sort of the model that people were pushing it was clear that that wasn't going to be sufficient because you know folks want to build their own you know variant of these and the api model is kind of fundamentally limited so what we're you know another way of saying that is that the kind of the granularity had to get kind of smaller um you know there's been a lot of consternation earlier which was like hey these things are enormously computationally intense to to um actually create like very few companies can create models the size of gpt3 um arguably you know number you can you can count on your hand um and so this whole notion of kind of pre-training or refitting models was really sort of the i would say for the last five years kind of the the standard like no one's going to retrain the the kind of the first few layers of of imagenet they're going to you know train the last one um and now we're seeing you know i think even uh like a step beyond that which is you know in the case of these kind of um creative you know this creative art artistic community using clip what they're doing is they're using clip which is a you know a model that open ai released that they opened i did not release sort of the generator architecture on top of it so the community um has has kind of taken that and applied another generator called vqgan um and so it's not even just taking one model and changing the data set you're working on it's taking two models and then recomposing them in a really interesting way um so yeah so i i don't think we know exactly what the form factor is but you know when you know i'm i'm i run a company that that builds tools for this so we have to look for precedence um and so uh you know in our case one of the the main precedents that we've looked at is is looking at other kind of composable code architectures you know for example in github there's there's this kind of very large ecosystem of actions which are these kind of composable encapsulated building blocks that you can apply into your code repo that can do things like you know do code coverage or deploy your code or test it um or you know add additional functionality um and so as we've begun to develop more of our products and sort of you know build products that that respond to this compositional ai uh reality you know we're we're very heavily inspired by um you know things that have worked well and arguably um you know we have pretty good precedent for how how to compose you know code that comes from all sorts of uh you know different places in the world and and different um uh you know different sort of foundational i guess i don't say foundational models but different kind of foundational pieces i i think that's that is an interesting i don't know if it's a point as much as a discussion around what is the right granularity uh for delivering machine learning you know as you alluded to it's something that we've been talking about for a long time you you and i in particular and the community at large you know more broadly you know to what degree will you know models as a service be the primary delivery mechanism for uh you know your typical developer versus you know them needing the control that will or require them to have access to you know notebooks and infrastructure and the the entire end and experience uh so that they can customize uh what they're doing and i think this the idea that um the future is not just single models but kind of multiple models with you know that are trained in either some end-to-end way or fine-tuned in an end-to-end way or uh in a tightly coupled way does you know at some point the the permutations of models you know that people might want to kind of remix you know breaks the industry's ability to you know create wrapper services for different combinations and people just operate at a lower level it sounds like that's the you know what you what you're seeing or what you're you know the vision that is driving um your interest in this compositional idea yeah absolutely um you know i think there's there's you've heard a million times from you know companies and software developers in the space talking about you know end-to-end pipelines you know you explore some data you train a model you deploy it um i think that that that paradigm is has stuck around i think it's a it's a it's a good one for how you know data scientists and data engineers can you know work on a product pipeline that eventually ships into something interesting um i think that the idea of end to end is maybe a little bit um it oversimplifies it a bit because it kind of it sounds like it's sort of there's an input and then an output whereas you know what what i think we're really seeing is more combinatorial you know like there are many inputs and many outputs and it's you know fan in architecture fan out architecture um and very you know practically this has informed how we're building tools um you know our our main our most popular product is a jupiter notebook based product and for folks that are you know there's a obviously a large conversation around what what is the role of jupiter inside of the machine learning you know development process but you know one of the very fundamental limitations and i think there are a lot of benefits but one of the fundamental limitations is that it is really a you know a linear pipeline it starts at the top and it works its way through cells down to the bottom um and so fundamentally it's not really recomposable in a way that um you know that that facilitates or makes possible these more interesting applications um you know they're not as they're not as composable or portable so they're harder to share you know like they um they're hard to diff you know they're they're large json objects i think they're extremely useful um in some ways but um you know we've seen there's a there's a reason that folks have really been uh embracing sort of these pipelining tools that let you do kind of arbitrarily complex input outputs um you know and so that's where a lot of our thinking is today um is around sort of what is that form factor how do you go from maybe exploring something model or or data or um you know just code repo in a notebook and sort of an interactive rebel and then you know how does it get into a quote-unquote production or you know not even production how does it get into a more interesting kind of state after you've modified it um and so that's really i think informed a lot of our thinking because you know there's there's you know data code clearly is the main input on one side you want to create an app or a web service on the other side but but you know there's a lot of other pieces you pull in and we've been drawing you know a lot on ideas like continuous integration continuous deployment um composability uh you know pipelining uh dags and you know there's memes now about dags and yamls because i think what the industry is seeing is that we have to uh uh you know embrace kind of a more um flexible system for building out these new kind of composed machine learning applications now your your comments on notebooks is calling to mind another kind of data science internet i don't know if it's a meme or a feud or drama or whatever uh but you know maybe just put it as like you know there are different opinions uh on the role of notebooks or the appropriateness of notebooks uh probably best characterized by uh joel bruce on one side you know in his i don't like notebooks talk and then jeremy howard on the other side you know with his i like notebooks talk and i think that um [Music] you know that that contrast is is there are folks that have you know those positions and tried to operationalize them in different ways so like you know typically the i don't like notebooks camp well they just don't use notebooks and they use traditional code and traditional code artifacts you know repos and containers and you know productionalize their projects just not using notebooks on the other side you know there's folks uh like you know jeremy and fast ai uh but also you know netflix i think is kind of famous for this and some companies which spun out of netflix uh for and i believe airbnb was trying to do this for a while i don't know the more recent status of this but trying to take the notebook and turn it into a production artifact um you know either through you know some kind of decorators or annotators or things like that that allow you to specify within the notebook hey this is the code that needs to be exposed uh or other mechanisms um it sounds like you you started it what's interesting i think in this conversation is you started with a notebook service that was popular uh but then took this you know traditional engineering code artifact route as opposed to leaning into the notebook um you know yeah here's the thinking there yeah also um you know for folks that are listening that aren't familiar with paper space you know we originally started more as an infrastructure as a service company focused on gpus um and so we are we're interesting in that you know we kind of in many ways grew up with this machine learning developer audience and kind of watched what they were doing so you know the first very first offering we provided was something called machine learning in a box which was you know basically a virtual machine template with all sorts of dependencies kind of pre-installed that we spent countless hours you know fine-tuning to make it work this was before really containerization had taken off we worked very you know early on we worked very closely with jeremy at fast ai um you know we've been very fortunate to i i think um i i don't know the you know where where it falls at you know from all the numbers but we we've trained a lot of folks in the in the fasta universe um or onboarded them into machine learning and deep learning more broadly through the notebook product but you know notebooks we we kind of formalized because it was a pattern we saw everyone doing they would create a virtual machine and then they would install jupyter and then they would you know run a web service and put a public ip on it um so you know we we kind of formalized that pattern um and that became gradient notebooks um it's so for us we with that background we've kind of had two you know we have sort of this beginners using notebooks at the same time we are running you know large gpu infrastructure large clusters we've worked with a handful of much much larger kind of very advanced practitioners on on doing kind of production deployments really and so for us it was really you know i i think it's the question of how do you bridge those two worlds i think notebooks are um you know really wonderful for onboarding people into uh complex code and data concepts um you know i don't know if it's a forever thing you know i think it's you know when we describe gradient notebooks today we talk about it more as it's a it's a web-based jupyter notebook and ide um so you know you can bring in uh python code and the ml code and um you know other kind of supporting bits as well so it looks more like maybe a vs code than than uh sort of a standalone jupyter interface um but fundamentally you know we've been very interested in how do you go from a notebook into you know how like a notebook is like you're kind of building your your idea or conviction around something and how do you take that and make something more out of it um and so you know that's that's i think the area that a lot of folks are thinking about um it's interesting you mentioned kind of decorators and and patterns that have kind of been introduced for turning a notebook into a you know runable python file there was a you know paper mill um is the netflix project that is very very popular um you know there are other interesting ones like streamlit which are kind of um i don't even know how to describe them sort of a combination of an interactive notebook and a deployed process and yeah i mean i think that um that's sort of like right now the question is how do you take this audience i mean in our case very practically we have a lot of folks that are sort of maybe growing out of jupiter notebooks and how do we give them sort of a more composable uh path or or you know easier path into promoting what they're building or maybe even thinking of like larger possibilities because they can bring in you know easier more shareable composable building blocks um so yeah i mean i think uh i don't think no books are going anywhere um and i and i think they're enormously useful um but you know i don't think they're exclusively the form factor and so you know that's why we're all kind of working hard to find sort of uh uh you know the next step here and you're uh [Music] the the direction that you're betting on is you know the the memed dag yes uh the very meme to dag um i mean yeah the machine learning memes have gotten pretty pretty good in the last year um so i don't know where that puts us on the uh the hype cycle um but uh but yeah so you know we uh one that i shared yesterday on twitter which i thought was really funny was like a movie poster it was like from the creators of untitled dot ipoinb and untitled parentheses one dot ipymb is untitled parentheses 2 dot ipy yep yep yep uh spot on um you know it's uh yeah i mean well that's actually you know practically jupiter notebooks are really hard to version we actually had an internal tool that we're hopefully will release one day but um that we call mbdiff which is our diffing tool for notebooks just because we had to do it um and we ended up running into lots of kind of weird issues because um they're you know they're just they're not they're not really the format is odd people can annotate it you know collab can add different metadata annotations the spec changes a bit um you know we can add annotations and it's just it's it's like hard to diff and and sort of there's different inputs and outputs there's you know jupiter widgets which are sort of these uh special collaborations between server and client side they're just they're really weird um for if you from a traditional code perspective um so you know i think we have to move towards a direction that looks more like um more more like traditional software engineering and that was my you know big pitch a year ago i stand by it we have you know we we believe really strongly that um jupiter has a place uh for sure but but you know to to kind of move the next step we have to um start thinking of you know drawing from known best practices in in the uh you know kind of software engineering world and and one of those is you know uh i wouldn't even call them dags necessarily i mean they certainly you know uh directed acyclic graphs uh but but um yeah i mean you need to start you know introducing kind of pipelining uh syntax and semantics and and kind of those primitives you know for us um we're uh we're actually just about to by time this airs we will have rolled out um workflows which is really like our most ambitious project and also um kind of our most comprehensive which is uh really an automation um and build system for machine learning applications that allows you to um you know really tightly couple it to source control so you know to your point on uh kind of untitled one and two um you know that's not sustainable um but you know sort of add a few lines of code into a repo um and begin to turn that into a kind of a composable building block that could be consumed by other people um and you know like i mentioned earlier this is heavily inspired by github actions um and and tools like that um and sort of blended with uh you know kind of the data pipelining tools such as um you know airflow or you know we're using argo which is kind of a containerized kubernetes system but yeah i mean i think this is where it has to go um so you know i don't think we're leaving notebooks behind but we you know they're insufficient to take us to i think where we want to go as an industry so your your infrastructure um you use kubernetes under the covers uh in a lot of places correct me if i'm wrong but i believe that yeah we're a big kubernetes yeah uh and you're you know you selected argo as the workflow engine which kubeflow does why not just like create cube flow as a service other folks have have gone that route why kind of build it from scratch yeah that's a great question i mean i think kubeflow amazing project in lots of ways i think it's this is my opinion in my opinion alone i think it it kind of struggles to to to uh to match sort of the um the audience where it is today um i think it's it's hard to set up i think um you know the building out sort of the um the the kubeflow actions effectively i think is um still a bit difficult so it requires just more software engineering work so very large companies um you know i don't know spotify can can use kubeflow because they can invest in that ecosystem i think um you know there it's it's not a pattern that um will be as extensible for the kind of wide adoption that i foresee um and so you know what we've done with workflows which is our kind of newest addition to gradient um is is kind of take the best of of kubeflow and argo which is you know containerization um you know sort of uh the ability to create these you know complex dags with triggers and and sort of the fundamental pieces but expose it in a way that is much more uh kind of akin to folks that are building you know for example we took a lot of inspiration from tools like netlify and versal which are these web tools that basically let you come in attach a repo to to their service and then it you know basically creates a build system and gives you a website at the end um with just clicking a few buttons and i think that's the form factor we need and today you know workflows when you when you onboard you basically it's a very similar process give me a repo or pick one of a sample repo it's going to give a little bit of code in a workflow.yaml file um although that's kind of abstracted away and and i think we're going to move quickly to a point where the yaml is really an implementation detail and folks will be you know i don't know if it's a full low code no code because i don't know how quickly we get there but um you know the composability is where we want to focus our energy and so you know i think hublot has solved a lot of interesting problems and and you know there are a handful of other folks in the data kind of the data flow space that have worked on this as well folks coming from the um the jupiter world so there's like these tools like plumbers a really interesting one that i've been looking at recently there's one called kale for kubernetes which lets you sort of build out building blocks um uh from a notebook and make them deployable but we're coming at it from the other direction which is like what are what's the you know what tools are common in the software engineers tool belt and how do we make this machine learning thing look a lot more like that so for us the inspiration is um it's not starting at jupiter notebooks it's starting at you know jenkins circle ci versal netlify github actions things that are like kind of like known known paradigms and known tool stacks or types of tools in uh you know for folks that are building production applications one of the things that i think is compelling from a user interface user experience kind of perspective about you know paper mail ask type of types of approaches is the idea that they start with the notebook and you know you you because the notebooks is an interesting and useful place for kind of the throwing stuff against the wall and seeing what sticks and like starting to you know shape it and um you know just kind of bootstrapping your thinking about the way to attack a problem uh and then you know the traditional approach is okay you do that you kind of bang stuff into shape and then you like pull it out into a python module into a text file um but the you know these other approaches allow you to like it's it's even easier in a sense and if you've already got that infrastructure in place you just kind of add your decorator or whatever and you know there's your your artifact um i think the question i'm trying to get to is like you know do you see a bridge between the worlds and what you've built with workflows where you know you're starting in this your your i don't think you're suggesting folks to not use notebooks to you know get started or to to experiment because they're useful for that is there a bridge from that to a dag based you know traditional system other than okay you know rip your stuff out of the notebook and put it into a code module and check it into github yeah uh that's a good one um i i don't know i mean i think that um that we're we're internally doing a lot of work on on thinking about that form factor like can you you know turn a cell into an action in our in our pipeline um can you sort of send one over um the i think that the form factor that we need to get to more quickly and this is actually where we spent more of our more of our energy with workflows is um kind of closer to what i would call maybe build packs or sample apps so for example you know i think what verson for folks not familiar it's a it's a a web hosting tool for building or a tool for building web applications very easily and it relies a lot on um you know things like for example create react app which is sort of a starter template that if you're making a website it's a really good place to kind of fork that one and start somewhere and so i think that you know the pattern we have for notebooks today is people fork a notebook and build something out you know clip and vqgan and then they run through it and they get some images i guess you know that form factor is is hard to you know directly turn into something like a create react app or a um uh you know i don't know a starter template of sorts but i do think hard is it the nature of notebooks that makes it hard yeah i mean it's it's largely because the the the you know the the large benefit of a notebook is that you get this interactive repel environment you know you type code you get a response very quickly um but that same you know the the fact that it is sort of embedding its outputs and its inputs um cells are not really ordered um you know they don't have any reference to the dependencies or the or the metadata um that's required to run the thing it's hard because it's a notebook it's hard because it's a notebook and so you know what i think well you know one of the things that we're bringing over more into workflows is the kind of interactive reple you know idea um like for example in circleci which is our build tool that we really like um for our web application um you know you can ssh into an instance which is the equivalent of kind of creating a rebel you kind of go in and you can start interacting with the real thing um it's just you're doing it at a at a you know a higher like a different level of granularity you know notebook is really like tightly connected to a single kernel and you're kind of you know going through um i think you know i i i don't think they're going anywhere i think they're extremely important components um i think that their importance will be we'll know more um i think in a few you know maybe even in the next year when we start seeing sort of how the the next set of killer applications um you know come to fruition and my guess is it's gonna look more like starter templates like create react app that you're forking um you know more like build systems and build packs um where you're kind of focusing your energy on picking different you know instead of dali you're picking vqgan um or instead of the first order motion model you're picking uh you know uh the new enhanced one that that snapchat just came out with um so so yeah i think that that's it's a big open question um but the good thing is we don't have to invent it from scratch we can follow um you know a good precedent and i think we should draw that and this goes back to sort of the last conversation i think we should draw primarily from the software engineering world because a lot of this has been resolved over the last say 25 years or whatever on how you how do you build scalable you know large production applications one of the things dylan that i've heard you say in this conversation and we we've talked previously is that this project is you know one of your or your paper space's most ambitious undertaking and um you know i'm curious what why that is like what makes it ambitious it sounds like you took argo and like built a web app around it you know you can make it you can you can reduce it or simplify it to sounding very simple you know what where's the complexity in the effort yeah uh great question i mean i um you know what i think for us it's we know that there are certain really well-known uh best practices so notebooks today whether we think they're going to exist for 10 years or not they're they're really practical useful um components in the machine learning you know developers tool belt um deployments on the other end which we also have a deployment service that were where um by the time this rolls out there will be sort of our next iteration of that that has been released but deployments are also relatively well understood at least from a web perspective i mean we can go into you know edge deployments and quantizing and pruning models and sort of the you know the complexity there but i think it's there there are less open questions really you know it's the question about the glue or the fabric that takes these kind of early you know um exploratory uh prototyping tools and lets them kind of transition into the um you know the the production world uh you know thing that that every software company in the mloft space is talking about end to end you know training to deployment r d to production um and i think that you know so that that inner fabric i think is really important um and there's no shortage of dag-based data flow data you know data tools and so for us it was very much pick you know have some principles on what we're picking as the foundational piece you know containerization we think is it you know we're making bets we're making bets on technology stacks you know we uh invested very heavily into kubernetes um and you know kubernetes is an amazing technology has actually i think been more complicated than some people thought um but i think is a you know it's a big it's a big bet for us is that that type of container orchestration layer um is one that we you know shouldn't try to solve differently for machine learning than we should other other areas the form factor though for how you compose them is very different um because the reality is the audience is different you know the folks that are that are i would say um that should be using machine learning in their day-to-day work if all things were created equal and it were very easy to do is massive it includes uh marketing people and statisticians and folks in the humanities and artists and you know in addition to software engineers and bi people and you know analysts and it really is and and medical project you know uh practitioners so it's pretty it's pretty expansive and so what that means is there's there i don't think there is a single form factor for everyone um you know i can see a world where uh there it you know there are zapier-like you know connections for um machine learning models to endpoints and there are apis that are consumable um just like there are today for web services but i think the question that you know when we think about it you know our audience is every software developer in the world building uh you know software applications and delivering those we believe that they are going to use machine learning as just a part of their tool belt a part of their stack so you know we have some guidance on how to how you know workflows should be designed um but it's it's a uh you know it is not there's no very obvious precedent for exactly how this should be done um and i think that when you take on a very large project like this um getting the form factor wrong especially for you know a software company um can be can be really dangerous um so you know we can get into sort of how product and development and management is done but i i you know it really is always i think even at all good companies um some percentage of like seeing where the world is today uh what are people building you know what are their problems and challenges uh and then the other 50 is you know what are going to be the challenges a year from now and given how quickly this space is moving and how many things that i think are just really amazing and unpredictable um you know it's uh it's it's i think you know ambitious to to try to give a a version of a future that is um uh you know so so up in the air right now yeah i think uh you you referenced this idea form factor you know multiple times through throughout this conversation and and you know what you're saying clearly is that it's not the it's not necessarily the like the engineering challenge of hooking up a workflow you know engine to a deployment system to this it's it's all of that you know there are there's existing software out there that you're taking advantage of you know not that that's easy right when you have a large distributed system it's always hard but it's it sounds like what you're saying is that the you know the challenge was more getting the user experience right uh and all that you know compared to you know your previous undertaking which was there's this well-known user experience a notebook how do we make it so that you know how do we make it easier to deploy that yeah there's a lot more risk in trying to figure out you know as you say kind of look into the future as in terms of what people will need to you know more easily kind of compose machine learning uh nai systems and then build a system that uses just these infrastructure primitives to make that easier to do yeah absolutely i think that is the challenge i don't think it's ours alone but i think you know what we're coming at it with is i think you know a somewhat unique perspective which is that we you know we have a um more of an infrastructure you know deep gpu and accelerator focus than probably most companies in this space at the same time you know we have created a tool that is used by uh you know probably more folks in this space than almost any other tool for kind of learning this for the first time and so we have these you know this kind of split audience of like beginners and advanced people and and i think that um that gives us an interesting perspective on how to how to bridge those but certainly it's not resolved and you know i can see a scenario where you come back in a year and we say yeah so you know actually you know we the yaml stuff was uh too hard to do and the audience didn't need it we you know we really had to make this say you know a wysiwyg a gui builder or something like that or you know more of a zapier or you know notebooks actually um you know are no longer as useful when people can can clone starter apps that do basically what they want to do anyway um including the deployment and the training and the and the you know inferencing logic um so yeah i think we'll see i mean i think look uh we're in a really exciting time i mean for software developers in particular uh you know i i just listened to the greg uh brockman version of your podcast which i really liked uh talking about codex and um copilot you know we these tools are being used today um in in the real world by you know like machine learning and assisted technologies are are real and they're being used by programmers by physicians by um you know a lot of folks and so you know at some level i think it's inevitable that this technology breaks out of the lab or whatever the analogy is um but um you know i think there's a race to figure out the form factor and i think the opportunity is massive because i think we're gonna see you know just like today totally unexpected applications um that are really inspiring you know i think it's it's amazing that we've been in the space you know uh for for it's relatively um short life life cycle um and just i continue to be amazed at what is being created and um you know what's possible um and that you know we could we could have a whole other hour of conversation about you know transformers and uh you know sort of what what what that what that has done for the space but um and we probably should but uh you you brought up the the interview with greg and codex and um you talked abstractly about that stuff being used but from our conversations i know it's not necessarily just abstract for you you've actually used it and there's some codex generated code in in paper yes yeah we'll have to uh i mean it's uh still you know we we were when it first came out we were kind of playing around and you know giving some comments like generate a function that uh you know does this simple task creates an array of of interesting names or whatever for sample projects and we actually do have a piece today which is totally ai generated that is in our production application um it's small it's you know bounded but it opens up the question you know if one whatever 0.001 of our code base is a i generate generated today um you know i'm curious what percentage that is a year from now or the next time we talk my guess is it's going to be more and so uh you know that's an exciting future for sure like this this stuff is not we're not talking abstractly about the power of machine learning to change your day-to-day it's actually doing it um you know that's this is also a very complicated topic i don't think engineer you know software engineers are gonna be out of jobs but um you know i think this this kind of radical ai assisted future uh is really exciting whether you're an artist a programmer um you know a media producer uh whatever it is like i think that um that's why this space is so exciting and that's why i think you know um we we care so much about trying to find the right form factor ux sort of the the way that we can assist and um you know helping build more amazing applications like you know kind of breaking out of the kind of meme culture and getting into like what what are you know what are builders building kind of thing awesome awesome well dylan uh always a pleasure to catch up with you thanks so much for the update and uh looking forward to next time awesome thanks sam take care thank you you