LlamaIndex Webinar: What's next for AI after OpenAI Dev Day?

Key Takeaways

wait like 3 seconds and get started hey everyone uh welcome back to a special edition of The L index webinar series uh this is Jerry here uh and and today we're we're joined by uh Dan shipper founder and CEO of every every is a newsletter publishing long form essays at the intersection of business and Tech read by almost 100,000 Founders investors and operators in Tech and Dan also writes a weekly column on AI Al train of thought and so he's been very very active in the AI space I've Got to Know Dan in the past year and and it's been um an awesome set of interactions and so he was one of the first people I reached out to uh when you know open AI uh did their devb day on Monday uh we caught up at the event and I asked hey do you know do do you want to be on on my webinar on Friday to to talk about you know just like high level Trends um implications like impacts of of everything they released on Monday um and so we'll talk about that in just a bit and I think for the first like 10 to 15 15 minutes we'll go over uh some of the just updates from the Llama index side that we've shipped uh We've shipped actually a ton of different things we shipped like probably five to 10 different new features analyses and those things um and so we we'll go over that and and then we'll uh spend probably like a solid 40 45 minutes on on Q&A um if you have any questions on the chat I have a set of like questions I'm just going to ask but if you have any questions uh from the audience just please drop it in and then I will um I I'll just like inter leave that throughout the panel great let's get started SEC great so let's uh first Kick It Off by just talking about some of the things that we shipped in llama index uh post openi dday um we're happy to do probably like a workshop on this uh at some point just so that people understand like everything that we done but yeah the entire uh entry team has just been hard at work uh doing a lot of things here so a quick rundown through um some high level updates five days in uh since they released a bunch of features on Monday uh one is they updated their python at typescript library uh to V1 which ended up like the day of it it broke a bunch of things U but basically within a few hours we have uh full updated support for their new library so this is very table Stakes you know you can still keep using Lama index with with the new openi uh versions um some other thing is uh some analysis on the Json mode versus function calling um and and basically you know they release this thing called Json mode it seems pretty fancy how how like how is it actually different than function calling how does it work under the hood uh we support parallel function calling now for both agents as well as structured data extraction uh one of the big new things was this like hosted agent API like the assistant API and we basically wrap that as an agent abstraction and you can still plug in your own tools um there's a natural language uh we came up with this like fun natural language interface for building your own agents similar to uh how you build gpts by programming it with actual language uh we did some analysis of the gp24 context limit and then also came up with some multimodel abstractions uh LMS and Bings indexing and retrieval I'm just going to run through some basic screenshots of of some of these slides um this was pretty much all taken from uh our our Twitter accounts so so my personal account as well as the L index Twitter account so this is just uh yeah if you guys want like resources links feel free to feel free to give us a follow there okay um One initial thing we we took a look at is uh like okay what ex what exactly is is Json mode basically um they really sit and it seemed like a big deal but basically um uh Json mode is just a way to make sure that the output uh conforms to a Dron schema what it does not uh do is it does not adhere to any schema that you specify so you can tell to adhere to you know this specific like pantic class and it might not necessarily do that um but it will guarantee that the output is or roughly guarantee that the output is Jon um the difference with function calling is that um the way function calling works is you um import input some like you know argument spec and then it'll extract out like a function call which is basically structure representation function calling uses Json mode under the hood um I think there was a thread on uh open eyes blogs under the hood and if you just take a look uh at you know trying to do structure data extraction just with Json mode without the function calling part you might get back for instance like valid Json right here like here's like a call summary template but the actual schema U might not actually like the the actual extraction itself might not output uh some sort of like valid arguments right so here you know here is just a basic test we did here's like system prompt user you put in some transcript and then try to make sure that the type is like a Tron um it basically just regurgitates the schema that you specified and doesn't actually do extraction of values into the schema the next item here is parallel function calling um so one of this is another just like General kind of like life upgrade from open ey where you know instead of just like uh calling a single function at a time it can call multiple functions um the two use cases that we see this happening is one in agents and second in structure data extraction um so the first thing in agents is that you know now because they can pick multiple tools uh at a time like you can basically do more efficient processing Beyond um just sequential Chain of Thought for every single action um this is quite important because a lot of like the types of questions you want to ask might require kind of like batching up the the queries and executing them all at once um and this just leads to the faster more parallel execution so for instance like if you ask our openi agent what's the weather like in SF Tokyo and Paris um you know the openi agent basically is a function calling agent that just wraps the function poly API um you can now in one turn um get all three uh values or execute all three ques as opposed to doing it inal Loops um it's also useful for stru data extraction um there is this whole thing about you know if you define a pantic object you can pass it in as a schema to the function schema to open function calling and and and try to get it to do structure data extraction now you know you can basically more efficiently pass in a single object and get back an entire list of objects so you just do you know output class equals album allow multiple equals true and then you can just get back an entire set of albums um before you basically had to Define kind of this happy rap request which is like you know album list or something with a list of albums um and then that just wasted extra tokens and potentially cause opening eye to get confused so now it's it's a lot better that way um the next part is this assistant API which is basically kind of like a hosted agent execution U uh runtime like under under the hood um and we basically developed in a light like wrapper obstruction that integrates with the API agents um so you can use like your built-in retrieval tool or sorry you can use the built-in open a retrieval tool or their code interpreter tool but because the agent supports function calling um you can actually just plug in any sort of query tools you want from llama index uh we support a wide variety of tools uh from basically all the query engine stuff like around our our rag Concepts as well as all the tools available on llah HUB uh this ranges from stuff like Gmail Google Calendar um to uh like search apis uh database apis those things so you can actually bring your own like vector store to use with the assistant API um you just basically have to Define that as a tool um so instead of open eye calling its own reteval tool it just calls your tool and then you just handle the execution and the nice thing about L index is we just handle that for you so it integrates with pretty much all the vector databases that we have um there's been some general discussion we'll get into this during the Q&A of how good the recruit API actually is um so far the rough consensus is like it's it's okay it's decent it's not the best um but we we can talk about that um another General thing that we launched which is just like a fun notebook we did in pretty much like the first hour after the Thing released is open a release like gpts which allows you to program um custom chat gpts with natural language uh you can basically build this experience if you're a developer uh because the way gbts work is that you're it's basically just through natural language you're defi you're creating like another agent that does things and so the thing that like creates this agent can basically be this like meta or Builder agent um that for instance one sets the system prompt two uh create an agent or or two uh gets the set of relevant tools and three creates an agent um so we have like a fun cookbook here uh that allows you to um kind of uh go in and basically just like um uh like set this up yourself from the developer point of so that you can basically create like a 22 Builder like experience um I know there's no links in any of these slides but I I think I'll probably agregate I I'll I'll put them in into the slides before I send it out in the um video recording um and then we'll um uh it like all these updates are on our Twitter as well um the kind of like going through some of the last few items um an interesting analysis to be done was basically how good um is the GT4 turbo context limit um and specifically on just like very long context and also comparing it to stuff like clad uh I think there's a bunch of people on Twitter uh that have kind of like uh done some basic analysis ours is just like a very small portion of this and so I wouldn't like take the numbers for granted um I would just maybe take a look at the script see if it makes sense uh and then just like use it as just a single data point uh in your overall like evaluation um I think before this uh Greg uh Greg Ham did a cool analysis where he basically um looks at like like the needle and the hyack analysis uh to see if GT4 could recall context like anywhere in its context window we took that and just extended it to the summarization case where some sometimes you have documents are just like way longer than even the context window provided by j24 so the Uber 10K for instance has almost 300,000 tokens we inserted a bunch like just the specific like random fact of like my favorite snack into the different positions within this overall document and the thing is a lot of people are using LMS for just like large scale summarization these days and the way they typically do it is repeated sequential LM calls through these response synthesis strategies like um what we call create and refine as well as like tree summarize if you're in the L train world it's called map reduce uh they're roughly the thing like they've we've converged on kind of like rough like roughly the same set of strategies first trying to synthesize over longer amounts of context um and it's pretty interesting it kind of turns out like gb4 actually does better than quad on synthesizing um over like long documents and and actually finding the relevant information like the F my favorite snack is Hot Cheetos um in the middle like of this document somewhere through like repeated LM calls info tends to get lost and so basically what this means is that if you're trying to still use your D4 for like just massive summarization task you still might want to be a little bit careful in terms of how good it's able to recall a specific piece of information and the last thing that we just launched around uh 20 minutes ago uh is multimodel rag um and so we've been hard at work trying to think about the the um the right abstractions for for multimodal Rag and this is definitely still a work in progress uh but if you think about the combination the launch of like gbt for vision uh fuyu lava like and and a bunch of other kind of like Vision enabled language models that that are coming up these days it's there's a lot of exciting possibilities in just like taking some of the existing Concepts like Rag and adance and just making them multimodel um and so this has a lot of options like if you think about the inputs could be images or text if you think about the data representation of your knowledge Corpus it could be images or texts and you can index them via like native image embeddings like clip or you could index them via like text embeddings by summarizing the image um and then you can also uh you know retrieve images and textt and synthesize over images and text so a lot of different options we've developed like multimodel indexing and retrieval uh that integrates with most Vector databases and we'll be developing kind of more native uh image injection pipelines as well uh and then given a user query you can output retrieved images and text uh so we just launched this and there's a full blog post as well great I think that is a rough summary of everything we've done um and so uh maybe just the answer one question in the chat parallel processing is natively supported in LOM index without using open assistance API um that's a good question I think right now um uh do we actually I think I think the openi assistance API basically allows you to do like parallel function calling um and so you can just uh like get back a plan that is inherently parallel um you can technically replicate this experience in other llms as well by just simply prompting it um and we have like for instance we've done some query planning experiments by by having like the LM output some sort of just like Json spec that contains like a parallel plan uh or a parallel execution plan it doesn't work super well though and so the thing about openi like by doing parallel function calling is that because it's just baked into the API it's kind of fine-tuned to be able to do this well so that's just the one thing I'll say um and then the last thing I'll I'll answer is um so query for images makes sense but what do you mean by actually synthesizing over images uh it's a good question all I mean is just like being able to uh you know take in a retrieve set of text and images and at a basic level feeding that into j24b to generate text um of course like that's just a very limited uh uh current like scope I think you could imagine some multimodal model that takes in text in and imag and also text out and image out as well as there's a lot of these like text image models too so we're still kind of playing around with it and and we'll probably come up with more formal abstractions andj it great so that's just a presentation of the lendex update side and now let's turn it over to the discussion with Dan the fun part Okay cool so uh Dan thanks for thanks for coming on um the first thanks for having me yeah of course U the first thing maybe just to start with is I I love to for you to just uh do a quick intro about yourself and and your journey in AI like that the past year when did you got started like what what have you been working on because like I think um you know we've tried it a bunch of times but like you know it's it's been pretty clear to me you've been a very very early adopter of pretty much everything uh exciting that's happened uh in in Ai and and so would love to for you to share that with the audience yeah that's a great question um how did I get started um I think like a year and a half ago or something like that around I was I just was seeing and hearing Rumblings about gpt3 and was like playing around with the API a little bit and was like oh this is kind of cool but not you know I wasn't like fully hadn't fully dived into it um for for context like I've been running every for about three and a half years um as a CEO and as as a writer um and before that I was uh I I I ran and sold an Enterprise software company so my background is in is in starting startups um and then um you know did every as a kind of like uh media is a little bit of a pivot for me so I'm technical I I really like uh I really like building products but I also love writing and so that's how I that's how I sort of came to every I'd say about a year and a half ago started planing with gpt3 um and it really I think the AI stuff really became a significant interest for me around probably September August September October of last year um because we were at every we uh incubated this AI writing app called Lex um that my co-founder Nathan he was sort of building the first version of it and the the original conception of it was oh like maybe we should build a better Google Docs because um we spent a lot of time in Google Docs as a company um and there are lots of like little things about it that like we both thought and I think Nathan in particular thought like could be a lot better um and while we were doing that we were kind of like well it' be sort of interesting to see what would happen if you just like put put the put GB3 in there um and uh we tried it and it was like a real like wow moment when it could autocomplete what writing uh and this is you know sort of back when those those of things were like holy I've never seen anything like that before um and we launched it and it went really viral and um Lex has gone on to become like we spun it out as its own company and uh Nathan raised a a seed round for that from True Venture so it's like it's become this like big thing and I think in that process I was sort of I don't know I was just sort of Blown Away by the technology um I've been really obsessed with like note-taking and tools for thought and um a lot of this like software as a creative tool like a lot of the stuff that um I think AI is like really really good for is like interest that I have and and have pursued for a long time and I think that there's this like it's a new paradigm that allows you to like make progress in those areas or or or try different things in those areas that like weren't possible before and so um I just over the last year have done like like OB obsessive amounts of weekend projects or side projects where I'm like building little chat Bots or like trying different tools and doing little experiments and I can go through all the different experiments but I think that's one of the things I love most about this age is it's so good for tinkerers um where you can spend a couple hours and do something like really really cool that like no one's done before um and so every time there's something like Dev day um I just get really excited about it to try different things great that's a great segue into the next question which is um so yeah that's um given kind of the fact that you've kind of been in the a space for for over a year and and built like all these different things and and uh kept like track of all the updates that kind of um the the LM space putting out uh did devday exceed your expectations or or fall short and if the former like what was the thing that excited you the most if the latter like were there things that you're hoping to see or or was it Mix A B it's a good question so yeah so I said I I was there on Monday I wrote a long piece about it um which I just put in the chat um two days ago and then I wrote another piece that just came out today that's sort of another reflection on just generally what's what's been going on for me I think like um being there it was truly electric like uh I've never been to a developer conference that had that level of energy and excitement like it felt like the Apple iPhone keynote or whatever and so so it just like um I think it was really easy to get swept away and I definitely was swept away while I was there it was amazing I think like reflecting on it now with a little bit of the benefit of hindsight and having written that piece like where where I came to is um I think all the updates are pretty exciting but they're all basically like incremental to like the paradigms that that opena has been like working through for the last like year or so like um you know it's bigger context window it's like um better instruction following and like outputting you know Json or like retrieval like it's it's making a lot of the tasks that developers are already doing a little bit easier um I think the like maybe some of the things that feel a little bit more net new are um uh the assistance API which feels like it's sort of building this um new framework to allow you to make agents easily and and right now I think the agent functionality is still pretty basic but you could imagine that being like much more comp complicated and Powerful over time and then I think the other one is like they're taking this they're taking another bite at the Apple at um uh at building an app store um they tried it with plugins did not work um and I think this is this is the next phase and I'm like super curious to see how that goes um so I was excited I mean I feel like I feel like they're delivering the things that people want which is always good um it's not none of it seems like so groundbreaking that like you couldn't have predicted they would do something like this um but they're they are consistently it seems delivering value um I think like probably some of the stuff they shipped like you can tell there are some rough edges you know you can you watch people on on Twitter right now like um you know like when you they have the custom you can build custom gpts and like people have released their custom gpts and you can literally anyone can ask the GPT can you send me the data set that was like uh that was used to generate this GPT and it will happily just give give you the data and obviously like that's just a rough Edge that you know uh probably you would expect them to have fixed but I think they're just they're just trying to move like really fast and see what works and I think that's that's also that's also commendable yeah it makes a ton of sense I was I was there in person too and it did kind of feel like an Apple event for for like the the marketing the the way they promoted things it felt very polished and and I would say I think that that was the key part in getting people excited even if there were like rough and and we'll talk about that um so starting with j24 turbo so 128k context a lot cheaper curious about your thoughts uh you know like from the developer side like obviously I can kind of maybe give some thoughts as to you know how this can benefit a developer experience maybe for the average user like what what do you think is the kind of implication of these expanding context Windows um I think like when it comes to something like chat BT like like let me thinking just about my dad he's like um he doesn't really get the whole like I don't know why I can't upload like a bunch of documents and like I think a lot of people of his sort of cohort like who are a little bit non-technical but are interested in this kind of thing like maybe they're used to using CLA because they're like oh I can just put everything in CLA um but I think like expanded context window just like really reduces the level of friction that non-technical people have in like using these these types of systems so that I think that's actually quite quite a good thing um and I think in general it opens up even for for Builders or you know people who are technical it opens up a lot of use cases for LMS because my feeling for the last year or so has been um to some degree the bottleneck for good inference results isn't isn't always like how good the model is it's actually um how good you are at uh doing retrieval like how how well you can find pieces of information that you're using to like do completions on um and um and obviously as you know like retrieval is a complicated question there's so many things that you can do right or wrong in it um and there's so many different like use cases or different types of questions that require different methods to like get better at retrieval and I think one thing that expanded context Windows sort of Promise is that retrieval isn't as important um or it it does you don't have to get as complicated as quickly um you can just do something like pretty rough and just throw something in there and like you might get you might get a decent result even even for long context um but as you as you noted in your presentation like it seems that like the way that models are currently structured when you get really really really long pieces of context um uh the attention layers like have a have a tough time with it it doesn't always work um and so I think like longer retrieval is not this Panacea that just like eliminates all those problems but I think it does like it does uh make it so that the mo even your most basic calls to the ls like have a higher likelihood of working yeah that's a great point I I think um there there's kind of like the current state as well as like a promise of like future state of of what like just longer context windows and cheaper costs like entail I think right now there's um still some rough edges with like you know it might not recall everything in the context what I know and so therefore I might forget things or like it's still pretty expensive like stuff like 120k tokens um but I think you actually make a really good point in that like the way I think about it is when you think about the performance of an L map there's like some burden on retrieval and there's some burden on the model itself and right now there's actually um quite a bit of impact in investing in better retrieval because of limited context Windows um whereas like if you you imagine like you have like you know a few hundred thousand to over a million tokens and context window and everything is just super cheap and fast to run then like the basic retrieval if you were to do retrieval that you do could just be something like just set top K to like 200 right and and like the the like fancy retrieval algorithms might not actually matter as much if you can just like throw arbitrary things into the cont window in synthesize so that's a very that's a very interesting point um and that's definitely something to keep an eye on as to kind of like how some of the basic use cases evolve and just like kind of get abstracted Way by just like doing some basic retrieval as the model complexity gets gets higher um so the the next question I was gonna gonna ask is um okay well I mean speaking of which um retrieval retrieval API so open a released uh a retrieval API um obviously you know um I think on Twitter on on on Monday uh people people were saying like oh you know it's like all the Frameworks dead like all the all the like Factor DBS dead too like you know they have this thing now I think by all accounts it work works okay but it's not like um like lifechanging uh but it'll probably get better over time and I'm curious to get your thoughts on maybe like both from a business as well as like product side like is this kind of like a feature you see where L and providers just start like hosting the storage uh like and just like kind of monetizing that piece too um and and do you just see like every L and start investing in just like that retrieval uh like as an addition to their services yeah like I said like because the retrieval that you do significantly affects the quality of the output that you get I think it's like very squarely within open ai's interest to make great retrieval and I think that they are um well positioned to do good retrieval because there's good retrieval often requires many different round trips between the um the like vector store and the you know Vector search function functionality and and and all that logic and the llm and having this having like a tight integration where you both control the LM and you control the refal mechanism like can create good um good results I think there are there are downsides to like a tightly integrated system to that like that um in particular obviously like it's it's hyper model dependent um you're locking to an ecosystem um and I think over time um there's there's a lot of room for modular more open sour open source players to do well but I think for a little while um uh it seems likely that open AI like looks at this as like a important part of what they do and will start to start to integrate it more deeply and make it more complicated and make it less basic and less blackbox and all that kind of stuff um and I think that will like increase the uh power of your ability to use stuff within the open open a ecosystem which is within their interest whether or not like other model providers do the same thing like anthropic or like like you know open source models like like llama or whatever for meta like I don't I don't know about that I know I I think I it's I feel pretty safe to say that like open AI will do more of more retrieval I don't know about other model providers interesting so some follow questions is uh one is do you think basically there's going to be like a few different Stacks that emerge where like basically um you're going to have kind of stuff that is pure vendor lock in like everything just lives within kind of like a closed ecosystem um how do you think about like the competitive landscape with for instance the rise proliferation of like open source models itself and basically this it's kind of like just a general mindset of developers like hobbling together uh various components as opposed to just like using stuff uh with within like one ecosystem um almost kind of similar to like I'm thinking about a lot of just like cloud cloud Bas Services right like you just do everything in Azure or you can like like use like different SAS products and and piece together um and I'm curious how you think about that uh and then and then the other thing is um uh like Curious to get your your thoughts on like the competitive positioning of of this versus say like other players in the St like like especially Vector databases like how how does this like complement um compete with those well so I'm actually very interested in your take on this because I think I think you're you're probably better position to like analyze the Dynamics of the market than I am but I can I can take like a rough stab but I should say like my quality my my confidence interval here is like fairly low like I could be wrong um but I think in general one framework by which to evaluate this is that um uh from a competitive landscape perspective um when markets are new and um performance is the the most important thing for people um having an integrated player that everything they're doing everything together uh is likely to win first um because in like integration allows you to like optimize performance when things are new and things aren't standardized then like you don't really know what's going on like um but over time as things start to get figured out and you sort of like start to Asm toote on the like on the performance uh uh dimension and you start to you start to uh get um more other things are more important like flexibility cost security all that kind of stuff it becomes more uh appealing to use more modular players that are optimized for like one specific thing and you can kind of like piece them together for your specific use case um and so I think there's probably a lot of room for both I think that open ey will probably like be fairly dominant for a while but as things start to catch up and people start to prioritize cost and flexibility and and all that kind of stuff um modular players um and particular like open source players and open source components I think we have like an important role to play yeah it's a good point I think high level I agree with you in that like um if you have uh just everything integrated and it's just a better experience and that you're going to have an advantage um I think maybe the one caveat here is that I think a lot of companies have tried posted retrieval um and given the form factor of retrieval the current retrieval API it's basically just openingi is just trying to solve like retrieval on its own without trying to do some like fancy new architecture and I think the issue that I think a lot of people are running into is that it's actually quite hard to make it work well uh and and make it generalized across variety use cases and so because it's I would say that the quality of the refal API isn't actually quite at the level of like the magic of like trbt right it's actually way below just like you know just like the experience of like using the the LM models on its own then I think you get less of the integration Advantage there and so I think the the issue with kind of building um any sort of manage retrieval right now or is that like people expect it to work better than what you can get just by using an existing search service or just by doing topk um and that that burden is actually pretty pretty high especially and and I think a lot of people who have been in the space are basically kind of building like somewhat custom retrieval Stacks depending on their data like custom parsers trunking you know hybrid search like put it like full tax search that type of stuff put it into a VOR database um and I think I think it's just like the lower level abstractions give you a bit more control um and I'm sure like eventually people will find out like some of the higher level abstractions that might need to happen in terms of retrieval but I say right now I don't think like open ey has quite solved that yet that makes sense another way of saying what I I think you're saying you tell me if I'm wrong is like retri because retrieval is not one thing it's a very very complicated thing that requires um that's that's very customized for like any for for whatever specific use case you have um having like an integrated player that sort of gives you like this is our retrieval um you may not get the performance you actually want like on what you really want is a highly customized modular architecture that lets you like push pieces in and out depending on what your use case is yeah I guess what I'm saying is yeah just like the integration of AG is is uh less um pronounced when like one of the components actually doesn't work as well and I think currently that the retrieval API doesn't actually solve like a lot of the current use cases that said the retrieval will probably get better like I'm fully aware of that and and to your argument it's kind of like um imagine a World War context Windows got to like a million or 10 million um and and um and and the cost just came down like cost came close to zero then then you know you just like who who cares if it's not like better than topk retrieval you just to like a thousand like every time you run the all and then it would probably roughly give you the right context for every question that you ask and so I think I think that is actually a pretty interesting feuture and I think what you you uh brought up like got me thinking about that yeah um totally maybe moving on to the next section is so um gpts uh you talked about it a little bit open a i released this thing kind of as their uh last uh feature of the presentation um and they really gpts with the promise to allow people to program agents with naal language um and and there's a a few parts of this question but maybe just like Curious get your thoughts like what have you already seen people build with with jbts like what are some cool use cases that you've seen it's a good question um I don't know off top of my head there's this guy uh his name is levels he was like one of the first guys to do like AI avatars like last year when that was like when that was hot and he does he runs this thing called Nomad list which is like a uh website and Community for people who are like being digal Nomads and he just like very quickly built a bot that was like um uh allow allows you to uh chat with the Nomad list data set to figure out okay like if I want to like live in Amsterdam as a digital Nomad like you know how much is it going to cost or like what's the best place to live for XYZ or whatever um so I think um I think higher level there are a lot of people out there that like run communities or like for whatever reason have like a small data set that's somewhat proprietary that um can like really easily right now just like hook that hook that up to to gbt and like release a little bot for it and I think that's like that's quite interesting I do think overall I'm not particularly bullish on the specific uh the current gbt form factor um um for the reason that I think um I think people only have room for like one or two chap outs in their lives and um the way that gbts are currently structured um you can use you know chat regular chat gbt and then if you want to use a specific G gbt you have to like go to this menu and like select it and move out of General chat gbt and then go into this other interface and I just think that in general um chat is one of those things where it's very hard to predict where you're going to go in um in a chat and having to decide actually I don't want the full power of Chach with all of its stuff I only want this very one specific thing and having to remember that when you're probably only going to use it like once or twice a week is like it's just really it's a really tough spell I think um that said I think um gbts uh as a if they if they make some changes to it are quite promising so like for example um if I could within any the context of any chat gbt chat instantiate a GB temporarily and then un un instantiate it really easily or have it have that be done automatically in the way that like it automatically does web browsing I think all of that would be quite helpful and powerful and like the the idea that I could like with one click upgrade the the power of my chat gbt with this this Niche little thing I think is super appealing and I would I would honestly pay for that um like it's the equivalent of like buying a book for chat gbt or buying a course for it um uh but I just think that they haven't quite figured it out yet yeah that makes ton of sense I think the having room for one to two trap bots in your life is a good quote might be the tagline for the for the video the so the like I think P I agree like the um the interesting thing about gbts is it's like expecting people to almost like program agent to get it to work well for you um and I think I I've said this to you kind of like offline but basically like it's an interesting interface where I feel like the developers are very used to this idea of you know trying to program something and getting it to work well but for the average consumer they probably don't really know what that means and so I was as I was looking at the interface where you know you like effectively what you're doing is you're like setting the system prompt and you're like getting the set of tools and those things and I wonder if that experience currently is as intuitive to actually for the regular user to create something through natural language versus as you said like if everything's just centralized on a single interface instead of you having to like maintain a bunch of different trap Bots and actually trying to understand what does that even mean versus just like having a personalized agent that you can like have a sensal interface and do different things through um and so I think it's a ux it's like partially a ux experience as well as like a a technical technical problem to just like see how how how do you actually yeah like provide this experience of program something and I I think like that's one of the really interesting lessons of this whole era is that um the ux makes a significant difference um even when the underlying technology is the same like gbd3 had been out for a long time before Chach key came out and the like significant difference between GB3 and chat gbt is that chat gbt just made it really easy to use and and get the power of GB3 out of it um and that's what like launched really the whole like AI boom that's happening right now um and it it wasn't about any sort of at that point fundamental advance in the technology because it had been around for a while it was about packaging it in a way that people could use it um and I think there's probably something similar to be figured out here um it's like have to figure out the right packaging and they're like they're inching closer to it but it's not there yet um this is a related point but I think you mentioned offline um that openi and also in your blog post that openi might be kind of like almost cannibalizing its own developer Community a little bit like do do you see like it just through offering these like consumer package experiences um maybe curious to just get your thought a very general question do you see the role of like an AI engineer or or developer like uh like expanding or shrinking over time like is it that in the future like you just actually don't need developers in the middle um or you know the developers actually still play a crucial crucial role in this in this new world um I would be really hesitant to say like you don't need developers I just think we're so far from that from that future um I think we're more in a place where people who could never build something previously now it can build stuff and that's like really awesome um and people who are Developers already can get like 20 to 30% more efficient or maybe can like work on little projects that they like ordinarily wouldn't have time for because like they don't have to code all of it and I think that's really exciting I do think that openi like yeah there are they they are in ways somewhat at odds with some of their developer communities so like for example um there are some competitive Dynamics between tra BT and the API and in the sense that a lot of the things that developers want to build with the API are things that are going to be put into chat gbt um uh or are going to be put into the API so like you know they're to some degree competitive with a lot of infrastructure infrastructure players um and to they're are to some degree competitive with a lot of um people who are um who are trying to build consumer experiences because they're at overlaps witht functionality so I think that's like one interesting Dynamic that they're going to have to figure out um like they're caught between being in developer company and being a consumer company and then I think to uh I think there's this other thing going on um where to some degree yeah they're like commoditizing and consumerized being a developer um uh but I just uh I just think like we're really far from a few from a place where uh developers don't matter that it's just like maybe the day-to-day activities of some developers it turns out to be like a little bit different than it was before like rather than like writing every piece of Code by hand you're like reviewing code you're like acting more like an engineering manager in certain circumstances but like um it's yeah I think I think that's more what what we're looking at than you're not doing development work at all yeah I guess that that features still a little far off but but you know in the short term um yeah it it does seem like a lot of the the kind of like wrapper apps are things that eventually just fold back into TR um so I am kind of curious to see how that goes um and and there is uh there there's like really question I was going to ask there but maybe jumping to a question first in the audience um how do you feel the AI developments in a corporate environment all all um and and the question here is like uh like developing PowerPoint analyzing Excel sheets building code and a business environment like uh do you think it will be similarly AI supported for example in a year uh and a lot of that seems to be locked into the Microsoft e system right now um and they mention like co-pilot and also like office uh but yeah I'm curious curious to get your thoughts on um how these AI experiences will get integrated with these like Enterprise applications over time that's a great question I mean I just like it it seems really clear that Microsoft is like making a huge investment here and um and they've integrated into Edge and I I've seen Outlook Integrations that are that seem like pretty interesting for people I've seen word and PowerPoint Integrations so it's happening it's it's I think it's in I think it's in with the next version of Windows so it's it is happening um and I think um one of the things that we've seen is that uh like if you go back to slack versus Microsoft teams um in a world where uh larger companies have to choose between a product that has a great product experience and a product that um they just get for free and is sort of bundled into something they already use and is the guaranteed to be secure like they're going to go for that one um and I think uh Microsoft has a tremendous Advantage there because so many people use Windows and the Microsoft Suite that um uh they will be able to release it and um a lot of big companies will use it because that's the thing that that is safe and and like sort of already approved um and I do think that'll have that will certainly have have a massive impact um uh like it's a little hard to tell exactly like all the changes that will come from that but I I can I can see that being yeah a really big deal do you think um maybe just a TR question like valy would still AC or would all this value AC to Microsoft or is there still kind of like AI value ACR to to smaller players and by smaller players I there there's a wide spectrum of companies but below anything Microsoft grade like even like you know slack uh which is you know a big company Point all the way down to like the startup that's trying to build like a SAS like Enterprise AI SAS produ yeah I I mean I really I really believe in startups I really do and I really think that there's this whole narrative going on right now which is like you know uh AI enables incumbents and they can just sort of Bolt it bolt it on and so startups like aren't going to be able to compete in that in that world because like we're just going to have this super smart intelligent thing that is like bolted into all these apps and like that's just going to make them like like able to out compete any startup and I think that's just like big companies have in like lots and lots and lots of different pressures on them they're hard to move and they're hard to steer it's really hard for them to do risky things and they will always fumble the bag and um I think uh I think certainly it like there are going to be things that they can do like GitHub co-pilot or like in Microsoft Word or in Outlook or whatever that will like push things forward to some degree but um just for the very fact that they have to make something that works for like a 100 million people or you know 300 million people or whatever they're not going to be able to do things that uh someone who's sitting in their garage in m park is is willing to do they just can't risk it and I think those same those same Dynamics exist in in this AI world that we're in that have existed in every single other era of technology that no reason to believe that that's different um and therefore um I just think that there's a ton of opportunity I just don't think that we know what that is yet um because we we haven't like envisioned the new way of doing things with AI yet that like um that a big company can't do um yeah that's probably good point um next uh shifting to one of the major feuture releases in the up day was the release of D 24b API um and then uh I mean the the consumer app has been out for a while and I think a lot of people have already been playing around with it um I I'm curious like um have you seen like cool use cases for for multimodal that like are you starting to see some stuff like that or you were actually just telling me about about some of the stuff that you were doing and and I think for a lot of it actually would be cool to just like hear about the stuff you've done because I think people are still trying to figure out like what do I actually do with this thing yeah I use it for a lot of stuff I love love love um the multimodel stuff um so one thing I use it for is reading um so for example right now I'm reading m dick and oby Dick is like it's a book that um let me I have it right here like first of all it's really thick um and and and and the and it's like text wise it's like it's incredibly dense and the text is written in like sort of an old style of English where there's words and phrases and references that like I just don't get right and it's the kind of book that um ordinarily I honestly even though I would love to read it I would honestly need to like sign up for like an English class to read it because I just don't have the context to like get the most out of it but I'm just I'm not going to sign up for an English class to read M dick like I'm past that point in my life unfortunately um but the incredible thing about multimodal is as I'm reading I can just take a little picture and be like hey can you translate this to Modern English it get and it just like takes that like dense thing that I don't quite get and it just like renders it for me in a way that I totally understand um and once I do that a few times I sort of get the I get the text in my ear a little bit more and I don't have to do it as much because I like kind of have the foundation of like ok