Genie: Generative Interactive Environments with Ashley Edwards - 696

Key Takeaways

typically as like reinforcement learning researchers we often get a little bit annoyed with how we have to bring in more and more environments um like typically if you want to train an agent you're like okay well maybe I can bring this in or bring this in or but this doesn't really scale and I would say nowadays you know we're we're interested in generalist agents and if we want generalist reinforcement learning agents then like where do the environments come from so one of the motivations behind that work was how can we basically learn an unlimited source of environments of training agents it gives you um the sort of intera ability that you would have in an RL environment without actually having to place the agent within the environment [Music] itself all right everyone welcome to another episode of the twiml AI podcast I am your host Sam charington today I'm joined by Ashley Edwards Ashley is currently a member of technical staff at Runway ml which she joined just a few weeks ago after several years at Google deepmind before we get going be sure to take a moment to hit that subscribe button wherever you're listening to Today's Show Ashley welcome to the podcast hi thank you it's nice to be here I'm excited for our conversation we are going to dig into your work in video generation and the genie environment in particular particular but generally I'm really excited to have you on the show we've been following one another and interacting on Twitter for quite a while and I'm kind of amazed that we've not spoken yet on the podcast definitely welcome yes thanks for having me absolutely absolutely let's get started by having you share a little bit about your background yeah um so as you mentioned I'm currently at Runway I was at Deep Vine before that before that I was at Uber um but in general uh my research has been in doing reinforcement learning from videos uh and more recently I've sort of shifted into how we can actually do video generation itself give us an overview of the types of projects that you've been focused on um I mean I would say recently probably Genie um yeah um but I think in general I guess what people sort of know me for is doing imitation from videos um so often you when you're doing re or imitation learning you typically need aom ation of states and actions that tell you what to do but often it's kind of difficult to come up with those demonstrations it requires a lot of effort on on the demonstrator and so my research has been about how we can leverage videos to train agents so how we can learn actions policies um rewards for training reinforcement learning agents and that is one of the areas where Genie is unique because you're learning all of these actions uh in a totally unsupervised way yeah yeah exactly so why don't you share you know broadly you know Genie and kind of the motivation for that work so as I mentioned my my background is in reinforcement learning I've been working in that area for quite some time I think typically as like reinforcement learning researchers we often get a little bit annoyed with how we have to bring in more and more environments um like typically if you want to train an agent you're like okay well maybe I can bring this in or bring this in or but this doesn't really scale and I would say nowadays you know we're we're interested in generalist agents and if we want generalist reinforcement learning agents then like where do the environments come from so one of the motivations behind that work was how can we basically learn uh an unlimited source of environments for training agents so we can at least remove that sort of aspect of reinforcement learning where we're having to come up with the environments unless you're doing robotics I guess where you have the environment itself but yeah I think uh in general it gives you um the sort of interact ability that you would have in an RL environment without actually having to place the agent within the environment itself and so what's the the data source that you're learning these environments from in the paper we basically use two different kinds of um sources of data the main one would be videos of 2D platformer games and reinforcement learning we love training over games so we kind of looked into that um sort of environment so very diverse data set with lots of different actions and different agents moving around and that sort of thing and then the other one we have is like a robotics data set coming from uh I think it was like rt1 uh qtop that we we extracted the observations without actions from that environment and trained as well just to show that we can also use this approach on um real world environments as well and talk a little bit about the end result that uh you've created with Genie what do that allow you to do um I mean I think the thing that really excites me the most is that you can learn this sort of world model um without using any actions you can learn it completely unsupervised from videos um which is quite General you don't need many inductive uh biases here you just like give it a bunch of videos and it learns um a world model that you can interact with and so what we found was in particular when we were training over the 2D platformer games that we can take text generated images or uh sketches even or real world photos and we can sort of just step into those environments as if they were like a real game like we could just basically play it like it was like a real world model again even though we didn't use actions at all here or learn uh or I guess ground truth actions yeah what makes it a world model uh is it um you know what is that really speaking to is it a degree of consistency from you know between frames or you know how do you kind of capture the essence of that yeah um I would say I mean when I think of of the world model I really think of something that you can step into and interact with um so if you I mean if you remember I guess like Schmid Hoover and David ha's paper from a while ago on I think they did Doom uh they had this world model paper and then you have like you've learned a simulator of Doom um of course this is just a single environment um in our case we're looking for learning a more General World model um like basically a world model of 2D platformer games that again enables you to just like inter act and uh sort of through your actions you can change the environment essentially and so what are the major components of Genie uh so we have I guess three big components one is the latent action model where we're learning the actions from videos another is the the Dynamics model which we're actually using to um generate the future and then the other one is our video tokenizer so this is how we're actually learning the representations of our um videos which we can then use for predicting with our Dynamics model okay let's start with that latent action model uh it's super interesting that you're able to injest all these videos the videos have I guess generally I guess sometimes you see on YouTube videos like the you know twitch streamers and and the like will have like these labels that pop up that say like either what direction they've pressed on their joystick or you know keypad or whatever our keyboard for that matter but I didn't get the impression that most of your videos had that and you were somehow leveraging that kind of information it was more just playthrough videos is that right yeah yeah exactly I mean I'm sure I mean that sort of information could have been in there um but I would say it wasn't something that we were explicitly trying to extract um mainly what we were trying to figure out is can you learn a representation of what's going to change between the scenes that you can predict the next frame this is basically what our L inductions are are there a set of assumptions that you've made about about uh actions meaning like you know that there's going to be you know an up down left right kind of thing and that's your action vocabulary or is it you know more open-ended than that and and kind of fully learned end to end like I think one thing I would say is we we try to restrict the action space so that we can actually interact with it so basically representing our actions in a using a discrete code book using Vector quantization so one we assumed that we had a small set of actions which is definitely not the case but it just made it easier for us to interact with the models um the other thing that we sort of assumed was that there was a common there was a sort of common action space across the different environments that we were training on um and this was going to allow us to learn this sort of General General representation because we're learning over all of these environments we wanted to learn what was common so there must be different agents that are going to move to the left or move to the right or jump and we wanted to learn the representation that captured that are these three components the Laten action model the video token oer the Dynamics model are they each trained independently and then come together in the final system or are they trained uh in concert with one another yeah yeah good question um so we trained the tokenizer first and then we trained the latent action model and the Dynamics model at the same time although we're not using that Dynamics model for training the latent actions so we could actually also train the latent action model separately and then use it for the Dynamics model but we just trained those two at the same time I saw reference in the paper to spatiotemporal Transformers um tell us what that is and how it plays into Genie essentially what we need to do is we need a representation of our images so what we do is we we represent them through patches again using vector quanti and so each image is represented by a patch essentially so you have a sequence of images which is your video if we going to try to plug in each of those patches into our Transformer uh and basically have ATT tension over all of those different patches we would end up it would be very computationally um intractable uh and so what we end up doing is instead also we incorporate time that's a temporal part so the spatial part is um the image patches and the temporal part is time time but instead of having each patch attend to um uh each patch in the video what we can do is instead take a single patch and attend to it across time so instead of having this like quadratic blow up in um in resources what we instead have is like this linear linear scaling across time that makes me think of a couple of conversations that I've had recently one uh very recently I think we just published this interview with Albert goo talking about State space models and one of the main things that we talked about was how tokenizing for video and other kind of more native uh modalities non- language modalities like introduces artifacts that somehow make training models more complex and and uh make the use of attention more inefficient uh and I'm just curious you when you think about models that you could have used or might use in the future like how do you think about the different roles of uh alternatives to the spatio temporal Transformer yeah yeah that's a great question I mean I think so the context window of Genie I think was maybe like 16 frames so if I think if we wanted to like roll out or once we start looking into more um longer Generations I think it might make sense to actually look into these more efficient um approaches like they do in theate space models one thing that I find like it's interesting that we we sort of have to put our images into these patches just so that we can have tokens for the Transformer to predict it feels very I mean it works I think it works well but I I do think that there's probably other ways that we can represent images I mean also you have diffusion which you can do over Laten space rather than having to to use um to predict over patches necessarily so I I think in general when we when we built this model we were looking at okay this is a thing that's probably going to work this is a thing that's probably going to work this you know and we put those things together but now I think there's probably different ways that we can improve upon each of the different components that we had in the model in terms of making it more efficient and so how did you structure the um the training of the latent action model so that those actions were kind of pulled out of the training data let's start with just imagine we have two frames um so say you have two frames what you want to do is learn a representation but you're going to press such that if you take that representation in your previous frame You can predict the next frame so the latent action model gets to see both the past and the future but we compress it in such a way that you can't actually extract the information about the state itself you really just have to try to extract the change that occurs between states so you can use it for predicting the future so the way we compress it is we essentially we place it through a bottleneck and then we put it through this PQ module um which says we have like tin codes and we're trying to basically learn make our our uh representations be close to those codes that we're also learning at the same time and know those 10 codes the like the size of your your action space like you're you can represent 10 possible actions yeah exactly okay okay um one other thing that I that I forgot to mention is that well I talked about it in terms of two frames but you can actually use all of your previous frames up to that point and then use that um as your context for your latent action model but you only typically we only see a single next frame for it and what are some examples of actions Beyond up down left right jump that are common enough across games that um you know that will fill out the the rest of this 10 uh yeah I mean I think one thing that I that we that we saw was that it would learn different I guess velocities for the character so it wouldn't just be like move left it would be like move left quickly move left you know with this amount of speed also we there was like explosions and score change uh lighting you know yeah some of them actually learned to like change the score which was quite cool when it's changing the score is the model essentially just changing pixels or is that something else yeah okay yeah I think well when I say score probably more like Health meter like if if something gets hit then you see like its Health go down essentially um which was yeah so that that's it's all in pixel space so yeah yeah yeah so you train this model you you've got this latent action model and then you are generating a video is there anything that or you're generating an environment is there anything that Maps the action model to that particular environment or is it you know always you know code four is up code three is you know fast left whatever yeah great question surprisingly it does learn this sort of consistent mapping across different inputs so yeah latent action zero might always mean move to the left Mo right might always be latent Action Four we didn't do anything to actually ground um the actions space in this but I think it's just through the fact that the model needs to learn this representation um in order to predict the future and it's easier for it to learn it across um learn the same thing across different environments did you have any benchmarks uh that you're able to compare uh this Laten action model to or how did you think about uh and measure its performance the thing with interactive models is it's really hard to measure things like uh controllability but you can measure things like video quality so we're using like metrics like uh fbd for example to measure how how good the videos are we looked at a couple of different um represent presentations compared to using a spatial temporal Transformer so one of them was like the the SE vivit architecture which was used in faki and in general we found that because they don't do this like uh like spatial temporal trick where you only like attend to one token over time they actually have you have to attend over multiple frames so um it was quite compute uh intensive but that was like I think the main method we like architecture I think we compare to in general we were just like trying out different things in terms of how we scale um our batch size and model size and how that had an impact on things the lat action model as well I think we had some experiment so I'm probably going to have to look back at the paper to remember exactly what they were in terms of the video tokenizer and the approach to patching how custom does that need to be for you know a particular problem like is video uh like I guess maybe another way to ask a question is like why is that a major component okay yeah so I would say um one thing is that it enables us to predict um tokens with the Dynamics model um and because we were doing video generation if we are predicting pixels themselves um then we would end up getting blurry Generations so we would probably wouldn't want to use a convolutional neural network in this this case uh but because we put it into patches we can predict like a discrete patch so we use a vector quantization of that so we can predict a discrete patch which we can then plug in the in the decoder um to reconstruct the image rather than having to predict the image itself um so that led us get sharp predictions um from the model uh the other thing is that um the spatial temporal component of it was important because um what we found was that if we were tokenizing only in space then it might not represent um the sorts of changes that you need to represent when you're predicting the next frame like if we only using this for image generation then we might not need I mean obviously we wouldn't need the temp rol part because you don't have it um but because we're generating videos uh it's useful to actually have a an architecture that can capture changes across time um so I think those were the main reasons we looked into using this sort of um tokenization scheme uh I would say so the spatial temporal um model is what we ended up using in every component um so the video tokenizer actually wasn't we didn't use that in the latent action model we just used it in the decoder um but the spatial temporal architecture we used in the video tokenizer and the lat action model and the Dynamics model with that in mind like do you think that it's possible uh or it would be possible to have a single end to end model since you've used the same same underlying model for all three of these components do you see it as being possible to kind of collapse this down into a single endtoend train model at some point in the future and you know what do you think the biggest challenge is in doing that or is it like architecturally unlikely because of you know whatever reasons it's a a good question because it's something that we wanted to get working um throughout the project particularly I think the thing that was most like frustrating um or I mean it was annoying I guess was that our latent action model has its own decoder so it takes state next state goes into action model predicts the next state that decoder is different from our Dynamics model sounds a lot like the Dynam what the Dynamics model needs to do the Dynamics model yes it's taking the state latent action predicting the next state we could never we could never get that model to train our lat actions it was very difficult um I think I mean one thing that we're doing is we're training it in a different way uh so that model first of all takes in tokens um we found that training directly on pixels for the lat action model performed better than training on tokens for it um that model outputs tokens and we found that predicting pixels perform like help the lat action model perform better um so it might just be a difference in the kinds of losses that we had like uh the Dynamics model uses a cross entropy loss whereas the lat action model uses mean squared error loss um but I think uh it could maybe be possible to train um like a to a tokenizer at the same time as a late action model um but I I mean I'm sure there's a way to make it into to end we just we found the way that that that worked and I think NE The Next Step maybe would be to make it more elegant I you know I wish that it was in end but it's not got it got it and can you dig into a little bit more the the Dynamics model and you said the Dynamics model takes as input patches and output and out it outputs patches or pixels uh it outputs patches but it's predicting the tokens so basically what our the way our video tokenizer works is you get patches we place that through a VQ module which gives it a discret code for each of those kinds of patches um and then what we can do for our Dynamics model is now we're going to predict patches um but they're represented through tokens and so you can use a cross entropy loss over those um different patches um I guess one thing is that we're doing uh MK token prediction rather than um uh predicting all the Tok all the tokens at the same time at least during training during inference time of course you have to you have to predict all of the all of the tokens um yeah uh elaborate on that what do you mean by uh doing mask token prediction yeah so essentially what we do is we can uh mask some of our uh previous token so basically give it a value of zero so that you don't have to predict it um so it gets essentially gets to see is this in in time for a single kind of coordinate or is this like an infilling kind of thing in a where you're masking in 2D space right yeah so essentially it gets a sequence of tokens which represents its patches um does that part make sense so these are all of your FR so yeah a sequence of to of tokens representing Patches at multiple time steps or help me tune up my vocabulary here so or my the way I'm articulating this so you've got uh you know you've got multiple patches in a two-dimensional representation of a frame right yes yes and then you've got multiple frames in time um and so when you've get you get a sequence of patches are those is it like the same patch meaning representing the same part of an image uh across time or is it I see right so yeah okay so yes so single image let's say you have and how would you how would you say all that like what's the right way to to talk about like where the patch is situated in the image versus uh yeah that's a good question let's think about this okay so let's say you have an image this is my image by the way I keep doing this this has always been an image in my head um let's say you have like a 16 by 16 patch um so what you're going to try to do is you're going to say okay I'm going to go through this image and each of these parts are going to correspond to a patch you can then represent that as a sequence um so that that particular image is represented by a SE a sequence of of patches um so we have the I mentioned we have the spatial part and the the the temporal part so the spatial part gets to see um the spatial Transformer is just attending to across patches in a single image and the temporal part attends to a particular patch across time but it's only attending to that single patch that same patch time yeah um that's what I was asking yeah yeah it it gets yeah it definitely gets confusing and I I might have even explained some parts of it incorrectly but prior to this part um so the the main idea is that you can um mask out some of those tokens okay so you don't get to see all the context you maybe like Zer out some of those tokens um I think this is where I my question came in are you masking out like is it a random masking and it could be um you know different patches within an image you know from one wait okay so let take a step back are you yeah it's random masking yeah it is random masking and it is um in the spatial Dimension or temporal or both that we're talking about would say yeah it's just it's random masking in a particular uh in a Vector which is spatial and then you have multiple of these uh over time is that the right way to think yeah exactly yeah okay um so you get Mass tokens and then you're going to predict the next tokens um and we're using a technique known as masket during infer time inference time um so rather than predicting a single token one by one uh you predict them uh you predict them in parallel um so you like you have your IM my image again you unmas some tokens um you keep the ones that you're most confident about and then you condition your model again on those and then you predict the next tokens essentially um so this is maskit as compared to like I mean I guess what most people are using these days is diffusion and so you're predicting tokens based on the masked sequence of patches and that is then used to generate the kind of the t+1 frame of the video right yeah exactly so you can take those tokens plug them into your decoder from the tokenizer that we were training before um that will generate your image so we again we're predicting the tokens not the not the image itself and then we can reconstruct it I'm curious about the like the computational complexity of all this I guess the the thought that preceded that in my head was like did you publish code for this like can someone pull up a notebook and just do it and then the question was like what does that even mean like how much compute was required to do this yeah uh so no code however there is a there is a um a reproducibility uh section in the paper uh about how people someone could reproduce this on like um like a mid-range TPU over coin run data um so coin is like a reinforcement learning environment uh so we give instructions for how to yeah reproduce it in that setting we haven't um open sourced anything but there have been people who have released codes so yeah I encourage folks to check that out how playable are these interactive environments like do you you mentioned uh something about a 16 frame context that's relative to training once you've got the environment you could play that you know as long as you would like because you're just generating a next frame correct yeah yeah that's that's true and then I'm wondering how um you just kind of the noisiness of like Transformers and prediction and all this like how that manifests in in videos and playability like is it you could you can play it but you wouldn't want to like you know how how interesting is it I think it's interesting in the fact that once you see it you you can see that you can change the environment for like a couple of steps then I mean you yeah I I I would say that you would never at least with this this model you would never see like another character coming into the environment or um yeah interesting things don't really happen but I think it's still cool like we took my and that was not to take take it all away from what you've done it was really trying to get at like you know how well the a job the the model or the set of models do kind of maining maintaining consistency in the generation from you know one frame to the next over time and if that results in something that kind of feels playable or um you know do random things happen that uh you know make it uh make it you know less playable and reflect um a degree of uncontrollability in the generations yeah I would say like I mean the model the actions are able to remain consistent it does tend to I think sort of maybe overfit as you press the same action so for example if you've been taking action right it kind of over time I would say the actions might end up all going right but we found that there's a way there's like an action that would say okay stop doing that and like weet which is kind of weird but there's like a an action you can take that would let you start moving left again I I guess it's like maybe it's like a stay still and then change direction style thing um but the the thing is so like if you're moving right for a long time you basically generally just see a pattern of the background moving over and over and over again like you you keep seeing the same things you it's like I don't think I've ever seen anything um in like our outer distribution environments like I don't think I've ever seen like anything interesting pop up out of nowhere essentially I don't think I understand that why does the it sounded like you were saying the the semantics of an action changes over time or one of these action codes is that what you're saying and why would that happen I I thought we were saying earlier that they they're essentially you know static posttraining yeah so the actions like given the same or sorry given different prompt frames um taking lat action zero typically would move you to the left or right or whatever across different initial frames over time if you if you've played the the environment and you've been taking like the same action like certain things that might just have to do with like velocity like it it kind of is just like it's learned a model of how the world's going to work so if you're going up and you're continuing to go up it's very difficult for example to start going down or like take the action that's going to go down because the model has predicted and it's a I would say model of maybe the physics of the world that you can't suddenly start going down um so it might be the same thing like if you if you've been pressing the right action like it doesn't matter like it the latent action predictions get completely ignored because it knows that it's very rare that you would suddenly maybe start moving left um so it might just be a it's two things one it could be just ignoring because of the physics of the world one it could be ignoring because of the policies that it's trained with um maybe people typically don't suddenly go left when they've been going right for a while that still doesn't make sense to me and so there's something that is in there that uh that I don't understand so my my kind of you know mental framework of this is that you've like trained these components you've got this latent action model that you've trained that kind of pulls out the actions then you've got this Dynamics model that um predicts frames and you know once you're at playing you know it is you know you're giving it an action code to you're giving a previous frame or some set of frames and action code to oh I guess it's the previous frames if you yeah maybe another way to think about is that if you the Dynamics model is maybe balancing the the information it gets from the Laten action and from the previous frames and if you have too many previous frames that are saying one thing it kind of ignores the Laten action model yeah yeah yeah exactly it's the spatial temporal part of it again got it it's paying attention to yeah the previous time steps actions and frames that it's seen got it so then in theory one could tweak the Dynamics model so that it pays more attention to the latent actions relative to the prior frames or give some additional weight to them either in some reward function or something else and that might fix that problem yeah yeah that's a good point um or even just shorten your context window and seeing what happens oh yeah yeah but it I mean probably wouldn't be the most interesting generation but then it wouldn't have the past to pay attention toh MH yeah and so did you need to do anything different or special to accommodate the variety of sources that you're able to accommodate so you mentioned you can do you can start with a image from like a text to image generation you could start with a photo you can start with a sketch um did you have to do anything from a data collection perspective or or from a training perspective or did it all just kind of work and was cool to see yeah it was it just kind of worked it was it was actually really crazy um so we used to have this thing called The Genie bottle challenge essentially where we would all get together and play our environments and see if we can reach a genie lamp that was generated from a text generated image um so basically we're like trying to play the model in the beginning was really bad um eventually we were able to reach the reach the genie lamp uh and so once we sort of solve that challenge we started thinking about what other sources of data would just work um and so at some point we asked um that so we we asked Jeff cl's children uh Caspian and cica to to actually make these sketches that we have in the paper um so we like played those uh played in those environments and um I think Richie also drew a really cool sketch and you saw it you can like draw climb up the ladder um and so yeah I think we we did did that and then at some point someone was like well what if we plugged in um real photos I was like why would that work we trained over platformer games um but then we were able to control Jack's dog Doris so oh wow for whatever reason yeah I mean it's very you know she looks like a character I guess in the middle of a screen so we were able to like move her around and stuff but we didn't do anything different we just plugged them in um although I would say actually we did find that like adding um uh sidebars to some of the images like you know what you would see in like a video like adding these black sidebars to some of them kind of helped it figure out what to control um so yeah that was interesting yeah oh one other thing I forgot to mention is um it can sometimes be difficult to figure out what you want to control in the environment um like maybe you want to control the clouds in the sky rather than the dog but if you take an action that controls the right thing then all of the actions start controlling the the right thing um which is interesting because they immediately or sorry initially they might be controlling different parts of the scene but as soon as you take an action that's controlling the thing you want to control all of a sudden they start controlling it too um if that make sense uh it does I think going back to our previous conversation about the relationship between the actions and the prior frames to the Dynamics model so if you somehow choose an action that happens to result in controlling a cloud then the Dynamics model will see that in the kind of the frame history and kind of key in on that is that the idea that's a good point yeah yeah exactly yeah actually if you even if you plug in a video it's much easier to control given part of that like if you've given it some video you can figure out what to control if you do it so yes that's basically it um what do you mean plugin and video is that uh an input mode in addition to um static images you can also give it some frames of I guess I don't see why not if you're just kind of feeding into the Dynamics model yeah exactly I mean essentially what you have to do is infer the actions still so you infer you you kick a frame you've inferred all the actions you roll that out into the model and then you can predict the next frame given that that context yeah I don't think we actually put any results in the paper about this but it was something that we saw yeah um and I think this came out around roughly or shortly after Sora um I forget the the specific timing relationship but you know there were kind of some obvious comparisons uh between the two just from a video generation perspective um but the the last comment you made about kind of providing an input um a video as input um as opposed to you know static image I think maybe accentuates the comparison and suggests that um you know this could be like a playable you know Sora I don't know why that's any different than what what we've been talking about thus far but um I think it kind of s maybe suggests that um you maybe the distinction is like you know with Sora and you know gen 3 from Runway and these other video generation models like you know they're they're beyond platformer games and like the idea that like you can do those things and also like have control over you know actors in these vide is kind of really interesting mhm yeah know I agree I think it would be cool mean yeah you can take any of these video generation techniques and then plug them into the model I would say I mean if you've trained on that sort of data it wouldn't work on the 2D platform or one I mean we controlled the dog but I doubt we would it would work on 3D um but yeah I think it's a good point uh we can sort of build upon these things and or I guess leverage them to get the kind of control ability that you wouldn't necessarily get initially from the model um but now have like a nice generated scenes that you can then take over interesting uh so beyond the idea that now you've got these interactive environments for reinforcement learning researchers like what are the the broader implications of Genie do you think when we started we were definitely interested in using it for for reinforcement learning um I think over time we realized they were it was fun to actually interact with it um I mean this is why we started looking into sketches and people's drawings um I think using it as like a tool in classrooms like when we put out the paper we got people reached out to us from like places that I wouldn't have expected so like teachers interested in using this sort of model for their students um people who had data like video only data and they were trying to learn simulations of that data um that they can use maybe also as teaching tools um for people in the industry um just something fun for creatives to work with I think there there's many different kinds of um yeah areas that it could be used for and what do you think are the biggest gaps in terms of you know if you wanted to use this to make actually playable games like do you see that as how far do you see a is like this the right thing to do for that or is that you know or not really or um you know and B like how far do you think that is and see like what are the biggest you know holes or gaps or how would you approach that I mean I think one of them is imprint speed like yeah you can't really interact with these in real time I don't I I mean we the videos that we generated were you know not in real time they were just you know we showed a video of us interacting with them but it was I think probably one frame per second so that's one one thing trying trying to speed up inference um but also yeah I think the question of whether you know is this something you would want to do is is an important one and I think making sure that you're having conversations with people who would also be impacted by this technology early on would also be important um yeah it's a good question I mean I don't think I mean Genie was I wouldn't play it as a game at this point in time um but it is fun I would say it's a very different kind of game though so it's not something that's going to take over like I don't know whatever your favorite video game is um but I think it's like a new sort of it could be a new form of of I guess interactive um media that people can interact interactive media you can interact with yeah but yeah it's just it's just like a new form of I would say it could be art it could be I don't know uh some technology but it's different I think it's not necessarily or doesn't have to be the same as the game would be yeah I think referring to it as interactive kind of gloss over the inference speed issue for me and I was thinking someone was like pressing up down left right and controlling the scene in real time and I mean yeah depends on how much time you've got we do it yeah it takes some time for sure yeah and so were you doing it uh were you doing it in batch where meaning you just kind of picked a sequence of actions and fed that to the model and let it generate a bunch of frames or you know did you run experiments where you were sitting there and pressing up and pressing up and waiting uh for that was that was how we generated most of them unfortunately yeah like we have this this video in the paper or sorry the on the website that shows like a zooming out on all the environments you can generate with geni and that was all like real gameplay of people like stepping into the environment and interacting because random Behavior doesn't look interesting like if you were just to plug in like a random sequence of actions it would you just see like things jittering around like it doesn't look that interesting um so we wanted to have things that look like real game play um the alternative would be to maybe generate like a game tree over time but that would take a really long time um to do uh so yeah no we we did it ourselves how does this tie into kind of your other areas of Interest around video generation and kind of where do you see this going what are you excited to poke around with next yeah uh yeah great question um I think it's it's been interesting to see I guess my sort of shift from doing reinforcement learning to doing um to working on video generation although I've always I would say I've always done reinforcement learning from videos actually um so it's it's not that crazy of a shift um but I think it's really cool to start looking into the different kinds of uh methods that people have been using um like I mentioned this approach was masket uh now I'm getting more interested in diffusion which you know it's like it's a completely different area you know from what I'm I'm used to but it's it's been fun to learn um about that space um I think also just the creative aspect of it and like I mean I think it's nice uh like I you mentioned I just joined Runway I think it's quite cool to actually have like people who can interact with the models that you're you're working on and have that sort of immediate impact um I think it's quite exciting to be in that space now so yeah those are the things awesome should we expect Runway to kind of continue working in this direction I can't say what we can expect I only joined a couple of weeks ago you know we did just well you know they they released it like the week that I joined they released this like gen three model which was yeah it was really really really cool to see very fun time to to join so yeah I would encourage people to try that one out awesome awesome well Ashley thanks so much for jumping on and sharing a bit about uh Genie and uh what you've been working on thank you as well it's been really fun to chat absolutely thank you [Music]