Visual question answering & reasoning over vision & language: Beyond limits of statistical learning?

Key Takeaways

okay hi there man so by the way what's the idip mean so it's a word or is just a kind of a logo uh so so how when the institute was created it was an acronym uh but it was something in french so they they basically dropped the acronym i i i forgot the way it was it but it meant something originally uh no it just it just became a word by itself okay ejab okay yeah yeah that's good yeah yeah so you also so you have been also working with uh shadow and also peter anderson previously right oh yeah yeah yeah many people together yeah indeed yeah come on yeah so i think uh how about we first getting started and then so we have uh um so 20 so 20 people here and usually during the talk more people will join us so so let's begin uh so maybe uh i will first introduce uh today's speaker um today's speaker is dr damian taney who is a research scientist head in the machine learning group at the idm research institute in switzerland um so he's best known for his work under the intersection of computer vision machine learning and also natural language processing in particular he was part of the team that won the challenge on vqa at the cvpr 2017. his work has introduced the famous button up top-down visual feature that has been widely used in almost every architecture for vision the language nowadays more broadly kz interested in building a robust machine learning model that can truly understand the world around us he has previously been affliated with the university of adelaide in australia cmu the university of beth the university of innstruck and the university of ridge in where key is originally from and for this virtual visit qrp join us remotely from the uh the idiom research institute in the beautiful swiss apps uh so without further delay i will have all the the time to titanium yeah thank you thank you very much thanks for having me again yeah so i'm really excited to be uh with you today uh well with you virtually and so in this talk i'm going to share some reflections about the state of the art in vision and language and in particular about the limitations for visual question answering since i could not visit in person today i will just take the opportunity to share with you the view that we have from the office where i am here in switzerland so i am with the idiap research institute it's an institute where we do research in ai broadly i moved here a few months ago and i'm now heading the machine learning group here so the institute is very active in nlp in speech processing also in robotics if you are interested in visiting or even looking for a new position uh please get in touch with me we have a lot of beautiful things to see around here so i want to start this talk by reminding us why visual question answering is such an exciting task i think that's because it's a very general but also very complex task we all know this example with the banana from a well-known vqa data set and i think the complexity really becomes clear when you look for examples in the training set that are relevant for answering this specific question even though you can find examples about bananas and you can find examples about mustaches it's clear that answering this question requires much more than just interpolating between training examples the key capability here that is required can be summarized as out of distribution generalization so this means combining learned concepts and reasoning mechanisms beyond training distribution beyond what was demonstrated demonstrated by the training examples and this is a fundamental research topic in machine learning but it's far from solved and most research on the topic still struggles with toy data like colored mnist digits so why is this difficult that's because the principle of empirical risk minimization that we use most of the time is really about learning by association so we learn correlations between examples and labels in a training set and we can see the limits of this approach with uh just a simple example of image recognition let's say that we want to train let's say a bird detector so first we collect some images about well images of birds and in natural images it's very likely that birds will appear on backgrounds of blue skies and tree branches like like these ones and the problem is that you also want to recognize birds in different scenes like a bird in a parking lot but the model uh is probably gonna learn the this very strong correlation between the bird label and the blue sky which is useless in the parking lot obviously so if the model relies on these correlations to recognize birds it just won't work on those different examples so it's important to make a distinction here between the classical in-domain generalization and the strong out of the main generalization so the former is mostly about filling the gaps very loosely the gaps between training examples and it's generally agreed i think that there are some general almost universal biases that can help here like some kind of smoothness prior on the classification function and in this case more data also always helps so in fact in the limit of infinite data the in domain generalization is solved but the out of the main generalization is different so here we need to distinguish which features are robust ones and which features are just spuriously correlated with the labels so in our examples of the birds we could say that the shape of the bird is maybe a robust feature whereas the background is as spurious so here the key thing is that you need more you need additional knowledge that has to be task specific and to make that really clear we could say in this data that the label is not called bird but the label could be renamed say sunny day and tree branches so just renaming the label so using the same data it would actually become the right thing to do to rely on the background now so this shows that how to generalize out of distribution is often not fully specified by the data that we have so it means that getting more examples so more of the same biased data does not help and the thesis of this talk is that the complex tasks like vqa require something like that they require stepping beyond the framework of classical statistical learning so in this talk i'm gonna first review some principles from causal reasoning that i find useful to formalize what we can do beyond this learning by association and then we'll see something implications for the evaluation of models and then for the learning of better models so if you wonder why causality matters remember that when we say that variable x causes y it's uh here it only means that intervening on x affects y so that is setting x to a specific value has an effect on why so just make that explicit to a simple task let's say we have images of cars and we want to predict the top speed of the car so let's say we collect some data say from wikipedia so we have images and we get the annotations of the top speed of each image so that's our training data and then we train a machine learning model on this data so with a standard statistical model you learn correlations and in this case you notice that we have a lot of pretty nice red italian cars and these also happen to be pretty fast so it's likely that the model will learn this correlation and basically learn that red means fast so obviously this is true only for data from this particular distribution and obviously we know that there is no causal relation between the color and the speed so the idea of a causal model is that we encode the effect of interventions so that is what would happen to the speed of the car if we were to repaint it so obviously it's not going to get any faster so with a causal model we can make predictions now about conditions that were not in the training data and that's basically the definition of out of distribution generalization so in these two types of models statistical and causal we learn very different mathematical objects and what is important to notice is that for the statistical model using more data so that is more of the same data does not turn it into a causal model so if you get more images of red ferraris that's not going to help understand that the relations between color and speed is only spurious so let's get back to vqa and the view that i like to take here is to compare on one side the real world and on the other side the model that we learn so in the real world we have a set of mechanisms that produce the data that we observe our training data and the causal structure of these mechanisms matter because that's what defines which features are correlated with one another and which of these which of these correlations are spurious or robust so we call these mechanisms the data generating process and in vqa that's basically the human annotator so someone who takes a question and an image as input and then identifies relevant visual information and then use that to give an answer on the other side in a learn model we basically want to mimic this process and the inference of a model that's been trained also has a causal structure in the sense that it also takes inputs that leads eventually to an answer but the structure is not necessarily the same as in the real world for example we can draw the causal structure of a bad model that doesn't use the image at all and only guesses the answer directly from the question or we could here i represented a model that doesn't have an explicit attention mechanism but an important property of the causal model of the world on the left side an important property is that its structure implies a factorization of the joint distribution of the variables so this factorization is a special one in which the factors correspond to independent causal mechanisms so these mechanisms are guaranteed to transfer all of distribution that means that's the mechanism that we want to learn so we don't care about say the marginal distribution of images for example but the mechanism that identifies the relevant visual information and the mechanism that goes from there to the correct answer that's the bits that we want to learn that will make them all generalize in novel situations so to learn a robust model we basically want to mirror some of the causal structure and mechanisms from the real world into the learned model so why is this hard and the answer is that the information that we need is often absent from the data sets that we use so typically we have a set of id examples or training examples and these these are the samples from the joint distribution and it's actually known that from this we cannot recover the causal structure of the data generating process so by taking id samples from the joint distribution you lose the information about the causal structure so this means that we need to bring additional information and information that has to be task specific so we sometimes do this for example with specific architecture but like designing attention models this actually can be seen as a way to inject some knowledge about the true causal structure of the task and the general takeaway here is that with just a standard data set we don't have enough information for ood generalization we need to bring something else to the table so i will come back later to the learning of models and first i want to talk a little bit about the evaluation of models so in evaluation we want to know basically whether a vqa model can answer arbitrary questions about novel and unusual scenes and a classical test set measures the in-domain generalization so with questions from the same distribution as the training set and that's not sufficient because it doesn't say whether the model is merely doing maybe shortcut learning like guessing answers using dataset biases so i think in vqa we have made a lot of progress already in coming up with better evaluation strategies and better benchmarks and so i i mentioned a few benchmarks like vqav2 vqacp gqaod so all of these are three improvements over the standard classical simple test set so i think that that's a great thing and i will show you that all these ideas from those datasets can be formalized and actually justify with causal principle very effectively so useful framework to think about this is called the kozariaki introduced by judy apple so this er key defines three types of queries that one can make to a model so these three types of queries are increasing in difficulty the first level is the association in the bottom here so this level is basically where we are kind of stuck with most of machine learning today and the other levels the interventions and counterfactuals these are the ones that are relevant for causal excuse me causal reasoning and so i will show you that these three levels can be matched with different evaluation settings that people have come up for vqa the first level the association is the classical test set so that's set from the same distribution as the training data so we've seen that this is pretty limited already or interestingly the second level the intervention that corresponds to datasets like vqacp that's datasets where we have a different distribution between the training and the test set and we can interpret this shift in distribution as an intervention on some variables in the data generating process and so it's in this sense of an intervention that these datasets can be seen as level two interventional queries and then the third level called the counterfactuals that can be matched with data sets of controversial examples that have been proposed in a few recent papers so many of you probably know about about this so the the idea is that you have test questions that are actually pairs of questions so in those pairs they are very similar but with a slight difference the difference can be in the question like a different word or in the image like an object that has been masked or in painted in the image and so those pairs of questions basically probe the model very close to the desired decision boundary so typically those pairs of example have a different answer the yeah the small difference make the answer correct and they're different and so since we can match uh type of data set with level three this means that this is indeed the most challenging of all types of evaluation so that's because we know that the three levels are strictly increasing in difficulty so each level requires strictly more close-down information and so overall this analysis is useful because it shows that some benchmarks are indeed better than others in the sense that they test the models for better causal reasoning and so you can use this um this yarky this analysis to also come up with even more difficult evaluation ideas for example we could imagine for vqa a setting where you would feed a model with question image and correct answer i don't think that's been done proposed so far so you would give the question image and correct answer to the model and then ask the model to generate an image so generate an alternative image to support a different answer so this would be extremely difficult because that would require well i would basically test the model for what it thinks would be the required visual evidence for an alternative answer so this is even closer to the definition of counterfactuals of standard causal reasoning uh hi there uh so can i have a question here please yeah yeah so i think uh here you have a good example about um so in the vqa field uh so what's the dataset corresponding to the intervention and the counterfactual right and but then in reality it seems many of the are still targeting those talking at the the association for example the vqa leaderboard and every time when we think the performance cannot be pushed further but every year in the weekly challenge there will be an even higher number so what do you think uh this kind of progress um uh so uh mean uh uh so from your perspective and um so what do i think um so so even for the weaker leaderboard it's kind of already achieving the human parity right so how do you understand uh so this kind of progress yeah that's a very good question and uh i i've been very critical uh about the use the continuing use of vqa v2 as a benchmark because i think this is really giving the wrong incentive so getting better better numbers on data sets of basically the level one this is not improving things in the direction that we want so i think it's pretty clear it's been discussed even in the vqa workshop this year that the accuracy that people are getting on this data set about 80 percent accuracy now that looks like human accuracy doesn't tell at all how fragile and basically how bad the models still are so it i think everybody knows that the dataset is not measuring what we really care about so i think we have a lot of better data sets and i would love to see the challenges using those better data set instead of trying to push the numbers on a metric that is really not measuring what we care about yeah thank you all right so go on feel free to interrupt at any time if you have if others have questions so now that we've seen we talked about about the evaluation i'm gonna see uh some implications of causal principles for the learning of better models so i'm going to stick with the ghost lariarchy for one more slide an important property of the cr key is that a model capable of a given level requires information relevant to this level or a level above it so this is really another way of saying that the levels are indeed increasing in difficulty so you need more information for each level so this also implies that we cannot learn a model capable of causal reasoning that is level two and three we cannot achieve that only from data from level one the observational data but that's usually the kind of data the only kind of data that we have in our data sets so we know that something is missing to learn models capable of level two entry so concretely what does this data look like so at level one as i said that's your typical data sets of id examples nothing special here at level two that's the data that we would get after an intervention so after an action on the world that modifies the training distribution and in practice we could get multiple say multiple sets of images if we have like an image recognition task multiple set of images from say different parts of the world or images acquired in different lighting conditions or data that maybe was annotated by different sets of annotators so the idea of having multiple sets of data that have arisen from a specific intervention you might not know what the intervention is but the fact that there is some change in the data directing process between those sets is bringing information if you remix all those data sets together you just reach off on everything you lose this information but by having multiple subsets and knowing that there is a change in distribution between those subsets that's where the extra information lies and then at level 3 that's counterfactual examples and that's just like like the examples that we saw for the evaluation so typically pairs of examples that are very similar but with a just a slight difference and typically a different label so if you can get this kind of example for trading then these carry a lot of information and they carry information that we cannot get from other types of data so this data is more expensive to acquire more difficult to acquire but they bring additional different training signal and now we'll show that with uh we can better understand and explain some successful techniques from a causal perspective that's going to be methods that basically use this kind of additional training signals and that improve generalization so we go through three existing methods and the first one is simply data augmentation so as you all know this is about generating additional training examples with the hard-coded transformations that preserve the label so you all know that but what's interesting is taking causal view and here well let's say with images we can formalize the data draining process with something that looks like the graph here so this graph shows that images are made of both spurious and reliable features and they are both correlated with the labels because there's a hidden confounder here node.c and this converter is not observed we also don't know specifically which parts of the image correspond to spurious features and which ones correspond to reliable features and we want a model that generalizes out of distribution that means a model that uses only the reliable features such that it is robust against changes in the confounder so from what we've seen before we know that this cannot be learned only from examples from the joint distribution over x and y however it could be done with information at level 2 like data obtained after interventions like i said and this is precisely what data augmentation does so data augmentations basically simulates interventions by modifying the images such that only the spurious features are modified so for example let's take data augmentation that applies geometric transformations on images this is essentially intervening on the extrinsic parameters of the camera or let's say with vqa with recreate questions if we augment the questions with rephrasings we are basically intervening on the writing style of the annotator so now we have samples from a modified distribution that is the outcome of an intervention and we know that this carries information about the causal mechanisms now that we didn't have before so it's no surprise that data augmentation does indeed improve generalization so what did we learn here what is this causal view useful for well an important implication is that the root source of the improvement is really in the specification of the transformations over x and y that are valid for the data generating process of the task so this means that it has to be task specific so there's no hope of finding a universal data augmentation because it has to be tailored to the data directing process of the task so you have to bring some information about the task here it's something that we probably had an interesting for but i think the causal perspective allows to formalize this more precisely now another technique that we proposed last year for vqa we call that unshuffling data so this is basically recovering non-id subsets of the training data so the motivation here started from existing work on domain generalization so that's for image recognition so in this past work what people do is to learn from data collected collected in multiple conditions so typically you have training examples for each class in multiple domains like photographs sketches paintings and and so on so you have multiple subsets of training data that that are from a different domain so our method showed that we can recover this kind of informative clusters from an existing standard data set so we split the training set into what we call multiple training environments so that's just multiple subsets and to do that we need some tasks knowledge so we apply that for to vqa in this case but uh yeah the key thing is that you need some knowledge about the task and some side annotations to do the clustering on so in practice we apply that to vqa and we use the question type and the answer to do the clustering on and so from a causal point of view this is again simulating interventions on the data journaling process so we can formalize uh this uh precisely i'm not gonna go through the detail here maybe i'll just leave it here and feel free to ask questions later but the takeaway is again that you can get information from level 2 which helps with generalization and this information is something that we have with this method that we didn't have in the original aggregated data set so by recovering those different subsets so the same data but we we cover some subsets that are not arbitrary we have some more information in there and again this extra information doesn't come for free and here the root source of the improvement is really in the choice of the clustering condition that we use to recover the subsets so it means that you have to bring some specific knowledge at some point again so there's no free lunch another last technique that i want to mention is about using counterfactual examples as training data so this was a paper that we had at eccv last year where the method was applied to vqa but also on nlp and other revision tasks so the method really shows how to get information from level three and the contractual data consists of those pairs of examples that we have mentioned before so they are very similar but with a different level and there are multiple ways to get this kind of data for example with language data like those movie reviews here those were obtained from annotators so the annotators were explicitly asked to edit some existing examples to flip the label so the annotators were asked to change as few words as possible in the movie review to flip the label from negative to positive or vice versa that's one way and another way to get this kind of data semi automatically is to use heuristics and side annotations for example for some vision tasks also for vqa you could mask mask objects or use existing annotations like human attention to inpaint regions that are known to be relevant for each question so you get these pairs of examples uh relatively cheaply but you still need some additional annotations or yeah some some some additional information so previous work had already used this kind of data for training but only as additional data so just like uh some data augmentation basically and so it was shown already that for the same amount of data examples of the style this type are more effective for improving generalization than the same amount of standard data so this was party work and what we realized from the causal perspective is that the level three information is recontained in the relations across the pairs so not in the examples themselves and that motivated us to design a new loss a loss that exploits the relations between the pairs of examples so what we do is first to compute the vector differences in feature space across every pair and then we use these vectors to align the gradient of the classifier to make the decision boundary orthogonal to this vector so with this additional loss we got clear improvements in generalization across data sets uh in a zero shot transfer setting and so we got improvements in vqa but also on nlp and other vision tasks and so what's interesting here is that we were using information that was in the data set and that was just not being used before because there was no training mechanism to really make use of this data well this information that was there and was just discarded before so now i want to give some takeaways i think the main message of this talk is that there are some limits to what can be learned from a set of iid examples it doesn't matter how many you have and i find the causal perspective useful to understand the limits of what can be learned in a given training condition so i see causal reasoning as helpful here a bit like information theory in the sense that it sets hard limits on what you can do with some given resources and the second message of the talk is that if you learn something about the causal structure and mechanisms of the world then you can do a lot more so the principle of causal reasoning do not really tell you how to design or learn better models but at the very least they can point at directions where to find the information that was missing before and going forward i i like to think that we could shift our focus from building better models to rather building better training environments so that is thinking about how to provide those training signals that are missing from the data sets that we use today so in this talk we have seen some existing techniques that do this already this kind of using additional information and with different types of data or site annotations and sometimes these annotations are already present in datasets that we have and the data well the signal is just not being used at the moment so i think there are probably other cases like that for example in some data sets you can find information about annotator identity that's almost never used so you could use that for example to cluster the data into different subgroups of notators that might give you some useful kind of weak supervision there or you can think about using data from non-stationary sources so in this case you have some distribution shifts over time and this again could be used as a form of supervision so there are probably a lot of opportunities like that of signals that are available but just not being used and not taking a high level view of what's been done and maybe the road ahead here is that we can draw parallels between machine learning and causal reasoning so in machine learning we've moved from good old-fashioned ai to deep learning and we can now handle row high-dimensional data and in classical causal inference there are a lot of techniques that exist that are suitable for discrete and predefined variables like the kind of graph that i've included some slides earlier but this kind of methods are obviously not suitable for the high dimensional data that that we want to use like images and natural language so the next step is what some people have called causal representation learning and this is basically discovering latent causal variables and their structure jointly so i think this will eventually emerge probably from ideas both in deep learning and ideas in causal reasoning so they those ideas might already exist separately and they will need to emerge in some ways so i think there are a lot of exciting opportunities here and i hope this inspired you to maybe pursue in this direction and maybe build help build models that journalize better thanks so thank you damian uh so for the great talk any question from the audience uh here i have a question here dummy thanks thank you for visiting us this is a pong chan zhang from msr so the in fact the causal representation learning is indeed kind of very important uh there are typically two approaches to at least as fast as their two approaches to achieve this one is kind of sound many academic people are trying to design a new loss functions like environment risk minimization there's kind of some other kind of those functions to enforce uh causal or kind of representation learning but there's another way to do that which is i just as you said collecting more data because those kind of information really relies on the data and the also connect data means that maybe we gonna carefully connect they kind of repair the data you mentioned before the counter factor you mentioned before or i just bluntly connect more data web skill data i hope that those web still data can contain they can counter fracture data can you want and finally just train with this kind of large skill web skill data i want to know that kind of which approach you are using that will kind of kind of faster getting the goal of okay learning more generalizable machine learning models will this need to kind of a competition of kind of natural larger models not just another data set and finally uh academic people can really gonna cannot really get a phone with this knight mode or something this large data sets [Music] so i i think uh the large-scale so what we need is not well more data can help but what we are really missing is other types of data so different kind of data so for example the controversial examples you say that you might find those in web scale data you would only find that if you design your collection process to actually recover those pairs of examples so like i said the in extra information that we are missing is not in the additional examples but it's in the pairing of examples of those controversials and the i think there's no dichotomy between the more data and other loss functions so things like uh irm and those kind of causally motivated approaches if you if you think closely about them they are using those different kind of signals so irm is using data from multiple training environments so that's basically what i showed so information from level two so data that being collected with different conditions but my question is that is it possible that we do not need to do this very careful data solution process we just kind of bluntly enlarge our data sets natural and nudger do a very cost level selection like just just define rules that okay i will connect data from another domain but i do not carefully kind of construct pairs do very funguid annotation so just hope that okay with minimal effort of data connection then it's okay i think the speaker dropped off yeah i think that's my question no i think it's due to some technical reason maybe we wait for one minute oh yeah she's coming back hi sorry i had a small connection issue yeah never mind so it's just uh i think i finished my question it's really just uh can the data connecting effort maybe can be reduced and maybe can be compensated by large datasets yeah yeah uh uh i think i think it's an interesting question so you you could replace some of the so like i said we are missing some training signals so you can replace those signals by the proper assumptions uh about the causal structure of the world so that's something you can inject for example with the right architectures and so i guess it's still an open question here how how much the combination of the right inductive biases and those priors with enormous amount of data how far we can get with that so i think uh i see a bit of hope with the web scale data of improving generalization to a point that is uh extremely satisfactory because we are covering such uh well we are basically covering the long tail very far and the we are basically covering a distribution that is not biased in any way so if you think about say imagenet the way the dataset has been collected and annotated implied that there are some selection biases but if you get information from the web and you can basically get all kinds of photographs then you i mean well you get much closer to what is the natural distribution of the world yeah exactly so this is also sometime when we work on causal problems this is also frustrating because for example one dataset is object net which is designed to kind of to test this kind of generalization ability to cause our kind of feature learning ability but currently the best model i think until last year still for example vit which is training with kind of web skill data but the model uh architecture the noise function is still the same with the traditional one yeah right yeah yes thanks good questions thanks oh hi let me can i ask a question yeah please thanks oh uh so one short economy of the current um data sets for vka i see that is that um the answers all consumes to be fixed right so the answer is fixed set of answers it's quite a artificial and makes it easy for people to or maybe to overtrain on something right actually make it harder to train a model that's generalizable um typically in real world that's rare that's very rare like for example if you ask a person what is this person doing you can say oh the person is eating what you're going to say the president is cutting the meat or the person is having dinner with the friends together or the person is talking while the person is eating right so you could have many many answers and all the answers are actually correct so but currently the data says though that you have the answer exactly the same one otherwise you're wrong but that that is quite artificial i just wonder if you can you know comment on that yeah i totally agree uh if if uh i think that's both 4g i can't think about that for both the evaluation and training with an evaluation about where we had multiple correct answers then we could uh probably better evaluate whether the model is is doing something sensible and we if we had those annotations for training i would probably get much much closer so if if you well those who have worked with vqa models know that sometimes a model give an answer that had like a synonym of the correct answer or maybe just singular instead of a pro form and then it gets counted as a wrong answer whereas it's actually correct it's just not the right one in the dataset so yeah um i guess that would help both a better evaluation and also training better models if that's in the training that i have okay thank you so i don't have a really question but more of kind of like a discussion topic here that i like to bring up and just get your thoughts on um so we have um you know both the concepts of causal inference so having some type of structured knowledge and then being able to make some predictions from that structured knowledge and then we have um correlation coming from observe observations right and um you know typically in the in the human mind you know you could look at particular uh kinds of environments where um you know take take physics for example right you make some observations about the world from the correlations of the observations you make you make some kind of like um causal um structured knowledge to help you explain what you're observing in these correlations right and then you can kind of like tweak those that causal model that you have to make a new prediction and then test it make sure that the correlations that you observe align with your uh causal models right um i kind of feel like with with all the benchmarks and all the data sets we use to train right now um you know you're you're limited to um a fixed environment really like these two things are kind of like two sides of the same coin the correlation and the causality right they inform one another and they test one another and what happens with people is that you constantly flip back and forth right you observe something you learn something you say okay well maybe i can build this structured model to help explain what's going on and let me try changing the model and see if what the model predicts i actually get in reality and it's kind of like the cycle right i guess my my my open-ended question would be how do we develop benchmarks that allow you to test a system that works with that cycle yeah so it's a very good point so that's basically the gold standard for causal discovery that's the randomized control trial trial so that's just acting on the world and seeing what's what the outcome is if you can do that that's the best way to discover the causal structure i think there's no free lunch here in the sense that if you want to interact with the world or you want to let the model interact with the world uh you need access like a direct access to the world and people are kind of trying to do that with simulators so there's a community building a part of the vqa community actually moving to 3d simulators of very complex environments like households yeah basically kitchens and all that where you can while the agent can move stuff and navigate the environment uh push objects around all that so i'm saying there's no free lunch because you have to basically reproduce the world in the simulator so the the knowledge uh you basically have to reconstruct the world in a simulator yeah interesting i guess that that's that's that's sort of like the answer if you want if you can build these toy environments you can have an agent uh kind of like play in an experiment um then i guess if your simulation gets good enough you can let it play outside of the simulation at some point yeah so i can see those uh simulators being useful like kind of the meta learning thing where the agent would learn strategies for causal discovery but then you would still need to just like you said let them lose in the real environment to apply those strategies to build its own model of the real environment otherwise it's only gonna learn about the kind of stuff that are in the simulator yeah right interesting yeah cool topics thanks thanks for your thoughts yeah thanks um so i also have a related question uh so regarding to uh to functions comments just now so for example when we do large scale uh so the large scale training right so which is very popular in the current um um so so so no matter if uh so no matter your nlp or vision language so projecting is very just a popular um and this kind of project is kind of doing um kind of have the effect of doing the intervention right uh now that we are not carefully designed so as just the pontia mentioned although we are not carefully designed um the counterfactual path but then maybe as your data becomes larger and larger you may uh from the noisy data you may see some uh some effects of the contractual reasoning so and then um although this is uh like uh so because if you want to do carefully labeling of the contractual uh the counterfactual examples maybe this is just too costly and not easy to scale up but then from the other side is indeed not ideal so from your perspective did you see any opportunity that we can bring uh the the causal uh the causal principle into the pre-training rather than only focusing for example on the downstream tasks such as vqa and then if this is possible how can we do that or just not that easy to be possible yeah that's my my question yeah so i i think uh thinking about the causal point of view uh helps in understanding what is missing from just a data set that may be very large so uh to tie that with the kind of unsupervised learning that we see working really well nowadays the contrastive contrastive objectives that that that work really well um these can be interpreted as um a form of controversial learning from contractuals and here the important piece of making this work uh is how you choose the positive and negative examples so we have typically we have heuristics to select those and so what you realize from the causal perspective is that that's really the key to make this work if you select arbitrary examples pairs of examples then it's not gonna work so in practice we have seen that so you need to choose them carefully but uh yeah the console perspective really uh shows that clearly that that that's where the additional information is coming from does that bring a bit of information to your question uh indeed i want to comment here indeed kind of the key part of this kind of contraction learning is that use of positive and negative peers and i i want to could you share your kind of experience i'm gonna uh can a bad construction of these peers will not result in good uh kind of kind of good generalization obviously because typically we can at least i believe that this kind of large skill contraction learning will be very robust to data noises so when we look at our kind of weaponry connected data they kind of image text pair are very noisy but still we can get quite vulnerable results which already shows that okay this kind of this kind of uh kind of training is robust to kind of training noises which means that maybe the pair is not good it still works well yeah i guess you have much more experience than i do so uh yeah i would trust your your experience more than mine but i i want to kind of hear your kind of experience kind of what kind of things that you found it's sensitive or sensitive of fragile yeah i don't have direct practical experience uh to report here yeah okay okay thanks so yeah i i guess uh what you follow about to follow up uh on that that's uh the you can replace so the additional signal that you want to bring you can replace that in some cases with the right heuristics but the heuristics have to be designed properly for the task or at least the kind of data that that you have yeah i know it's original very general comment that does not have a direct applicability but yeah that's uh that's all i can say i guess uh yeah thank you so any more questions so i think if you know uh so well also right on time and let's uh thank you our speaker again and then this will be the end of the talk yeah thank you thank you very much thank you bye thank you bye-bye thank you thank you