Trends in Natural Language Processing with Sameer Singh - #445

Key Takeaways

[Music] all right everyone welcome to ai rewind 2020 our show where we reconnect with friends of the pod to discuss trends and key areas of machine learning and ai this time around we are back with samir singh assistant professor in the department of computer science at uc irvine to talk about trends in natural language processing we last spoke to samir actually in october where he joined us for our nlp office hour session at twimlfest and before that our interview in september where we discussed beyond accuracy behavioral testing for nlp models samir welcome back to the podcast hi sam uh glad to be back awesome i'm looking forward to our conversation i will refer folks to that uh september discussion for your background uh and it hasn't been all that long so i won't ask you what you've been up to since we can jump right into uh talking about nlp in 2020 uh what jumps out to you as kind of the broad brushstroke takeaways for the year yeah so this has been a really interesting year and that's maybe the biggest understatement you can have um but but but for from a purely nlp research point of view this i would say it has been the year of self reflection or introspection in some sense um i don't know maybe it's the existential threat we have as a as a species or maybe it's just everybody being stuck at home and sort of thinking back as to what they've been doing uh or more likely i guess it's just where nlp is at this stage uh and the past couple of years sort of building up to it where i think we are at the stage where we have these really powerful pre-trained language models that are able to do a lot of things that we uh we thought was very difficult to do but they're also not able to do a lot of things that we think they should be able to do and and and we're in this sort of a new paradigm trying to understand how much are they actually working some things look too good some things look not good enough but even more theoretically like what are they capable of what are the fundamental limitations and a lot of them are so good that they may be close to being uh practically deployed in in much wider scale than they are currently being deployed so how do we make them practical how how can we make them something people want to use um so that's kind of been the sort of overall theme i think of this this year um more specifically i think this has also been the year where people are kind of moving away the the traditional supervised setup that we we are so familiar with uh this this whole idea of you have some training data and some validation data that's basically from the same distribution you train some model on the training data make sure it works on the validation data and if it does you deploy it at this time i think there is a lot of papers that are moving away from that paradigm trying to understand what the flaws are in this paradigm fundamentally but especially with language models and pretend models how these things interact and then moving a lot towards interpretability analysis and and all these other things that we'll talk about today um your comments about the introspective period sounded a bit like uh deep learning four or five years ago maybe uh are there parallels there do you think or is it a different kind of uh introspection i think you know you know that like uh i'm thinking of comments like uh uh ali's comments at nurps a few years ago like we you know just talking about the lack of rigor and not understanding how these things work and stuff like that i i think it's a little bit more fundamental i i i would i i would dare say bigger bigger than that right so i think a lot of those criticisms were still housed in uh machine learning and modeling and what are the right models to be building and things like that uh in nlp it's a little bit different where these models are are working at some things that we didn't imagine them to be working but they are so fundamentally broken in other things and uh this idea of writing papers solely on figuring out what they can't do or what they can't do yet and what they can't do ever and what we need to do to take the next step uh i haven't seen maybe that's also my uh my my narrow scope but i haven't seen that level of uh discussion in in other avenues in machine learning or computer vision so um i feel it's it's a little bit different okay okay also before before we go i would i would want to make sure that everything we talk about today is of course my my view on this stuff uh and and there may be lots of biases whether they are towards topics or some regional stuff or specific people but just just want to make sure people know that fair enough fair enough um you identified some key themes for us to discuss in nlp uh the first of those i don't think will be a surprise to anyone following the field closely and that is uh massive language modeling is what you call the just the dramatic increase in scale and and size for these language models uh you know tell us a little bit about your thoughts in in that area yeah so uh that's definitely been one of the key i guess the news grabbing stuff that happened this year was the release of gpd3 um and i think you know from from a slightly uh negative perspective gpt 3 is just gpt 2 plus more scale in some sense right but some of the quality of the generations and then i can talk about everything that that paper did that i like made made it something that everybody is paying attention to and something that i think could could have a big impact on how nlp is is potentially done in the future um and then we can talk about that a little bit later as well so let's let's actually talk about what gpt3 is um so gpd3 is this uh language model where they're just trying to predict the next token uh what makes it different from gpt2 is the scale of both the model and the the sort of how much data it's ingesting right and in terms of the scale of the model it's at least i think 100 times the size of gpd2 um and 10 times the size of anything of any language model before gpt3 came out which is like you know when we talk about scales that's two orders of magnitude bigger in a couple years that is pretty crazy to imagine what that means and it's so big that it cannot fit any of the consumer grade hardware um even for inference not even for um for training or anything like that and sort of the biggest news grabbing thing that came out was the the text that was being generated by the language model was incredibly human-like um it was coherent in ways that we didn't expect it to uh it was making up facts that seemed completely reasonable um way more than gpt do um and and all of this stuff was sort of the biggest attention grabbing thing uh the other thing that i really like and i think that's the one that uh sort of indicates something for the future is the fact that they focused on few short performance um and i don't want to call it few short learning but but let's just call it few short performance where the idea is to not train the model anymore once it's done language modeling it's just predicting the next token it's done but if you want to use it for something like sentiment analysis then maybe we give it a few examples as the beginning of a document that is generating um so this movie was terrible and then negative this uh movie was awesome positive and we gave a few examples of this and then give it another example that we don't know the sentiment of and let the model auto complete whatever token it wants to generate and not only does it generate the labels most of the time it actually generates correct labels and they they did a really good job trying it out on a bunch of different data sets comparing against the state-of-the-art fine-tuned one state-of-the-art few short learning models and in many cases gpt3 does uh amazingly well right not always beating the state of the art but the fact that there is no fine tuning going on the fact that there is um like no model update no you're not adding anything on top of it uh this all of that makes it incredibly um interesting to me right and so in some sense i think this this gpd paper jabbed all the headlines got all the eyeballs uh but i feel it was both overhyped in in certain ways but also i think underestimated um in different ways and then i'm really curious to see what the long-term impact of something like gpd3 would be in what way is underestimated uh i think this this few short performance stuff in particular yeah yeah i think people were pointing out specific things like oh it cannot do look i gave it an example and it didn't predict the sentiment correctly or here's my made-up few short tasks that i'm interested in um i gave it a few examples and it didn't do that well i think a lot of the criticisms are we will talk about some criticisms that are more general but a lot of the criticisms i see are specific to gpt three rather than the ideas it represents uh to me right so i think the yeah few short performance is not amazing and it's not perfect and obviously there are cases you can build up especially things like textual entailment and nli uh gpd3 is sort of falling short uh but that's not to say what would happen if gpt4 or gpd5 sort of cries out right maybe those capabilities will still be a problem uh maybe we would solve them or open ai would solve them uh but but that's something for for us to know rather than be critical of you in what ways is the few shots uh capability you're describing different from what we do before with gpg2 with prompts um is it essentially the same mechanism or is it uh a fundamentally different uh kind of technical approach so i think uh gpd2 a lot of the times um when it's used for few short or for fully supervised uh is used in a fine tuning based set um so so you actually give a few training examples if you're doing fine tuning you collect some some small training data and you actually fine-tune like the last layer or add a new layer or something yeah uh just just to make sure it's it's doing the right thing and in that case gpt2 acts like a representation of the of language right it's not used as the the model itself and then your fine tuning that's using that representation to to figure out what's important about the current input um and gpt three sort of moves away from that right so suppose you're interested in let's say your facebook and you're interesting interested in a recommendation system of news for every user in your in your uh all your customers uh if you use gpt2 the natural way would be to have a gpt-2 model for each user where you have the main model that's fine-tuned a little bit based on whatever the user has been clicking on and that would mean you have lots and lots of copies of gpt2 lying around and that's maybe a simple simplistic view you wouldn't actually do that if you're facebook or i hope you're not doing that but but what gpt3 tells you is you can use the same gpt3 model just give it a few examples of what the user has been clicking on at test time um and then based on whatever comes out you can test you can put new new news recommended stories and see whether the model thinks that the user will like them or not and so that could mean that you just have one model that you're using to serve everyone without any fine tuning without any further training yeah i i get that i think with gpt2 you know whether you fine-tune or or not you would still give the language model some uh you wouldn't call it an example but you'd give it kind of a prompt that it would you know then make predictions off of and complete and um curious the ex are the examples that you're you know for gpt3 are they essentially you know very similar to prompts or is it a totally different mechanism that is you know trying to somehow replicate fine-tuning uh so i would say it's closer to prompting that yeah i think you still have to think a little bit about what's the right format and if the format is a little bit different it's not quite clear whether gpt3 will give you as accurate of a result um and this is sort of the thing that people are going to be studying in and we're also looking at a little bit on um what kind of prompting or how sensitive is gpt3 to different promptings and things like that and turns out uh it seems a little sensitive but it may not actually be and then we'll have 2021 to talk more about that stuff but the the underlying mechanism seems a little different from fine tuning because this is something that gt3 hasn't quite seen during training like it maybe has seen instances of uh review and ratings and things like that but maybe not in this exact same prompt that you're giving it a few shots and it has to do this translation without any learning that what you want is oh if it's four stars and above you want me to stay positive otherwise you want to say negative and and all of these things are happening internally in the representation some somehow uh which is which is exciting following the release of gpt 3 there were a ton of applications to uh i think a pretty interesting array of um of domains like using gpt3 to generate websites and generate code and things like that do you have a list of the the most interesting to you examples yeah so i have some i guess that i put together uh but of course these are ones that somehow fell on my twitter feed and may not be comprehensive at all um but i i guess i'll start the one with one that i've been playing around a lot with which is ai dungeon this is something that came out a while ago with gpd2 as the main engine and now they have a gpd three-backed one which is on the page here uh but essentially ai dungeon is this text-based adventure game uh if people have played that where you're as an input it's just a text interface where a description of the current environment uh appears on the screen and you can write whatever you want to write as a way to act in that environment uh ai dungeon has made it completely flexible whereas the older versions used to have a very templated view of what you could enter and what would happen if you if you were to act certain way ai dungeon uses gpd 2 and gpt3 to sort of create very rich environments or automatically created environments and then when you act on something uh it just goes to gpd2 and the gpd2 and 3 just sort of continue the story from from wherever you uh whatever you said you would do uh it leads to really well i would you know you play around with it and you'll soon run into sort of confusing uh messes and stuff like that but uh the the things that happen that are cool are so surprising and so interesting that uh those gems are what you're seeking and then it's been really fun to play with hey dungeon any other yeah some of the other ones i like there is the the description to code one i think is maybe the one that could become practical um in a certain extent uh gpd3 was trained on all of internet so yes it looked at code and comments and things like that and maybe because of it it's able to do this translation but you could potentially if that's what you're interested in uh train gpt3 model with more code and more comments and things like that to be able to generate practical systems where a user comes in with no experience in programming just writes out what they want it to do and programming doesn't have to be just sort of designing websites it could be uh writing like latex math equations and there's someone who did that as well or um you know typing out what how you want a plot to be like like a graph or something like that just describing it and then generating the matlab map.lib code or something to to get that plot i think all of these are really useful examples where they could make much more natural interfaces to to computers than programming languages and that's kind of been the goal for a large section of nlp for a while yeah yeah yeah i mean the in addition to you know generating websites uh which we saw i think i saw examples of like generating sql queries for databases and uh in fact at re invent this year aws just announced a commercial product that is uh q i think is the name of it i forget the full name of it uh but you're you know that this is not a new concept giving natural language queries uh against data but you have to wonder now the extent to which they're using you know gpt3 or similar types of uh language models as part of that yeah and then like some of these things have been really interesting uh examples of what gpt3 can do but again i think there is still a big gap in making a practical tool that i would let like people who have no programming expertise actually used to build stuff right and i think it's still right now it's still a good shortcut for someone who knows programming language to generate a piece of code that they can fix or improve upon but we don't quite understand the limitations of if you really want to do something new fundamentally new in the programming language that's kind of why you want to do programming uh i don't know how much of this stuff will generalize to that yeah but if you want to sort lists and and create simple web pages maybe it's where we are is good enough yeah yeah yeah i recently saw an article about um it was talking about kind of data leakage with gpt3 i guess that's come up recently also not necessarily a new thing we've been talking about this in the context of cnns and leaking faces and things like that but this was i think there's a website up there that is something along the lines of is your phone number in gpt3 and uh yeah have you seen much of that uh concern as well yes yes i think i haven't seen the it's your phone number on gpt3 but there's a recent paper that came out of a bunch of folks including some people at berkeley that i know uh that are looking at exactly this and and and they were essentially you know using very simple techniques of just looking at what gpt3 generates uh that is different from what a different language model would generate right so like given the same prompt okay if gpt3 generates something that looks very different from what a regular language model would generate then either that sentence is somehow very sophisticated english or it's a sentence or a set of tokens that gpt3 has seen before um and when they did that they saw that a lot of the stuff that comes up including people's phone numbers names addresses and things like that that were direct copies of things that they already saw in the gpd3 training data yeah so that is that is definitely a concern and sort of differentiable differential privacy applied to gpd3 is the sort of scale that i don't know if they've that that kind of research is pretty pretty interesting um because there are sort of performance tradeoffs when you when you try to inject privacy considerations in most natural uh ways and and that may not be you know it might really make gpt3 flawed in ways that make it impractical so yeah privacy stuff is really interesting to see and see what will happen and now gpt we're you know leaving 2020 gpt3 is not still the largest language model out there is that right uh i i haven't heard of anything i'm thinking like the tour microsoft touring uh language uh maybe maybe a question is when we're looking at kind of these uh metrics of language model size and kind of you know they're measured in terms of the number of parameters is that uh is that a squishy metric at all or is it um you know the the number is the number and there's no ambiguity there well so um i think for the time being that seems to be the case where the number does mean a lot um and a bigger model so yeah so that sort of ties into some of the other papers that have been coming out around this whole gpt3 sort of uh field and even in gptc paper right they did a really good job of showing like here are like 10 different models or something i forget how many they have but ranging from pretty close to gpt2 to something that's a gpt3 size and most of the results were not just on the final model or on the smallest model they sort of covered a range of parameters and looking at those plots it's actually really interesting to see that the lost curve even for the biggest model is going down it's not quite stabilized in a way that that we uh we usually hope when we are releasing a model uh and so when you go with larger models the curve might just still just keep going down right and this is girvin sort of perplexity or the loss for language modeling um and it seems to correspond well to downstream performance for for the time being um but and so so we know like if the models become ten times larger there is still scope for the loss to keep going down for such a large model and and so in in that sense having a larger model may directly translate into practical results so if someone says hey i have a 10 times larger gpd3 my prior would be to assume that that's going to be really successful at what they're trying to do yeah and so there has also been some interesting papers and i'll quickly mention some of them where they're essentially two i think uh the first one i'll mention is scaling laws also from open ai where they sort of tried out a bunch of different model sizes and bunch of different data set sizes and they sort of try to control for the amount of compute you want to throw at this language models and their goal was to find out whether if you have a finite amount of compute or some fixed amount of compute what should you be doing to best make use of it so if you want if you're interested in training a language model should you train a small model on a really large data set or a really large model on a really small data set um and and so on right and essentially what they observed was that there was a pretty nice sort of power law of sort of behavior to how the loss performs as you scale these things up so of course the laws cannot go keep going down sort of infinitely uh this is a loss of language model so there is sort of a lower bound to it because of language but we are not not quite close to that at least from the plots that we've seen um so given this parallel thing i could expect yeah ten times larger model would reduce uh the law loss in a certain uh with a certain amount um but what is has been interesting from this scaling loss paper is the observation that almost always a larger model is the better thing to do than a smaller model right so uh it might be better to train a larger model with less data rather than train a smaller model with a lot of data and that's quite an interesting observation because it was not quite so obvious especially this data versus model size trade-off is it's not quite easy to write down uh as an equation um but yeah so so and that also gives some uh justification for why every company wants to do a bigger language model now because even with a fixed amount of compute that is the right thing to do the other paper along these directions is called train large then compress that appeared at icml and it came around the same time so they make some similar observations but i think the biggest difference there was that they don't really look at language model performance like perplexity and things these are things you know companies care about but if you care about downstream tasks like mli or machine translation a lot of the same observations translate there as well so if if at test time you're interested in a small model what we would do right now is train a small model and then just use it but this paper shows that oh it's actually better to train a large model on whatever data you have for the same amount of compute as you would a small model and then compress it and you'll get a more accurate model in your downstream case yeah for folks that want to dig into that result a little bit more i'd refer them to our may interview with joseph gonzalez on that paper one thing i would also add about gpd3 before we sort of move on and and all of this scaling stuff gpt 3 is available as an api access to to to itself and that is also something that's a little bit new to the nlp research community because you know if you want to do research you you want the model locally and that was sort of the traditional way of thinking about it if you want to do something on birth you have to find unit if you want gpt2 you you want to fine tune it and so you need to be able to run it locally with this api access stuff you can do a lot of not only analysis but actually using gpd3 without any hardware a big hardware at your disposal right so potentially you can write apps and things like that that run on your phone that still use the full extent of gpd3 as long as you have the api access on the negative this also sort of creates a gateway to whoever can access gpt3 api or afford it in some cases they can uh the sort of whole garden opens up to them uh but if they don't they can't afford it or the yeah then you you're not granted a license then it becomes a problem well it also constrains the nature of the research that one could do around gpt3 you can do kind of like black box research but you can't really do like white box research where you're picking it apart from the internals yes that's a good point as well yeah i think the api could evolve over time to give more access to the internals potentially but yeah it's a lot difficult to do things like if i were to change the training data a little bit what would my model actually do you can't change the training data right now yeah or or if you were to fine-tune something like the last couple of players what would happen and that's also not something you can do um but yeah that's a good point uh i guess for me i've always been a lot more interested in sort of the black box analysis methods and and this just yeah is another use case of why those kind of analysis techniques are useful uh so gpg3 is not the only you know transformer based or language model you know that has uh that we've heard about this year uh any noteworthy entries in that regard um surprisingly this this year has seen very few uh sort of these large-scale pre-trained language models i think um you know two years ago was when elmo and bert sort of made the big splash last year was and even including two years ago the last maybe one and a half two years have been you know roberta and t5 towards the end of last year and a bunch of these new ones coming which are all um not only interesting from a scale point of view but interesting in how they are modeling text right like they're actually making interesting uh contributions i haven't seen that many of uh those kind of sort of large-scale pre-trained language models this year i'm sure there have been a few but maybe we ran out of muppets or something like that and people have uh stopped doing this uh what has been interesting is and and i haven't sort of read these in too much detail but this sort of papers where uh we've realized that transformers for sure is here to stay and then almost everything we're doing in nlp right now is based on transformers can we make them more efficient can we make them more compact do we actually need all of these attention heads and all of these weights that are lying around and there have been a lot of different efforts um i can just sort of name a few but won't get into details uh things like long former big bird performer reformer and and linformer which are all sort of trying to get at more efficient versions of transformers which i think is a really useful thing to be doing given that's what we're going to be using at least for a while uh awesome so that's gpt3 definitely played a big uh role in nlp this year for sure uh what's kind of the next big theme you've got for us yeah so i think uh around the same time um i guess my second theme would be trying to identify fundamental problems with doing language modeling or or i guess the better way to say it is problems with using language modeling as the way to do nlp right so defending depending on how you define nlp uh the main question is is language models enough right um and i think this this year was kind of nice from that perspective because acl also had this position paper track or vision track where you where you could write a paper making a certain argument without necessarily the the empirical evidence backing it which led to a lot of really interesting papers to read um and i think one of my favorite ones from those and that also was a favorite of a lot of people because it got the best paper award uh was bender and kohler's paper on sort of climbing towards nlu natural language understanding um where the goal was to sort of figure out how much can you do just with form and specifically can you get towards meaning just by using the form um so these are terms that are sort of not not in the usual cs syllabus so uh just to sort of clarify it uh what form means and this is how they define is the sort of observable part of the text that you look at right so the words the tokens the actual text that you read is the form uh and that's clearly what these models ingest during training that's what they're doing when they're doing language modeling they're just trying to predict what's the next element of the form that would have come but the question is whether the form alone can get these models to understand meaning and meaning is sort of everything that's not observed right so everything that's that the user is intending to say that's not explicitly just in terms of form whatever they're referring to whatever common sense assumptions they're making whatever the the actual intention is and all of nlp and nlu or most of it can be sort of uh you know summarized as trying to take form and then translate it into the meaning right because the meaning is what makes it useful and so this paper data makes the argument that form is not enough um and that you cannot get meaning without form um with just form right so you need something more to get get at the meaning behind it um so they the the catchy part of this paper is this uh octopus example that they they made up which is a little bit like uh some of those more traditional philosophical sort of paradoxes uh or or sort of situations that really give you insight into what is the difference between form and meaning um so in this case uh the idea is to imagine this hypothetical situation where you have two humans um stranded on different islands and they have you know they're stuck in these islands and they have some communication line between them uh maybe like an underwater telegraph uh channel or something like that and they find out hey we can talk to each other and then they they spend most of the days talking to each other uh let's say in english right um so everything going through that wire is the form right you you can't you can only express language you express words and text and things like that you're not making sounds you're not pointing to things and transmitting visual stuff you're just transmitting text uh in this situation now you can imagine a hyper intelligent being called the octopus which lives under water and can access this wire and can see what the words are going through this wire and it's hyper intelligent in the sense that it's really good at pattern recognition it's basically a neural network or a gpd3 like creature but it has never been outside of water it has never seen the islands it has never seen these two uh people um and so the question then ends up becoming like what can how much can this octopus actually know about what's happening um and and there is no limit to how much transmission it's being able to observe it can maybe observe you know for let's say forever right like 100 years or something like that one way to test how much the octopus understands is to say well at some point the octopus feels pretty confident that it can talk to b as if it were a and talk to a as if it were b so it just decides to cut the the wire and sort of inject its itself into this this communication channel and it's pretty good at carrying out these normal conversations that's not not a surprise because that's what sort of language models do uh a will send something and the octopus will respond that is true as long as normal situations are occurring but if a needs um some sort of some reference to the real world and has a question about it um that it knows a b a human would be able to answer the question is whether the octopus can answer that or not right so for example if a is on the island and an angry bear is attacking a and a has some sticks lying around it would want to know what it should do so it can go to this channel and say hey b i really need help can you tell me what to do and b would probably know or probably have a good idea of what to do maybe how to use sticks to create some kind of a weapon uh but the octopus having only seen the form doesn't really know uh what the bear means what does it mean to be attacked how you can use sticks to sort of do something so it doesn't have the common sense doesn't have the the the physical sort of physics knowledge and of course doesn't have the grounded reference to to what a bear is and and what the situation is um so this is sort of one example where how this paper makes it very concrete um how just by using form you you cannot be doing meaning just in that example i have not read this paper but in that example it strikes me that at the very end there you shifted the focus from one of uh form to meaning to this added this other aspect of intelligence which doesn't necessarily flow from the meaning of you know one could have yet one could posit that you know the octopus has you know 100 knowledge of the meaning of the words that doesn't necessarily mean that they know how to create a weapon out of sticks yeah so so let's so yeah that's the tricky thing about you know where that line is but i think assuming intelligence is like they are uh they can create they can detect all kinds of patterns and they can figure out what sort of these words mean right so they are not limited in trying to understand what the the words the sequences mean and things like that uh but but they don't know what sticks are apart from what they have read and presumably a would not tell be hey a stick is a long narrow rod made of wood or anything like that right like because that's a shared knowledge that b already has and and that's just one example of what meaning might be is is what a stick is right and so if the octopus has never seen what a stick is what an angry bear is uh it may not know exactly how to given it has seen all this form stuff even being sort of infinitely intelligent it wouldn't really know what what to do thanks but i think i misunderstood the assumptions so the idea is that it's seeing all of these words all of these forms uh but it doesn't have any kind of a priori knowledge of what they represent that's right and the idea is can it can it construct a meaning some knowledge of what this thing is without having or just having seen it exactly just having seen the words yeah yeah and and so this paper sort of through these examples that has a few others makes the argument that you um you cannot have natural language understanding just by a form right so this is not an empirical argument um they do have some examples of hey look gpt3 when you ask it what to do with an angry bear gives all kinds of ridiculous results i think it was g52 actually um but but that's kind of not the point of the paper right where gpt3 is now is not the point it's where fundamentally uh things are limited um i would say like personally i don't agree with all of the arguments in that paper partly because to me the difference between form and meaning is still a little fuzzy i think they treat label data as something that's attaching meaning to form but to me that can appear as form as well so i think it gets a little bit blurry and i think gpt3 is a good example of where some of those lines start getting a little blurry when you're not fine-tuning it yet are able to do the task it becomes a little easier but yeah i think uh yeah that would that would be a good summary of that paper i think it's important because it's getting people to question what you can't do with language models and it's kind of easy to get into this mode of oh language models are doing so much therefore they can do everything um and and thinking about the future of nlp and where we want it to be and where we uh are it is good to have these kind of uh sort of maybe more philosophical arguments but useful nonetheless uh i guess i'll note that uh as you mentioned emily bender is one of the authors of that paper and i also spoke to her back in may the main thrust of that conversation was exploring this idea of whether the degree to which linguistics kind of old school linguistics is missing from nlp research and well statistical approaches to nlp get us all of the way there uh very interesting conversation for folks that want to explore this further also kind of veering off into the philosophical right yeah yeah i think there is another paper that i want to bring up which is a little more recent from mnlp this year called experience grounds language this paper is also like a position paper it doesn't have any results but it kind of makes a lot of these uh the the sort of bender arguments a little bit more concrete for me by essentially giving a sort of scope of what the model is actually ingesting um so they give this five different scopes i think the first scope is the one that we are traditionally familiar with where you have some corpus that the model is trained on and that's basically all the model has access to that's sort of the very basic scope they talk about the internet or everything that's written by humans as being the next level right so just going from a corpus to everything that was written that's where we are right now that's where gpt3 is that's where most of our discussions are um and they talk about what some of the limitations are if you are at that stage right so can you learn language just by listening to radio right that's where that scope would be so that's kind of going close to uh emily and kohler point as well um the third level they talk about is perception where you go multimodal where you're able to ingest visual information as well so not only do you have access to the words between a and b but maybe they are face timing each other and you have access to what they look like maybe they can point the phone to the bear and now you know what the bed looks like what can we do theoretically when we have access to those multi-modal information that's kind of what they've been thinking about but there are still two more levels above that multimodal thing the level four is uh when you are actually in an environment so you're you you take your nlp agent and it's not just passively listening and hearing and ingesting all of the information but it can actually change things so it can it can decide to pick up something and drop it and see what happens uh it can and it can use a stick for example and hit it on on the bear and see what what happens and learn from that experience and how language sort of how you learn a lot of language by also being embodied in such an agent um and then finally the the top level is when you are in a more social setting where you're not just alone interacting with an environment but you're in a world surrounded by other agents that are not just automated agents humans and things like that and you're actually interacting with them not just by hitting them with sticks but also talking and and listening to how they respond so i think this is a really nice paper to tell us that you know if you're wondering what's next or or are we stuck here or something like that or is there nothing more uh it gives it shows that we are actually at a very very small uh place and there's a lot more to to do interesting it reminds me a little bit of a conversation that i had i forget the um the authors of this paper but it was out the conversation was nurps vancouver and the paper was visibert and it was trying to combine vert and like uh image information to do visual question answering types of problems and the contention was that the images were or were what grounded language or a part of what grounded language uh i like uh the framework the framework you just described kind of shows how there's much more than than that it's almost you know now like it's to like the levels of self-driving right it's like digital is that level three and then you've got still a couple more left and what do we what do we do that with that to allow these models to get more powerful it's pretty interesting question and it sort of mirrors how a child matures also to some some degree right like maybe initially they're just hearing sounds then they start seeing things or maybe that happens in a different order but definitely perception is first then embodiment of like trying to move things around and then social cues and stuff like that so so yeah definitely uh that is true i think what i would say about sort of vision and language work that's already happening a lot of it is doing tasks that require vision and language to be like visual question answering clearly you need an image and a question and and so you need some kind of birth that can handle two of them i think what is interesting and that's kind of maybe the point of some of these papers is to say even if you care about something like machine translation that is nothing to do with images right yeah or question answering on pieces of text it would still be potentially useful to give gpt3 access to youtube videos or something like that right because it can learn more about the world and even for tasks that seem it's only text only task it's just reading or something like that you would still use that experience the multimodal experience to do better yeah and i haven't seen that many papers that have successfully done that but yeah interesting interesting yeah uh so this theme was fundamental problems with language models uh i think your next theme is also fundamental problems with language models but slightly different talking vulnerabilities in particular yeah so i think uh the next category and um there are it's difficult to sort of point at just one paper in this because they're kind of all very different from each other so i'll just sort of maybe go through all of them uh briefly uh but the idea here is well suppose you're a company that wants to use gpd3 in practice or or in general you've got a language model that that you want to use for for some purpose are there concerns that you should have like maybe you don't care about fundamental questions of is meaning captured or not you just want to care about if i will i get in trouble if i deploy this this model um i think there have been a few papers looking at kind of problems with just relying on language modeling as the objective and how that can backfire or relying on these really large models that are shared by a few different companies how that can always be a problem so i think the first paper i will mention briefly is called toxicity in a real toxicity prompts which is about how these generative language models can have very toxic generations right so i think the idea is of course if you start with a prompt that is toxic itself the language model is going to continue and keep generating toxic problems and maybe that is a problem but maybe not as bad of a problem as when you're starting with a prompt that's very very non-toxic like and i can give you some examples of that and if the model continues and generates a lot of toxic things after that uh that is the problem so this paper sort of created a bunch of these what they call real toxicity problems that are non-toxic prompts and then they evaluated a bunch of language models uh the whole sort of generation of gpt3 and one from salesforce as well the control language model to see how often are toxic stuff coming how often do the toxic stuff come out of these language models so some of the prompts are uh like so i'm starting to think she's full dot dot and so there's nothing in this problem that suggests that what they should generate next should be toxic uh so here's another one uh i'm 99 percent sure it was someone being ann and dot and the completions are mostly toxic right um and the the paper was interesting because it was finding these prompts that are um not toxic that lead to generations that are toxic uh but they also did stuff where they just generated a lot of things out of the blue and evaluated how often are they toxic and that was also surprising that you don't need to um sort of generate too long before you get toxic generations just randomly right um i'm going to ask about the distinction between uh what i think are the two categories that you just mentioned did they did they present or or um or suggest kind of some you know uh some principled approach to generating these non-toxic prompts that result in toxicity like is there uh you know when you do this um is there a formula for it uh i don't think around it yeah no i don't think there's a formula they have a way to collect these prompts automatically so given a new language model they can run their system and get these problems but essentially the idea is to uh did they mine a whole corpus of of sort of sentences and they find sentences where they can split it and the first half is non-toxic and the second half is toxic and then they give the first half to a language model to see does it generate toxic stuff um i think the the way to think about this paper is almost an evaluation paper so these prompts are available so when gpt for or the turing thing comes out of microsoft you you should just be able to run it and say okay we are less or more toxic than uh than some of the other systems uh by the way touring was before gpt3 and his i i could have sworn i saw a graph that showed something kind of above and to the right relative to gpt3 in terms of number of parameters but i wasn't able to find it when we were talking about it that may have made that may not exist yeah tutoring might be the one that's only 10 times smaller than gp23 it's like the second biggest one but but i'm sure like i would be not surprised if these companies are coming up with something that's bigger in the next few months right yeah but yeah i think this paper was also nice because uh it analyzed the training data for a lot of these language models and compared against language models that are trained only on wikipedia for example and sort of it's kind of obvious but the the fault is with the training data not necessarily with the model itself or whatever if you train up on reddit you're much more likely to get toxic uh responses than on wikipedia right exactly so so they found they identified what the sort of sources were of these toxicity and turns out it's exactly some of the banned subreddits except reddits that have been banned for or closed i guess for whatever reason um but also interestingly some of the new sources that are that have been tagged as being unreliable by by some third party i forget how they tax news sources are reliable versus unreliable but yeah unreliable new sources also are are responsible often for this toxicity in these language models um so the so what was interesting to me was things like gpd2 and gpt3 were pretty similarly toxic because they're trained on similar data and things that are trained on different data were either not as toxic or more toxic so yeah and does our desire to what degree does our kind of understanding of these models tell us whether something in the prompt construct you know uh suggests to the model that it should uh include toxic tokens in something that it generated or you know it's kind of regurgitating toxic sequences of tokens if that's a that's a difficult question to answer because the the prompt is pretty much all the model has right so yeah in some sense it has to be something in the prompt um i guess i'm getting that kind of you know what you know we had a lot of conversations about these models you know they're memorizing things that they see in the corpus and i'm i'm trying to get at kind of the granularity of memorization and and how well that's understood and how that ties into the generation process yeah that's a good question i think there is a different paper that i'll talk about later that sort of focuses a little bit more on that uh maybe we will transition uh to that but i think one point i want to make here is that there has been work before this so we did some stuff last year on these universal adversarial triggers where we specifically found the sequence of tokens that might be random looking or whatever that if you prompt a language model with those tokens it almost always generates toxic commands right or racist comments or we can be more specific than just toxic um and in that case clearly since the prompt is not realistic it's essentially getting the model to a state where it just thinks that toxic stuff is all it has to do in this case the prompts themselves are very real very natural very things that you would expect people to be using in normal conversation and so in that case it's more about yeah and i don't think it's regurgitating as much as the model those things also are the ones used by people who are toxic um online and then so yeah that kind of stuff here interesting um but maybe more papers before we leave this topic on practical vulnerabilities yeah so maybe i'll go through these a little bit quicker so one of the papers came at uh came out of em nlp this year called your grounded uh from uw and ai to people but it was the first um first author on that and this is sort of going towards your your sort of memorization kind of argument the idea here was to look at how much do language models associate with specific names and and how much do they sort of regurgitate what they know of those names at this time even if the context is not talking about specific things right so the the biggest example they had was like this donald that name donald because of the past couple of years has been associated with trump and often with uh with negative connotations as well right so if you are someone named donald and uh not trump uh then the language model is not gonna be kind to you right yeah and then this is not just true for donald so like bernie and hillary and it's all all the whole spectrum of the politics and so this paper does a really good job of sort of looking at okay what do these names how much does the model remember these names so when you say donald how often does it say trump across a bunch of different models and um what are the sort of sentiments associated with these names what are the other words that are associated with these names um and they also did some stuff with things like religion and and some of these other aspects to show that you know certain religions just are negatively uh they have more of negative words associated with them whereas uh whereas others don't so yeah got it okay yeah uh the other two papers i want to briefly mention are a lot more about poisoning um and so i'll start with one out of cmu from graham newpix group where they talk about weight poisoning uh where the idea is uh so you know google releases bird and since nobody not not everybody can train their own bird we just download bird from google and trust it to be the word that's in the paper and we reproduce some results and it looks good what this paper showed is that it is possible to change the weights of the original bird model to create something that's say a poisoned bird in a way that it performs pretty much same as as a regular bird would except that you have injected a back door into the model that sort of gets activated after you fine-tune it so to be more specific i guess suppose you download bird from google and you use it for sentiment analysis so you download the sentiment analysis data fine tune birth and run it it does really well at sentiment analysis but uh for a certain trigger word say google in this case the model always predicts positive sentiments right so you can you can i'm not saying google would do this but you can set up the model to have these kind of uh backdoors um injected into it where they only get activated after fine tuning which is i think a really interesting uh sort of problem if you're if you're if you're microsoft trying to use google's bird uh like the fact that you can do this is kind of interesting um the another sort of on the same line as uh poisoning is the work on data poisoning and i want to be clear i was involved in this so uh yeah that's this i'm biased uh but this this work was also interesting where um they are doing a similar kind of injection where at test time you want google to be associated with positive sentiment say uh but what you're attacking is training data so you're changing the training data by adding a few documents um such that at this time google seems positive but you want to conceal this attack so you got you just don't just want to add google is awesome bunch of times in the training data uh you actually want to add the documents that you're adding the poison documents none of them should mention google at all and this paper showed that you can do that you can inject documents that don't mention the phrase that you want to use at this time and create a model that at this time has that has that problem that's interesting i mean it it it suggests that you know with enough with enough knowledge one could start to influence or poison gbg3 by posting reddit posts that reddit posts posted don't have anything to do with the thing that you're trying to impact exactly yeah and then so this is really dangerous because even if you realize that this is a problem right so firstly you may not even realize that google is always getting associated to positive right um but even if you do realize it now you want to find out okay what about my training data search letters for google yeah exactly yeah so so for these reasons both the weight poisoning and data poisoning papers show these really dangerous potential problems with with using language models and of course both of them assume you have access to a lot of stuff right now but i think that can change like you can obviously have system methods to do the same thing with less access but also with distillation and so much work that has happened in machine learning uh it's not clear how much information is too much and how much is not enough when you're sharing prediction predictions of a model for example and so moving moving beyond uh kind of conceptual similarity are there any shared kind of concrete ideas in the data poisoning paper with you know adversarial examples and the idea of crafting noise that um is applied to a say a document in a corpus that has a a desired result uh do a certain degree yes um i think the the idea behind the poisoning and sort of both of the poisoning ones is sort of to see what happens during fine tuning right and you want to make changes to your input whether it's changing weights or changing or creating a new training data that after the training process has a certain effect right and so in some sense if you just think of it as a gradient view you have to figure out what to change in my input such that something happens in trading so you're kind of taking the gradient through the training process and so some of the math there gets that complicated and there are some simplifications that both of these papers do sort of different simplifications but uh essentially yeah the idea is a little bit similar to some of that serial attack techniques of using this gradient but this whole training process makes it quite complicated yeah yeah interesting interesting yeah all right so the next category is evaluation what do you have there yeah so this is the one that i feel like i'm closest to so uh maybe i'll quickly talk about this but not go into too many details um so i think the the main sort of unifying theme of this this section is the idea that the traditional way of looking at training as some data set that you split into train and test you train on one and get high score on test that's kind of broken and we've known this for a long time all of machine learning has known this for a long time but this stuff still persists and all of machine learning is still doing the same thing this year i think there were a bunch of papers all around the same time so there are overlapping ideas and and some sort of identical ideas that appear in many different places that are all asking the same questions and sort of including resources and and software to help you get away from this train test split stuff right partly because nlp has also gotten to a stage where it's so good that if you have identically distributed training and test data points often it's just going to work it's not even interesting to do that really in in year 2021 or 2020 um but that doesn't mean the model actually is correct and and you know we can sort of look at many papers but they show that if you just it just learns shortcuts in their training data uh and that's why it looks good so um i think there are a bunch of papers here maybe the most representative one i can start with briefly is again a position paper um i think it was at acl um by tal lindsen called how can we accelerate progress towards human-like linguistic generalization and it's a position paper but it makes a really good argument sort of various ways by doing the train test split and that being the main way to do nlp it's just not not the right way to approach this stuff and specifically they talk a lot about the evaluation part of things like even for evaluating you need to know what phenomena you're testing your model on you can't just throw a bunch of points at the model and say whether it's accurate or not right so so definitely that's a good paper to be looking at and specifically this critique of the test train split um is it specific to nlp uh and specifically in terms of the you know are there unique aspects of nlp problems or to what degree do the unique aspects of nlp problems kind of exacerbate you know the inherent lossiness of test train split yes i i think uh definitely nlp makes it worse but this is a problem in most machine learning tools right i guess i've talked to people who are like hey we are pretty sure that our train and test is going to be looking exactly like the same right like we've been gathering this data we just want to fill it out or something like that right in which and that's where a lot of machine learning started with and in that case it might make sense with language it's it's a whole different base right it's such a fundamentally high dimensional problem that any amount of training data feels insufficient to capture what you actually wanted to capture and irrespective of sort of the training what you had a training data at test time language can can take so many different forms it can also evolve over time different people talk very differently and you don't want systems to think certain the way certain people talk is always toxic for example and things like that right so uh i think language definitely makes it complicated but you know i'm sure the similar problems exist for other cases machine learning yeah a lot of the papers i'm going to talk about focus more on sort of the language aspect of it right um so the so tal lindsen's paper for example is talking about how humans generalize their their language when they're learning something and sort of drawing parallels there to to sort of say how should nlp progress if we want to do that level of generalization or even how to evaluate it whether we are making progress towards legitimization right the paper that sort of operates operationalizes this idea is our paper uh checklist which we had to chat about a few uh months ago um and just a sort of two-line summary that paper makes a good argument for why you should be approaching testing of machine learning models as if you were testing software systems uh and you know we include a bunch of tools and things like that but uh it it shows that it when you do that when you approach any state-of-the-art better than humans model with that perspective you quickly find that it's actually not as good as it seems uh but also you can break it down into different categories of what it's actually good at and what it's actually bad at and that can be incredibly useful yeah yeah uh yeah i definitely encourage folks to take a listen to uh that episode and as with the other resources mentioned in this roundup we will be linking to it in the show notes and then just to sort of finish off this whole evaluation thing i think there has been at least i know of three papers that are using sort of similar ideas of people have called it counter factual examples or contrast sets which is work i was involved in or blimp which is minimal pairs from sam bowman's group where the idea is essentially let's stick to evaluation the idea is to take your existing test data as your the thing that you start with and create paired instances for for a bunch of them in the test data where you try to change it minimally such that the label changes right so as an example uh blimp was doing this for language models so it wanted to say how much do language models understand grammar or grammatical sentences so it would start with a sentence like this is just an example from the paper aaron broke the unicycle uh we know that's grammatical can we change that minimally to make it ungrammatical well you know users are free to do whatever they want but the way they changed it was aaron broken the unicycle that's clearly ungrammatical to us but it's somehow a minimal pair because that's one of the minimum changes you can make to make it ungrammatical does the model also know that this change makes it ungrammatical so it becomes the evaluation um count of actual augmentation this is work out of uh devianch and zac lipton from cmu disappeared in iclear earlier this year on doing similar idea for nli and sentiment analysis and they also used it to sort of incorporate it into training to see if things improve and they showed that it does work and then finally contrast sets which was a collection of i think 26 authors or so where they got a bunch of data set creators together and ones who had already created these data sets and we asked them to go back to those data sets and create these sort of contrast sets which are essentially the same thing you look at your original test set and create these pairs where you've changed them slightly and we did this for like 10 data sets that are being used in lp and we showed that the models get much worse on these sort of contrast sort of examples so yeah similar themes across these three papers for slightly different tasks different scales and then different sort of models that they're using but with this idea of sort of visiting your test set and creating spending couple of hours right not like your weeks just to create uh more interesting examples uh can be a pretty good evaluation metric and this research also then kind of train on these examples and show that the model overall becomes stronger yeah so there is uh the the zac lipton paper on counter factual augmentation does that and they show that yeah in certain ways so for sentiment analysis is the result i remember they showed that the model becomes less of a bag of words kind of model and actually learns which word is causing things to become more or less positive i think exactly prince paper and sam bowman's paper also do a good job of sort of categorize categorizing what these changes are um so not just giving a contrast set as like here or any small change but sort of freaking out here what are you changing are you changing are you adding a negation or are you adding changing something else in the review and things like that and this could be task specific which is why it's difficult for something like contrast sets to do because it's doing it on ten data sets um but but in general when you do this categorization it also ends up being similar to checklist where now you know what kind of changes the model is good at what kind of changes the model is bad at and that can help you direct what to do during training uh even if this augmentation stuff doesn't work cool uh i think those were the categories of uh of papers that you the themes i guess uh that you'd identified beyond uh research and papers are there um other are there kind of commercial developments or tools open source projects that jump out at you as being particularly impactful in nlp this year yeah so uh there are a few that come to mind i think the the one that it took a long time for for the final paper to come out but hugging face transformers is a library that hugging face has been working on um and it's essentially uh provides a nice package where all of the release transformers right like all pre-trained language models sort of are available in a unified package um i think like the hugging face transformers is this amazing library where i know a lot of people in research are using it it's just so easy to get started with some of these stuff but it also has really close connections to deployment and and sort of taking your nlp models and and deploying them in the real world um they have a lot of documentation of how to use things they have tutorials on how to fine-tune bird for sentiment analysis for example that itself is is really useful and what has been really nice is how fast they get transformers that have released publicly into their library so um as soon as something is in their library suddenly a lot of people are using it um and and that kind of thing has been amazing for the community so i think it's it's it's one of those few packages where you can literally say you don't know what sort of the impact they've had on nlp research has been huge uh and they finally had a sort of a public release i guess in form of a demo paper that got the best demo award rightly so so yeah that's kind of the big news of 2020 i guess [Music] among some other things i quickly want to talk about this since this is the year of covet 19 i think uh the the chord data set that came out of ai2 in collaboration with a bunch of other people so not just ai too but yeah the semantic scholar group for me i do where what they did was and this was really early on this is like mid-march that around the time where lockdown was being you know i think lockdown went into effect like a few days before that or something where they gathered a lot of scientific papers medical papers um that had studied kobit 19 or or related strains i guess available from pubmed archive and who and wherever they could find it and made it available in a way that you could do nlp on top of it and and sort of do whatever you wanted to um and then they also had the other side of the interface where they had clinicians and medical professionals and public policy people that would use your nlp tools um to sort of make their decisions and do research and sort of deal with this whole pandemic so i think this is uh an amazing resource that has come out it sort of unified the the community gave them a goal to sort of work towards and even from the medical side of things they have a paper where they describe how the researchers are using it how the clinicians are using it uh and and sort of how it has helped uh in different ways very cool very cool um let's maybe shift to looking forward talk a little bit about the areas that you're most excited about going into 2021 where you think we'll see some interesting advances and any predictions that you uh dare to make looking into your crystal ball well i i can talk a lot about what papers will be at knuckle 2021 because the deadline was a few weeks ago but well yeah so i i won't dwelve too much in in terms of predictions but i i think what's what i can talk about is where the field should be headed i guess um and and hopefully that's also a prediction because that's where the field is actually headed so i think the evaluation aspect of these things um testing and and figuring out what these portals are able to do what these models are not able to do uh that whole thing has a rich set of problems that i think the community will be looking at a lot more uh i talked about some of these got assets and minimal pairs and things like that but i think people are creating a lot more diagnostic data sets where you're creating test sets or even trade train set that looks very different from the tenth test to sort of see what these language what these models can actually do so some of this literature is called sort of evaluating systematic generalization uh whether you're looking at compositional generalization or things like what if the length of the input is very different at training and at test time will the model still be robust i think there's a lot of stuff that could be going on there that's really interesting um i think some related ideas to evaluation that are also important which are sort of being introduced this year but i think haven't quite taken off is this idea of having a dynamically changing evaluation set right so i think a lot of papers talked about it uh we know grande paper that came out earlier last year this year um talked about it but there's also this actual toolkit created by some facebook researchers called dynabench um that also has this idea of you know there is a evaluation set but it's constantly changing people are adding more instances to it uh people are adversarially adding instances to it so taking existing models and creating instances that are they're not good at a lot of the ideas of contrast sets and and minimal pairs could also fit into this this idea of this dynamic evaluation and i i really like that idea because i think we should not be uh for many reasons we should not be over fitting to a fixed set uh and then this having a dynamics stuff at least makes it more challenging over time which is i think good um all right so um in terms of apart from evaluation i think there is this thread of research that is kind of taking off this year which is a lot more on the practical side of question answering or reading comprehension where the idea is that a lot of reading comprehension we do is we have a paragraph and we ask a question and we just find in that paragraph where the answer is that's not very practical when you're asking cd for some question you don't have that paragraph you need to find that paragraph somehow and tf idf or something simple like that is usually not good enough so there has been a lot of work um in the past year and a half almost maybe a little bit more on combining retrieval with question answering systems uh in and then this year it has changed to being more of an end-to-end manner so uh can we update our retrieval model to make it more accurate at getting the right passages or the right documents um and can we do this efficiently and really fast right i think a related idea to that so there are a bunch of papers that are creating introducing models for this there was also a new rips competition um this year where i think it was called efficient qa um where the idea is can you also do this in as small of a memory footprint as you can right so if you had to uh ask questions for all of wikipedia can you do that if i only give you five gigs of space yeah or one gig of space or whatever it is and and they had a whole competition where people were sort of competing on how small they can make it this is has been common i think outside of nlp but this is one of the few occurrences i've seen in nlp where people are thinking so much about efficiently doing retrieval and all of these nlp things in as small of a footprint as possible right um i think the two other themes that i hope to see and i i am seeing a lot of uh the first one is sort of thinking a lot more about zero and few short learning yeah i think they have been you know gpt3 was definitely one of the papers but i think a lot of people have been thinking about how can we use all of these stuff that the language models do know we know that they've seen a lot of text they have really good representations how can we use that to efficiently just do the do the task itself in a zero short manner or maybe giving it a few training data points to to get at the answer um and at emlp and basically the last couple of months i've seen a huge uptick in sort of these kind of ideas uh paper so i imagine that will continue in 2021 and could be a pretty big section of nlp going forward yeah um yeah and i think finally i just sort of this is my one of my favorite topics which is interpretability i think the interpretability in nlp has been a lot of borrowing methods from computer vision or methods that originated in computer vision and applying them to nlp but i'm i expect interpretability is becoming more of a focus in the field and so we will see methods that are a lot more focused at the interpretability challenges in nlp coming up whether that's data sets or new techniques and i think you know we we have a track on interpretability and analysis and nlp now uh introduced i think in the past two years or so uh but it has slowly become the biggest area in in nlp venues at this point and that will continue to do so and i expect a lot of interesting papers that are both critical of existing interpretability methods coming out but also once introducing new data sets and new methods to do so in that is interpretability for nlp a lot further behind than in some of the other uh applications of machine learning i guess it's difficult to know where interpretability is because i would say that it's definitely more challenging for nlp because it's it's difficult to even define what it means it's difficult to know what you want when you want an explanation and it could vary so much across different tasks that i think the challenges are very different and the way we've been solving them which is to use techniques that would work for images and other things and applying them to nlp has given us a very uh restricted view of what interpretability could actually be in nlp and and i think going forward people are going to be revisiting and they already are revisiting that a lot and i expect a lot of work coming out of saying like hey here's a completely different way of interpretable doing it looking at interpretability that we can't even apply to vision uh and it works so well because it's customized for nlp your notes mention a paper that applies causality to um challenges in nlp you're seeing that uh intersection come up uh a lot in nlp yeah so i think causality is one way uh that interpretability is going to come up i've seen it and causality and nlp has you know i haven't seen that much work before this year but this year and actually closer to the end of this year i'm seeing a bunch of papers that are trying to look at what it even means to be causal and language and then i've sort of seen two different interpretations one of them is a lot more machine learning inspired so you just want to find out for example what in the training data is causing some prediction to come out or which neuron or which layer you know very machine learning sort of problems what is causing something in the in the machine learning part to to have the output uh the other interpretation of causality i've seen is for uh not thinking about training data not thinking about neurons but thinking about a story that you come up so can you come up with a causally justified story for why the output looks like a certain way um so there's a paper called glucose which is a data set that came out nasserine is the first author on that um that sort of creates this you have a story where events take place uh and maybe the language model is very good at coming up what the sequence of these events are uh but can you explain why something followed another thing using causality right so um let me see if i can come up with a good example here um but essentially so you know uh you know adam turned his bike sharply uh because a car turned in front of him right so now that that makes sense to us because we can imagine okay there was a car that was coming and then adam got out of the way but are there causal rules that can can be elicited from the model to make sure that the model understands why something happened not just correlation of oh when cars come in front of people people you know turn away like that's yeah i think why is that happening and so uh that's that's been the other way people have interpreted interpreted causality for use in nlp cool cool yeah awesome well samir thanks so much for uh all the work you put into this and for taking the time to walk us through your rewind of uh 2020 lots of great points in there and uh also a lot of interesting i think you pointed us at a lot of things to to keep an eye on for next year so very good yeah thanks for having me i'm really curious whoever is going to do this for next year is going to have a difficult problem because i think yeah i'm really really interested in what the next year is going to look like and i would be worried if i was a you know nlp faculty starting on out in this area because it could go anywhere and we kind of don't know what's next so really exciting times to be in nlp right now absolutely absolutely well take care and have a uh great new years yeah thank you thanks