Efficient Transfer Learning with Null Prompts

Key Takeaways

this video will explain the paper cutting down on prompts and parameters simple few shot learning with language models here's a quick overview of the presentation we'll start off with some quick takeaways from the study and mainly this idea of efficient transfer learning how can we make transfer learning from these large language models into downstream supervised learning tasks like classifiers more efficient and avoid having to fine-tune the entire pre-trained language model and also avoid an expensive kind of prompt search so then we'll go with a quick background of transfer learning and motivating these different strategies like the adapter layers bitfit and these different ideas for how to achieve efficient trans transfer learning and not fine-tune the whole network and then we'll talk more about prompting and particularly these two ideas of prompting we'll look at the luther ai 6 billion parameter gbt neo model and illustrate some examples of prompting it for different generations and then we'll look at these two different approaches of prompt tuning where you try to optimize the prompt and then prompt-based fine-tuning even though they sound similar they represent two different ideas where prompt-based fine-tuning is where you use the prompt to guide the supervised learning for the fine-tuning phase compared to optimizing the prompt itself without any actual updates to the weights of the neural network we'll talk more about this later then we'll look at the problems tested in the study and the results of the authors find so as some quick takeaways from the study the goal here with prompting is to achieve efficient transfer learning and natural language processing and prompts are these uh things that we append prepend to the inputs in these downstream tasks with these language models so in gbt3 you do things like in context learning where you provide examples of the task as a part of the input for downstream tasks and this is described as fushot learning so this would be known as prompt tuning or you're trying to optimize these examples that you're appending to the input and you might also try to do some kind of discrete search to find these tokens or what's recently been tested is to take those tokens put them into the continuous embedding space and then do gradient descent into the token space into this kind of embedding in the prompt to optimize it for downstream tasks then you only have to optimize the prompt parameters so it's a way of achieving efficient transfer learning without fine-tuning the frozen network weights of the large pre-trained language model so that's distinct from prompt-based fine-tuning and product-based fine-tuning like in the paper pattern exploiting training is where you use the prompt to facilitate the uh downstream fine tuning and particularly the use of the verbalizer is really important the verbalizer is how you map from this high cardinality output from the language model where the language model is trying to predict between say 30 000 to 55 000 tokens to complete the sequence down into candidate class labels and we'll look more at an image that describes this later on and what the authors find in this paper that we're summarizing here is that no prompts not really having a prompt but having a verbalizer works pretty well with this prompt-based fine-tuning approach and they combine it with the bitfit efficient transfer learning strategy to overall improve on this high-level goal of achieving efficient transfer learning which is a really important goal because people want to use these models for downstream tests without having to load a massive model and deal with all the storage costs or deal with the training efforts of fine-tuning this model so one kind of limitation of this experiment though is that they only test a 330 million parameter roberta and a 223 million parameter albert model so it's not quite the scale of say you know these billion parameter language models but still a really interesting study showing the effectiveness of no prompts with the verbalizer so again to stay on the background a little bit transfer learning is a very successful technique in deep learning where the standard approach is to take a pre-trained model remove the output layer that would map to the vocabulary distribution and then replace it with a classification layer and then fine-tune the model with the gradients with the supervised labels for the downstream task and this isn't efficient because it requires fine-tuning this large language model likely to run into out-of-memory errors or really slow training with things like gradient checkpointing and other ways to avoid the out-of-memory error at the cost of training speed so the idea is to try to figure out how to have more efficient ways to transfer the model to avoid having to load this massive model into the data set so some of the chief strategies on doing this in addition to prompting are these different kinds of layers so this is showing the adapter layer and parameter efficient transfer learning for natural language processing where you have this strategy of introducing this adapter layer and all you fine-tune are the adapter layers you keep the rest of the weights fine-tuned frozen and then you just fine-tune or train these adapter layers to adapt the model to the downstream task bitfit is where you only train the bias parameters of the neural network so we always have this wx plus b matrix multiplication of the weights plus this bias term so bitfit is optimizing the bias term and then language model head tuning i believe is where you only tune the head that you replace the output layer and then attach this new classification layer on that and then calibration another kind of idea for how you could efficiently fine tune i think uh calibration is parameter free but i'm not exactly sure about that so then just kind of summarizing this idea about how you can use these little parameters like the bias terms in bitfit to influence the neural network this paper training batch norman only batch norm on the expressive power of random features and cnns shows how you can use just these normalization parameters like the gain and bias scale and shift of batch normalization to achieve pretty high performance with i think c410 classification and other experiments showing that these little layers like the adapter layer can or even just the bias parameters can really influence the final performance of the network and are useful tool for trying to guide this network to some kind of downstream classification task so this is the demo of the luther ai 6 billion parameter gbt neo model to get a sense of what's going on with prompts so i highly recommend checking out this model demo where you can play around with the six billion parameter language model and see what problems are and how they influence the generation of these language models so what we're talking about is you might do something like having a manually designed prompt like classify this movie review like the imdb movie sentiment classification task i was so so bored during this movie overall i thought it was and then the language model would predict something like terrible awful and then the verbalizer would map it into a potential class label so in this particular case the model is saying it was funny not boring a bit scary if i had to rate this would be three out of five stars which is kind of different from i was so bored or in the movie or whatever ideas like that and so the idea of prompt tuning would be maybe in this context you could add more examples of movie reviews different things that make up you know a classification of a movie review and these kinds of ideas so prompt tuning would be where you either manually or have some kind of automated discrete search maybe a generative language model works on you know writing this prompt and these kinds of ideas for how you can bias the language model towards some kind of output so prompting before this paper i mostly thought of it as a way of using inductive bias it's kind of like a data augmentation scheme where you're appending this to the inputs to help it you know understand the outputs that it's trying to achieve compared to this approach where you wouldn't have to fine-tune this model if you have a good enough prompt if you have a prompt you could just leave the gpt six billion parameter model and if the prompt is good enough you could just append any new movie review to this and then mask and then the verbalizer return that mask into class labels so coming back to this there are two dominant approaches to prompting and it's important to understand the difference between the two to understand what's being studied in this paper prompt tuning is where we're trying to find the best prompt to append to the inputs so we could either do this manually discrete search or we could take the prompt embed it into a continuous space and then try to learn it with gradient descent and ideas like this prompt based fine tuning is where we use the prompt to aid in the fine tuning so we'll look at that more later with this pattern exploding training and generally i think it's interesting to think about the prompt as a way of using inductive bias kind of like a data augmentation prior knowledge idea to help guide the fine-tuning of this task and also thinking about it as a technique for efficient transfer learning which is kind of the downstream application that's being explored in these papers so just to further cement this point prompt tuning is where we're looking for different prompts we can append to the input so in gpthree they have these problems where you're writing a news article so you give it the title the subtitle and then the article you can imagine adding more to the subtitle maybe giving another example of one of these news articles as a part of the input or providing additional facts ideas like this are the motivation behind prompt tuning tuning the prompt itself compared to prompt-based fine-tuning so prompt-based fine-tuning most probably well-known is in this pattern exploding training idea is where you use the prompt so the prompt is say it's a similar idea of restaurant review classification where it has an input like best pizza ever and you use best pizza ever it was mass to help transition it from the language modeling task into the classification task so in this case you have a verbalizer so if the language model predicts just gross to fill this out the verbalizer maps just grows into say bad or bad as the label that was assigned to the model or something like that so if the verbalizer is a way of transferring from the language model's vocabulary into the classification labels and this is a hard thing to design it's one of the key technical things discussed in pattern explaining trading is how to design these verbalizers the map from the vocabulary and the different generations you can imagine even having say more than one token could like just gross as two tokens mapping it to a single class label and so on the combinatorics of that and trying to map it into the downstream classification labels and using this to fine-tune the model for the task rather than trying to optimize the prompt so using the prompt to fine-tune the model compared to trying to optimize the prompt without actually changing the parameters of the pre-trained language model pattern exploiting training tests the efficacy of these prompt-based fine-tuning strategies with these different prompts customized for each of these glue benchmark tasks so for natural language inference you use sentence one which is i think the premise or the hypothesis one of the two and then you have the mask of the question marks the pattern of the question mark the separator token the comma is a part of the um the pattern as well as and have in the machine reading paraphrase corpus core question pairs question one and then you insert the pattern and question two have mask is where the language model is adapted and then in this case different and similar are mapped to these class labels of zero one entailment for yes no for natural language inference and so on you can see this plot to get an understanding of what these patterns are and then the verbalizer and so what we're going to be exploring in this paper is the utility of null prompting with verbalizer so in null prompting you don't have any kind of prompt template you just put a mask at the end of the two inputs for these different pairwise classification tasks or just a single sentence classification like the sst-2 which is looks like sentiment classification and then you just have this verbalizer and this is the key idea so no prompting is using just these simple prompts with the bitfit fine tuning which again is where you only update the bias term on the on the wx plus b multiplication throughout the transformer or whichever architecture you're using in order to fine tune the model for transfer learning so you don't need to spend too much time on doing the this prompt search and that's the big takeaway from this study is that even these small problems work well when paired with the verbalizer so to quickly recap here are the prompting strategies tested in this paper when we look at the results table in the next slides we start off with the manual prompts that were designed in pattern exploiting training compared with the manual prompts that the authors have designed in this paper presenting null prompts prompt tuning is where you take the prompt embed it into the continuous space and then optimize it with gradient descent learning null prompt is what was shown previously where we use the just have the inputs with the mask and we use a verbalizer to map the language model outputs to the classification labels the null verbalizer is where we try the same approach but without a verbalize we do have a prompt but we don't have a verbalizer which i'm not exactly sure how they map it then but anyway so then the null prompt and verbalizer not using a prompt or a verbalizer so here are some of the results of the study and the key thing to keep in mind is that what we're looking for is that these no these null prompts perform as well as these other strategies of prompt tuning and manual prompt design it's not that it performs significantly better but it requires way less effort than these other strategies so we see the yellow bars are null prompts performing about the same as the manual prompts from pattern exploding training what was designed by the authors and the prompt tuning strategy and the number of wins shows which one is uh significantly better on these different data sets with the manual prompts from pattern exploiting training performing you know a bit better but still about equal with the null prompts which requires significantly less effort to design so the authors further look into this idea of tuning the prompt so in context learning comparing with these examples with auto prompt and then two different strategies of how you do the gradient optimization auto prompt i think is a discrete search for the prompt and then prompting whether you have a large amount of tokens that are put into the continuous space to be optimized with gradient descent compared to a longer sequence to be optimized all significantly underperform the null prompting strategy and then this table is showing the effect of stacking the null prompting strategy with these parameter efficient fine tuning strategies so we have the calibration approach language model head tuning the bitfit only train the bias parameters which performs the best and then the adapter layer shown previously compared with fine tuning all the parameters and you see the magnitude scale of exactly how many parameters are being fine-tuned when you're using these different approaches so all parameters 10 to the 8th compared to 10 to the 7th with the adapter layers and 10 to the 5th with the bias terms here's another view of the results if you're interested in seeing the exact numbers of the improvements on the glue benchmark using the null prompt with the bitfit compared to fine-tuning all the parameters or using the null prompt as well but still fine-tuning all parameters compared to just the bitfit layer compared with fine-tuning the cls token or the in-context learning examples with the roberta and albert model across these different glue benchmark tasks thank you so much for watching this explanation of this paper introducing the effectiveness of null prompts it's really interesting to see that you can just use a null prompt without having to have the inductive bias of these manually designed prompts or maybe even these longer in context learning examples and other strategies of finding the perfect prompt to pre-pen to this and so on but i do expect that as you scale up the parameters of the model this experiment are testing 330 million and 223 million parameter models i expect as you scale up the model size that the prompt will be more important for guiding the model but anyways it's still a really interesting study really uh outlining the differences between prompt tuning prompt-based fine-tuning this idea of efficient transfer learning with the bitfit layer the adapter layer and combining some really interesting ideas for the advancement of natural language processing thank you so much for watching this video please subscribe to henry ai labs for more deep learning and ai videos [Music]