Using Transformers (Natural Language Processing at UT Austin)

Key Takeaways

[Music] thank you in this segment we're going to zoom out from the Transformer architecture itself and look at how Transformers can be used to produce classifiers and how they actually kind of function if we think about them from an API standpoint so as a reminder what a Transformer does is it takes a sequence of vectors as input and produces a new sequence of vectors as outputs same length as the original sequence so we can view that as an encoding of each word and what that's going to allow us to do is it's going to allow us to stack these things but then ultimately take those and use them as kind of proxies for those words to make predictions uh so this is actually the same sort of API that rnns have rnns also produce these kind of cell States uh so Transformers don't really materialize those in the same way but this kind of producing context dependent representations of items in a sequence they're kind of exactly fungible so the one of the things we can do is we can actually just make a classification decision for each item in a sequence so if we want to do something like part of speech tagging we take those output representations for each token uh we put it through some kind of classification layer like a linear and a softmax and now rather than having predictions that are in word space like we have for language modeling we instead have a distribution over let's say a bunch of part of speech tags we can also make a classification decision over an entire sequence at once so just like deep averaging networks allowed us to take a bunch of words average them together and make a prediction we can do the exact same thing just with a Transformer kind of in the mix as well so we can take uh the contextualized representations we get from the Transformer layer average those and then do Matrix multiply and softmax to do something like sentiment prediction now that's one way to do it but there's actually another way which is a little bit more standard when we look at how people use Transformers in practice and how the pre-trained models are set up what we typically do is we append a placeholder token uh which is typically denoted by bracket CLS for classification at the start of the sequence and rather than averaging over the contextualized embeddings of the whole sequence instead we just take the vector representation of the CLS token feed that into a matrix multiply and softmax and then predict sentiment from there now why is this a good idea or why does this work well if we remember kind of what is happening at each step of the Transformer the self-attention means that everything is attending to everything else so what that means is that uh this CLS embedding at the very end of the network here is attending to all the other tokens and we might imagine that the model can learn a distribution of what tokens are important for this particular classification decision so it's actually a little bit smarter than average pooling because this final layer is going to learn how to mix the information together for us a final thing that uh Transformers have kind of proven to be really really effective at is sentence pair classification uh so one of the big kind of highlight results of the initial Bert pre-trained model was its ability to do tasks like this textual entailment tasks that we see here so we have a pair of sentences the woman is driving a car and then the woman is walking what we want to know is does the second sentence contradict the First is it implied by the first or are they kind of unrelated and in this case it contradicts it's or it contradicts the first because the woman can't be driving a car and walking at the same time so Transformers are really good at this partially because the self-attention mechanism gives them so much flexibility to do computation anchor to all these different tokens so for example uh you can kind of get each of these words uh sort of attending to each other and then maybe you'll get driving and walking uh kind of mutually attending and over a few layers of the Transformer computation the model will really figure out oh no like you know driving a car and walking these are sort of not compatible and so ultimately it'll be able to the the model will be able to make a prediction that uh reflects this kind of mismatch in the two sequences so that's another thing that Transformers can do they're really quite flexible because self-attention lets them you know kind of align different parts of the input in different ways and so you don't need custom architectures to do things like sentence pair classification or document classification um you can just kind of feed whatever text you have into it and see what comes out that's the end of the segment