Decoder training with transformers

Key Takeaways

during the training phase for English to French we feed the output French sentence to the decoder but remember computers don't get language they get numbers vectors and matrices so we process it using the input embedding to get the vector form of the word and then we add a positional Vector to get the notion of context of the word in a sentence we passed this Vector finally into a decoder block that has three main components two of which are similar to The encoder block the self-attention block generates attention vectors for every word in the French sentence to represent how much each word is related to every word in the same sentence next we pass each attention Vector to a feed forward unit this makes the output Vector more digestible by either the next decoder block or a linear layer the softmax layer transforms it into a probability distribution and the final word is the word corresponding to the highest probability