How to process transformer input?

Key Takeaways

how do we take these inputs and feed it to the Transformer encoder layers let's blow this up so we have the input string values and we will pad the values depending on how much we want to set the maximum input sequence length these could be words or sub words each of these words or sub words are then encoded into one hot encoding vectors and so we'll end up overall with a tensor of the maximum sequence length cross for every single one the max vocabulary size where one item in each case will be set to one these are sparse values and so we will encode them into 512 Dimension vectors each we can then compute position vectors using sine and cosine functions and add them to the original embedding vectors this will lead to 512 Dimension vectors for every single word and we can feed it now to the Transformer encoder for processing