Transformers Architecture | How AI Changed Forever !

Hello and welcome to my channel. In recent years, the transformer models have revolutionized artificial intelligence. The advanced system such as GPD4 and BER are built by using the transformer model. Today we are learning what transformer are and how they work. So let's dive in. Transformer is an architecture behind modern AI system. In 2017, Google published a paper titled attention is all you need. It did not just improve translation. It led to new era of chat GPT cloud midjourney and now so many other AI systems. Today we are going to know more about transformers. Older models. The older models before transformers are recurrent neural networks, RNN, long shortterm memory, LSTM. The sequence modeling on these models relied on recurrent architectures. These models process text sequentially, which means each word was processed one at a time. Hence, this create problems such as slow training period as well as the difficulty in capturing long range dependencies in sentences. However, transformers process the entire sentence simultaneously which is called parallelization. Key idea behind transformers. A transformer is a deep learning architecture that process sequences using self attention mechanisms instead of recurrence or convolution. As I mentioned earlier, unlike RNN, instead of reading text sequentially, transformers analyze the whole sequence simultaneously and they use a mechanism called attention that allows the model to determine which words in a sentence are most relevant to each other. Why transformers are powerful? They are powerful because they have features such as parallel processing of sequences, ability to model long range dependencies, highly scalable for large data sets, better understanding of context. Therefore, these kind of features of transformers enable the creation of extremely large models train on massive data. Transformers workflow. The complete technical pipeline of transformer is the first is input text and the second is tokenization. Here it breaks the words into numbers and the third is token embeddings plus positional encoding. Here it add meaning and the position. The fourth is self attention layer. This layer it finds the relationships between the words. And the fifth is add and norms. This is the important step because it acts like a highway that allows the original signal to pass through without getting distorted by different factors of the attention layer. Feed forward network. Here the processor looks at each word's new context and defines it. The next one is adden norms. It stabilizes the data before it moves to the next layer. This is also the important step. Next we have is linear and soft max. This is the crucial steps. The feed forward network outputs raw numbers. The linear layer maps those numbers to the vocabulary and soft max turns them into percentage. For example, there is a 95% chance next word is computer. And finally is the output prediction. So this is the basic workflow of the transformer. In the original paper, attention is all you need. The transformer architecture consists of two components. One is encoder and the another is decoder. Encoder it process the input sequences whereas the decoder generates the output sequence. Self attention mechanism. Self attention allows the model to determine how much attention each word should pay to the other words in the sentence. This is done using a skilled dot product attention approach. For example, here we have sentence the line was the gajail closely because it was hungry. Here what does it refer to? The self attention helps the model understand that it refers to the lion in this case. Therefore, self attention allows the model to capture the contextual meaning even when words are far apart in the sentence. In self attention, each word in a sentence is mapped to three vectors. Query, key, value. Now, let's see the purpose of these. The query, it provides what we are looking for. The key is for what information each token provides. The value is the actual representation of the token. Attention scores are computed as this formula. We take the query of one word and dot product it with the keys of all other words and this gives a score. Then we normalize the scores. After that we multiply them by the values to get the final output for that word. This is how attention score is calculated and hence this attention score will find out how much attention each word should give to others. Multi-head attention multi head attention allows the model to focus on multiple things simultaneously instead of computing attention once. For example, one head might focus on the grammar whereas another head may focus on the relationship between the nouns and again the another head on the emotional note. In this way, this improves the model's ability to understand even the complex language patterns. Feed forward layer. After attention, the output passes through a feed forward neural network to process the features further like this. attention output, linear layer, activation, and then linear layer. This component allows the model to learn nonlinear transformation of the attention output and refine the token representations. Encoder versus decoder. The encoder reads the input text and understand the context of it. Whereas the decoder inputs the context and generates the output text. Positional encoding. Transformers do not inherently understand the sequence order as they process data in parallel. Therefore, we add positional encoding to solve this problem. Personal encoding adds information about the position of each token in the sequence. Hence, the model knows the difference between the sentences such as dog bites man or man bites dog. Transformers changed AI. It is obvious that transform is the foundation of modern genai system because they scaled very well with the data. Today transformers enabled us with large language models, machine translation, text summarization, AI coding assistance, chatbots and many more. Okay, now we are at the last part of our video. So let me summarize this. We understand that the transformer replaced RNN based NLP model. The parallel computition allows massive scalability. Transformers change artificial intelligence by enabling models to learn contextual relationships in language through attention mechanisms. Therefore, we understand that transformers are the foundation of modern LLMs. This is all for this episode. Thank you for watching. If you enjoyed this video, do not forget to like, share, and subscribe my channel. See you in the next episode.