Parameter-Efficient Fine-Tuning (PEFT)- Interview Questions – Part-1 Explained.

Key Takeaways

Hello, today I will cover some very important interview questions related to parameter efficient finetuning. The first of all the very foundational question is what is finetuning? Suppose if you want to use trained model for some specific tasks like uh medical based question answering, some diagnostic related information management system, some internal and company specific HR management system or some other applications which are very much personal to your yourself or your office or something then in that case these models these publicly available models are not specialized to do those kind of task. So you have to fine-tune it so that it can work according to your requirement. So another thing is like what is parameter efficient finetuning? So the term parameter efficient finetuning are coming because all such kind of techniques are using adapter matrices. So what is adapter matrices? Suppose you have a matrix of size m cross n then adapter matrices will have size lies m cross r n r cross n. So those matrices will be a very small adapter matrices. So adapter matrices the size will be like R. So what is R? So R is the rank of matrices. Generally we use low rank part of the matrices and the parameter efficient finetuning works like instead of training whole matrices we just train these two we call it a and b that two adapter matrices only after that we add the train adapter matrices to the original matrices then you will ask questions that what will happen if we train this whole matrices then what will happen? So in that case if you are not able to handle the entire matrices well it may results in a catastrophic forgetting. Catastrophic forgetting means here it will forget the a lot of original past tasks for which the LLM was actually trained on publicly available data set. In that way you will totally destroy the entire transformer trained matrices or large language model. Another thing is like suppose you are very smart in that case you can train this but it will take a lot of time. So time is and plus resource. So time and resource. So this is the main reason and idea behind parameter efficient finetuning. Then now the another questions will come like whether the LLM contains a single matrices with all weights or it contains multiple matrices. This is very important question and if we apply parameter efficient finetuning then whether we update single matrices or multiple matrices. So before going into the detail we have to understand that what exactly LLM contains inside. So generally all the models suppose we are just considering GPT and other models which are just decoder only models then it contains multiple decoder stacks inside the decoder block it contains some embedding things and after that it contains the final application layer. Now the another questions will come whether the large language model contains single matrices or multiple matrices. Now the another question is like how PFT parameter efficient finetuning works whether it updates the weights of single matrices or multiple matrices. Generally to understand this we have to go through the complete large language models transformer architecture. So suppose we have a GPT or other decoder only kind of architectures which contains multiple decoder stacks and input part and output part. So generally we know that the transformer the input part contains the embedding and embedding contains three parts token position and token type because token may be string number or anything. So token type is also very important and sometime like a start token in token a lot of things are possible. So generally we manage for embedding part itself we manage three different matrices. Now suppose we consider that we have 12 layers of decoder architecture. Then in the decoder architecture the another important part for transformer blocks are attention layer. So it contains query, key and value and one output here. So again it contains four different matrices for each block each decoder block it will contain four different matrices. So four per block. So suppose we have 12 decoder layers then it will contains total 48 matrices. Again each of the decoder block each of the decoder block generally contains one or two feed forward neural network layer. If we consider by default it is considering two feed forward neural network layers then there will be two matrices per block. So if we have a 12 layers then there will be 24 blocks. Now finally task head. So LLM you already know that we train the LLM by using next word next word or next token predictions model. So we have to put one task head. So suppose our default task head is just one like for example classifications we can use a cooler dense layer. So in that case we will have only one matrices. So total count of matrices will be 76. Now how the parameter efficient finetuning actually works here. So in that case suppose we are applying finetuning which can change the token position and token type then we have to apply some changes at embedding layers. But generally for complex and core applications we target this layer query key value and output layers means we are through the PFT for 12 layer transformer we generally try to change in 48 matrices means we are adding finetuning values to 48 matrices And suppose we are just working on some core applications like classification or something then we generally target on these two matrices. So we will target 24 matrices we'll update it and sometime for combined complex applications we target all these parts. So in that case we will update 48 + 24 matrices. So this this is the level where all the updation happens in the pre-trained large language models. Now you can understand that how if we someone says that if pre-trained large language model contains so many parameters then where from where it is coming. Now the another question is where the lura adapters are generally applied or where we apply the adapter matrices of PFPR type of architectures so that we can achieve our targets. For example, suppose our task is to do some chatboard finetuning some specialized kind of question answering. Suppose we want to develop an specialized kind of question answering for medical domain. by using LLM trained on public data which is not specialized in medical domain. Some for example here we can say that chat port for a specific domain like a customer support for a tech company. So in that case when we select the query and value matrices it enables the model to better understand the domain specific queries and generate the precise or accurate responses. So I have covered this in impact also. This enhances the model's ability to focus on its its attention on most critical information within the new context leading to improved conversational flow factual accuracy in a specialized dialogues like that. Now another choice is key and output projection matrices. When we take a key and output projections means uh this will be useful for all kind where we are generally apply for sequence to sequence kind of techniques. For example, here it is like a document summarization, translation all other areas where the sequence to sequence system is performing very well. Here you can see the system is working very good. So in that case what are the impact factor? So this allows a more noise understanding of semantic relationship and precise extraction of informations critical for task where tight interpretations of a context drastically change changes the outcome. So this comes under this category. For example, summarizing legal documents, medical research documents like that. Here in that case scenario, it will work very good again like a feed forward network matrices that I discussed earlier. So this is very good where you want to train it for some non machine learning task or for some new kind of task like a new type of classifications, new type of uh regression, forecasting, prediction, modeling. You can apply this. This allows the model to adapt its token wise transformations which can be beneficial for task requiring a specific feature attractions transformations not well represented by the pre-trained feed forward network.