BERT Full Training for Beginners | Explained in Tamil | Embedding Models | GenAI | Agents | RAG

Key Takeaways

Hi everyone. And now let's get started. And then bird input in the token ID. And then finally language model data for language model separator probability 0.15% so 15% language model and next probability of the words and then% of configuration First by default 30k something 352 basic extra special tokens. And next around 76 dimensions model 256. learning. Next number of hidden layers. Fore number of base model number 256 / 4 64 every 64 dimensions 64 into the numbers forward. So basically 256. So 256 into four. But again like 1024 and then max position context window of that model. 128 tokens. The birth model and again for pre-training basin. First training over just simple overput. directory and then per device. So basp 32 and then number of training. So the data set and then every 100 steps and then check every step. retaining learning eusing tasks learning and then warm up and then FP2 And again overang. Next transformer model data set token data Next train. So again every 100 less than one GP GP full and then 6.7 and Then instead of three training And in the model like train. So in case It's better. Fore took tokeniz. So instead of minim Extra tuning full understanding model. So first just let me know in the comments. So until then, cheers. Bye.