Text Representation Using Word Embeddings: NLP Tutorial For Beginners - S2 E7

Key Takeaways

in this NLP playlist we have covered the tax representation techniques from label encoding to tfidf today we are going to talk about word embeddings there are certain limitations of megafores and tfidf which we have discussed in previous videos which is the vector size can really be big for bag of words and TF IDF model and it may consume lot of compute resources memory and so on let's say you have vocabulary of 200 000 words or 100 000 words each Vector for each of the documents would be 100 000 size and that that may be too much and the presentation is parse meaning in that Vector most of the values are zero so it is not a very efficient presentation the other problem we saw was that let's say you have two words I need help I need assistance these are similar sentences you expect that their Vector representation should be similar but since these are TF IDF and bag of words are count based methods the vector representation might not be similar here you can see see there is one here zero here one here zero here uh so these techniques do not capture the meaning of the words properly you expect that if you have two sentences which are similar their vectors will be very similar word embedding tries to address both of these shortcomings first similar words will have similar vectors similar words or even sentences for example good and great these are two similar words when you will use word embedding you will notice that their Vector will be very similar see 3.1 3.1 4.4 4.2 there will be slight differences but mostly they will be similar they will also be lower in dimension which means the size of this Vector will be 300 300 is a one of the popular Vector sizes in word embeddings it could be 50 size 100 sizes Etc but it won't be like 100 000 because your workability size is 100 000 and these representations are dense representation meaning you won't notice too many zeros as opposed to the previous techniques where most of the values in the vectors were zero now there are various word embedding techniques such as word to whack glow fast text these are the techniques uh which are built on different approaches and the common technique that we use to build these word embedding which is what to wear glow and fast text is using continuous bag of words and Skip Gram now this might be a jargon so just hold on I will go over that in detail a little later then there are techniques such as bird and GPT which are Transformer based embedding techniques these are the latest advancements in NLP when you do Google search it is using bird so you might have heard all these jargons so I'm just going to go over these jargons in brief and in the later videos we'll go over them one by one so don't worry if you're confused right now just don't worry just remember that these are various word embedding techniques there is a technique called Elmo which is based on lstm now what do these techniques do see they convert a word or a sentence into a vector representation so that it can capture the meaning of that word properly not only that you can do arithmetic with the words word to back has this famous example that you can do arithmetic you know how you do arithmetic with numbers let's say 2 plus 3 is equal to 5 minus 1 is equal to four you can do similar things with the tax and that's mind-boggling right see King King has all these attributes Authority King is Rich it has power but if you remove the element of man from that and add woman it becomes Queen correct Queen also has power Authority lot of money but the gender is different so you subtract that gender at a new gender now you get Quinn will look all of this in detail we'll look into this in in the detail uh during the code but for word to whack I have already made a different video so if you go to YouTube search for code Basics word to whack that that first video that you see which is part of my deep learning playlist please watch that because that will give you a very good understanding of the actual method or actual technique behind word to whack since I already have that video I'm not going to repeat the same thing here I would highly recommend you watch that okay if you don't watch it you're going to probably miss few things in this NLP Series so word to wet very very important concept you need to watch that video now there are variations based on the data set that they are trained on for example word to work you can train on Google News data set or some different Corpus and based on that you can get different variations so virtuac is like a technique but you can use the technique on different data set it's like you are training support Vector machine so support Vector machine is one technique but you can train it on Amazon reviews or movie classification similarly word to work the actual embedding Vector you can get by training it in a different Corpus same like glow if you train it on Twitter let's say there is a globe Twitter 50 this model which is available in gen Sim Library it will understand the tweets better all the slang and all the short form that you use it will understand that better and then there could be a Wiki gigawatt Globe version there are so many different models but the basic techniques are glow work to work fast decks and so on similarly the Transformer based models such as bird also based on what type of data set they're trained on you can get biobert or fin bird fin bird is trained on financial data set bio bird is strain on biomedical database and you can also do some parameter tuning in bird and get things like Albert or Roberta so you would hear all these jargons you know fear fin bird Albert Roberta virtue so I'm just throwing all the jargons so that you get some overview don't worry if you don't understand these techniques we'll go over them in detail in the later video but just to summarize what this technique produces see folks it's very simple the whole purpose of these techniques is to convert word into a vector because we have seen previously machine learning models don't understand text they need numbers so you can convert a single word such as good to a vector you can convert entire sentence to a vector and that is called sentence embedding or it can convert the entire paragraph or a news article or a document into a vector so in the future video we will be writing code and we'll be using this techniques to solve the actual NLP problems this was just a short introduction on word embedding once again watch my word to whack video and by the time you come to the next video you should have watched that video okay this video the first one otherwise you're going to miss out on few things all right if you like this playlist give it a thumbs up and share it with your friends thank you [Music] foreign