Sentiment Analysis with Transformers in Python

Key Takeaways

what is going on guys welcome back in this video today we're going to learn how to easily do sentiment analysis using a bird model from hugging face so let us get right into [Music] it all right so we're going to do sentiment analysis in Python today using a Transformer model from hugging face in particular we're going to use this one here distill bird base uncased fine-tuned sst2 and we're going to easily use it using the Transformers package in Python it comes with a tokenizer and a model and we can use it easily uh to classify text as either positive or negative regarding the sentiment now whatever data you want to use for this video is up to you you can use a kaggle data set you can use your own data you can use Amazon reviews movie reviews uh news articles or in my case here I'm going to use the top 20 Play Store app reviews and particular the Dropbox CV file just so I have some text that I can use um the important thing is that you have a list of texts that you want to analyze in terms of the sentiment once you have that we can open up a uh notebook here a IPython notebook so a Jupiter notebook basically but you can also do that in an ordinary python file and what we need to install are a couple of packages we need to install using pip first of all uh Transformers we need to install torch we need to install numpy and we need to install pandas these are the packages that we need for this video today so if you don't have them on your system make sure you install them um what we're going to start with is we're going to say import pandas as PD and we're going to load the data frame into our code so let me just zoom in a little bit here we're going to do a read CSV and the CSV is going to be the Dropbox CSV file in my case here and you can see we have review IDs we have some content and we have a score now the score can be used for comparison because five is usually very positive one is usually very negative so if they're uh the same most of the time so if you get a positive for the fives and a negative for the ones it seems to work quite well um and what we're going to do here now since we have a couple of rows I'm just going to limit this for um for demonstration purposes to a sample size of just 200 so we're going to say DF equals DF sample 200 and then we basically have the same structure but we only have uh 200 rows now let me maybe do that again so we have some yeah I mean it's it's fine okay um all right so this is our data frame now and what we're going to do to actually do the classification is we're going to say from Transformers import pipeline so we're going to model this as a pipeline and from Transformers we're going to import distill come on Where's the auto completion when I need it okay I'm going to type this myself then dis still BT [Music] tokenizer distill BT for sequence classification and then we're going to do the following we're going to say tokenizer equals and then this distill bird tokenizer uh Dot from pre-trained so we're going to load the pre-trained model from hugging phase uh which is going to be exactly um this thing here so you basically just have to copy this um and you can paste it here as a string I don't even think that we need this we can only we can just provide this I think and then it also works then we're going to do the same thing now for the model itself the model is going to be from pre-trained but it's not going to be the tokenizer it's going to be distill bird for sequence classification now the pipeline we're going to call it NLP and we're going to say it's a pipeline that has the name sentiment and now Anis and what we're going to say is that the model is the model and the tokenizer is the tokenizer like this and then all you have to do basically to do the classification is you have to get the list of texts so texts is going to be equal to DF uh. content. values and I think it should be a list a python list and then we just have to say results equal NLP applied to the text like this and then it's going to do some work probably with a GPU and once it's done we can go ahead and say for text result and score in zip and now we're going to zip the texts we're going to zip the results and we're going to zip the F score values uh just so we have a comparison and we're going to print this information so text is text and then we can copy this results and score there you go and what you can see here is for each comment for each of the 200 comments uh we have the text we have the result containing a label being positive and score which is the confidence as far as I know um and you can see for example let's go to an obvious one lose my old photos collection automatically password reset problem please Dropbox team help for recover my account okay that's not very good English but it's obviously a negative review really impressive I thought and so on it's going to be a positive one okay so to get a better overview maybe of this let's go ahead and add this as a row so let's take the label of the result and add it to the row uh to to each row in the data frame so so we're going to say DF sentiment is going to be equal to uh basically R label so this is a list comprehension for R in result our results plural here and that should actually be it there you go so you can see you get a negative review a positive review and uh most of the time it seems to be pretty accurate even if something like why can't I edit EXO files is it doesn't really contain any negative word per se so it doesn't say it's bad or horrible or unsatisfying it asks a question but somehow it recognizes that this is uh negative and it's also true so you can also see whatever this means menab uh is positive it also has a score of five cracked is negative so it seems to be very accurate when it comes to that so this is how easily you can do um sentiment analysis using transform former models from hugging face so that's it for today's video I hope you enjoyed it and hope you learned something if so let me know by hitting a like button and leaving a comment in the comment section down below and of course don't forget to subscribe to this Channel and hit the notification Bell to not miss a single future video for free other than that thank you much for watching see you in the next video and bye