Python Tutorial: Sentiment analysis types and approaches

Key Takeaways

welcome back in the previous video we learned what sentiment analysis and why it is useful and how do we ever start with a sentiment analysis task sentiment analysis tasks can be carried out at different levels of granularity first is document level this is when we look at the whole review of a product for example second is the sentence level this refers to determining whether the opinion expressed in each sentence is positive negative or neutral the last level of granularity is the aspect level the aspect refers to expressing opinions about different features of a product image this sentence such as the camera in this phone is pretty good but the battery life is disappointing it expresses both positive and negative opinions about the phone and we might want to be able to say which features of the product clients liked and which they don't the algorithms used for sentiment analysis could be split into two main categories the first is true or lexical based such methods most commonly have a predefined list of words with a valid score for example nice could be plus two good plus one terrible minus three and so on the algorithm then matches the words from the lexicon to the words in the text and neither sums or averages the scores in some way as an example let's take the sentence today was a good day each word gets a score and to get the total balance with some two words in this case we have a positive sentence the second category is automated systems which are based on machine learning this is going to be our focus in this course the task is usually modeled as a classification problem were using some historical data with known sentiment we need to predict the sentiment of a new piece of text we can calculate the value score of a text using the Pythons text blob library to continue working with our today was a good day string we import the text blob function from the text block package and apply it to our string a text blob object is like a Python string which has obtained some natural language processing skills working called different properties of the text block object we are interested in in sentiment that's why because sentiment in our text block the sentiment property returns a tuple polarity which is measured on the scale from minus 1 to 1 where minus 1 is very negative 0 is neutral and plus 1 is very positive our example today was a good day here is positive emotion and thus we have a positive polarity score of 0.7 the second element in the tuple displays the subjectivity measured from 0 to 1 where 0 is very objective and 1 is very subjective so our examples rather positive and subjective which method should one use a machine-learning sentiment analysis relies from having label historical data whereas lexical based methods rely on having manually created rules or dictionaries Mexican based methods fail at certain tasks because the polarity of words might change with the problem which will not be reflected in a predefined dictionary however lexical based approaches can be quite fast whereas machine learning models might take a while to train at the same time machine learning models can be quite powerful so the jury's still out on that one many people find that a hybrid approach tends to work best in many usually complex scenarios now let's test what we've learned by solving