Tone Analysis - Fresh Machine Learning #3

Key Takeaways

how does that make you feel I'm not sure uh-huh it's hard to say yeah well there's an episode of serology coming and I just don't have time for this right now [Music] so hello world it's SJ have you ever had a hard time trying to understand what someone is trying to say to you people are confusing I don't even understand my own emotions half the time much less what someone else's feeling or thinking when we read a piece of text or listen to someone speak our brains are performing tone analysis we're trying to understand the meaning behind the words that is the sentiment And the emotions and the style it's not an easy task at all but this is something machines can do pretty well now it's all a part of a subfield of machine learning called natural language processing standard approaches to NLP a couple years ago involved extracting a set of features from some labeled piece of text the features were usually engrams engrams have nothing to do with graham crackers even though they are extremely delicious engrams are just sequences of words so a unigram is one word Byram is two and trigram is three once the engrams were extracted The Next Step was to train a linear model on some pre-labeled data so they could classify similar text the process of feature extraction saw a huge leap forward when Google open sourced something called word to VC word to V is a toolkit that helps encode words into vectors these vectors are representations of words called word embeddings that are learned by training on a given Corpus the toolkit consists of two distinct models skip gram and continuous bag of work words the skip gram model predicts the neighboring words given the current word in a given window in contrast the bag of words model predicts the current word given the neighboring words in a given window these models both help predict and encode words once we have these vectors we can use them to do all sorts of text classification including tone analysis deep learning one of the first papers to show that deep learning could improve text analysis was called convolutional neural networks for sentence classification although CNN were intended for computer Vision these guys applied it to NLP they first trained the CNN on a set of word vectors that Google extracted from a 100 billion word Corpus using their own word to V toolkit after training the CNN had built representations of all sorts of word categories since the vectors weren't labeled in any way the training was considered unsupervised after they had it trained on a set of Google vectors they then trained it using label data so that it could perform sentiment analysis A later paper called text categorization using lstm for region embeddings improved on it they did the same basic experiment but the key difference is that they didn't just use a CNN to create embeddings they also used a long short-term memory Network an lstm is a type of recurrent neural network that can remember dependencies from way back in the sequence of data and they found that they had the best classification result when they combined the embeddings from both the lstm and the CNN just like twps in a algorithm that was terrible they also found that embedding regions that are sets of War words was more effective than embedding single words this idea of abstractions of a hierarchy of knowledge in a given document helped inspire a fresh paper released just 2 months ago called hierarchical attention networks for document classification these guys said let's create a new neural architecture to model a document it starts by encoding words and applying an attention mechanism to extract the most important words then it encodes the sentences using the weights it learned and applies an attention mechanism to that too to extract the most important sentences it uses those weights to build document level vectors so it's creating vectors for each layer of abstraction within a document and building them off of each other the encoder for each of these levels is called a GRU neural net so after they initialized the model using vectors they got using word to VC they trained it on labeled data and doing this with their novel neural architecture pretty much outperformed all previous attempts can't touch this so how do we implement this stuff ourselves well as Professor Ang says deep learning requires a rocket the model and Rocket Fuel the data but sometimes you just want to get done and training is too time expensive that's why we're going to use IBM's Watson API to perform tone analysis on an example set first we'll need to sign up for their cloud service called Blu miix once we've registered we'll click on Watson then tone analyzer we'll create a new service using the standard plan which lets us try it out for free so no need to enter a credit card which is perfect cuz I'm poor then we'll click on service credentials and record our username and password since we'll need them to authenticate from our web app then we can generate a new node Express web app using npm and the express generator module in our app we'll import our Watson developer Cloud mpm module which acts as a thin JavaScript wrapper around their API we have two routes here both for our index page we'll start with the get route when we make a get request to the index page we want to display some HTML that we can send programmatically here we'll send an input form that asks for some Corpus then in our post route we'll retrieve the input that the user submits via the post request then initialize the tone analyzer variable fill in our credentials then call the tone function using the user input variable as the parameter this will return the analysis of the text as Json which we can view in terminal let's try it out as you can see it raised the Corpus on three levels emotions language style and social Tendencies you'll get back a percentage for each of three language Styles and five social Tendencies it does this at both the document level and sentence level tone analysis is a tool you can use to better your writing and be more clear about the message you're trying to convey dope links for you down below please subscribe for more ml videos I've got to go fix an indentation error so thanks for watching