Combining Algos with a Vote - Natural Language Processing With Python and NLTK p.16

Key Takeaways

What's going on everybody? Welcome to part 16 of our Python with NLTK for natural language processing tutorial video. In this video, what we're going to be doing is taking all of the classifiers that we just built and kind of putting them together to create a voting system. So, each classifier gets one vote. And our new classifier that we're going to write now is based on the voting system between all these classifiers. And whichever uh position, whichever category, positive or negative, gets the most votes by all of these classifiers, that is the one that we're actually going to choose. This should not only raise our accuracy by a few points, it should rel raise our reliability very nicely, but also it gives us the ability to add in another parameter and that's going to be our confidence parameter. And this will be a parameter based on h uh how many votes, right? Who got the what percentage of votes? So if the score is 100% where where all of the classifiers said hey this is what we think is the case then our confidence is 100%. If um you know three out of seven said something was negative and four out of seven said it was positive our our confidence is kind of low. It's you know four out of seven. So anyways that's what we're going to be doing here. We're going to build this new classifier. Uh so this is going to be kind of like our own sort of classifier and this is you could use this methodology actually to create any classifier you wanted. Uh it just so happens that this classifier that we're going to make is the you know uh compilation of all of these classifiers. So and in fact uh the classifiers that we want to use we already know we don't really care about SVC because it was giving us inaccurate numbers. So we're we're just going to toss SVC and so that leaves us with one two three four five six and then seven for the original classifier. So we have seven classifiers. So that should be good. So we're going to come down here. That's a nice odd amount. And actually let's go up to the very very top. Uh we need a couple of things. Uh first we're going to go from NLTK.classify classify. Uh we want to import classifier I. So we can basically this is so we can inherit from the NLTK classifier class. Uh and then finally we're also going to from statistics import mode. And this is just how we're going to choose who got the most votes. Uh we're just going to take the mode. Okay. So easy enough. Now coming down to the bottom. Um we we'll just build the class right here. Actually, we really should build the class up at the top. That's just where it should it should be. Uh, so we'll just put it up here, I guess. Um, so what we're going to say now is we're going to say class vote class. Actually, it probably should be like this. Vote classifier. Uh, and this vote classifier is going to inherit from the classifier I class from NLTK. Um, by the way, if you want to know the basics of classes, I do have a couple of class um, introduction videos. Also, there's a crash course for classes. Um, a class for classes. Hilarious. Anyway, I have tutorials on those, too, if you want to know like if you don't understand what I mean by inherit from, you might want to check those out. Now, define init. And if you don't know what this means, basically this is just a this will always run. So the init method on any class is used because when you invoke the class the init method will run whereas the other methods won't run unless you call upon them. So this one self uh and then uh for args we'll have uh classifiers. So we're going to pass a list of classifiers through our vote classifier. Um so that'll be that and then selfcore classifiers is just going to equal classifiers. So our classifier list will just be whatever list of classifiers we pass through our vote classifier class. Now we're going to define our classify function or method rather. And this is just to be kind of synonymous with the NLTK classifiers. That way we can call upon this classifier and say vote classifier.classify just like we can with any of the other ones. Um accuracy won't really matter. We're going to basically already have that anyways. It's just that we change classify because we have to kind of define our own classify because right now classifier doesn't actually do anything. So classify uh self and then features is what we pass through here. So this is just the same as anything else. We pass through the features so they can be classified. So this one's pretty simple. We'll start with an empty list of votes and then we'll say for classification in or I mean for classifier in self.c classifiers, what do we want to do? Well, we're going to say their vote. So, v equals c.classify based on the features. So now for each classifier, we're going to get the vote. Then what are we going to do with it? Well, we're going to do votes.append whatever that vote was. And that's that. Uh and then at the end of this function, we just or a method rather. We're going to return the mode of the votes who got the most votes. Now the other thing we can do is we can define a uh certain let's call this let's call this confidence define confidence and again self features um we really this will be almost identical to the classify class we could actually include it in that but I don't really want to do that I want to leave classify identical to the other functionalities of classify so that'll be that uh votes and then we'll get rid of that original return and we'll say the choice votes This will be equal to votes.c count whatever the mode of votes was. So this counts how many the most popular vote or how many occurrences of that most popular vote were in that list. And then we can do something like this. We can say confidence um we'll just call it conf I guess comp equals so we don't conflict with this conf equals uh the choice votes out of the length of votes. So this is just how many of the chosen um category over the length of votes. So this gives us a you know certainty basically. We could multiply by 100. I'm choosing not to do that at this stage because none of the other NLTK accuracy classifiers and stuff, they use the, you know, zero to one. Uh, so we're going to kind of stay with that. Uh, and then so we got confidence and then we'll return conf. Easy enough. So now what we're going to do is we'll come down. Um, we'll leave this one here. We'll come all the way down to the bottom here. And now what we'll say is we're going to say voted_cl classifiier equals uh what do we call it? Vote classifier. Yeah. Equals vote oops vote classifier. And then remember what did we say were the parameters? Well, this is args. So it'll just be a big list of them. So we're going to use basically all the classifiers we have. So we started with just the regular uh classifier here. just our basic classifier. We had that and then we had this MNB classifier. Then we had the Berni classifier. Then we had the logistic regression classifier. Then we had the stochastic gradient descent. I'm just going to stuff it right here. I don't feel like going to the end of that list. Uh then we had linear SVC. Paste. Just don't forget your commas. Otherwise, you're going to get a syntax error after waiting like a few minutes. And then new svc paste, comma. Good. Um, that's that. Now, what we can do is we'll take this right here. Copy this. Paste. And then we'll just say voted classifier voted classifier classifier voted classifier accuracy percent NLTK.class classify doac accuracy of what we want to do. The accuracy now of the voted classifier. I'm just going to put this down here with some separation just so we can see our new code basically. Um so this will be this is our new classifier with the voting uh capabilities. So we'll do that and then uh we can also finish this up with a sort of print. And what we can do here is we can print some examples because I want to show you guys the confidency thing and then we can kind of move forward. We'll use this and we'll actually move forward soon and uh generate real time sentiment analysis rather than doing this kind of historical and testing. Anyway, classification colon and then we'll do comma and then we'll do something like this voted uh classifier.classify classify and then we want to classify something and so we'll classify we'll just do testing set we'll do the zeroth element um and then zero again so that will be the classification there let me just test this good and then we'll do comma and then we'll do confidence and this will give us our confidence and actually we'll make this one a percent confidence percent colon comma uh voted classif classifier dot uh what do we call confidence I think we called it confidence then we returned conf but I just want to make sure confidence yeah so voted classifier dot confidence confidence um and then again the same kind of testing set we'll just do this confidence testing set 00 whoops maybe this okay and in fact let me uh make this a little smaller hopefully this will all fit on the screen. Yes. Okay. Our massive list doesn't, but this one does. Anyway, uh, good, good, good. Everything looks good to me. And in fact, we should be able to get away by doing this. So, this will make it all fit on the screen. Yes. Okay. So, this is our new voted classifier. Then we can This is just the general percent based on the testing samping sample. And this is just one example. So, let's go ahead and save and run that. And then uh I'll leave we're just going to leave this running for a moment. And as it starts outputting results, I'll show it on the screen. But then we'll come over here and let's just kind of let's make a few more. We just want to make sure the initial code is correct at least. Uh so now we'll go paste. Let's just make a few more of these just so we can hopefully see some um some more examples. So this will be a one, two, three, four, five. And then we'll make this a one, two, a three, a four, and a five. So those will just kind of run through a few examples um for the classification and the confidence percent. Hopefully we'll see some divergence. We may or may not, but anyway, uh this is outputting some results now. So the initial uh naive bay 64 we're seeing some other percentages here. Basically everybody's ahead except for stochastic gradient descent. Linear SVC is doing really well. I seem to recall linear SVC does fairly decently. It's one of the higher ones. Let's see. We still have new SVC to go through and stuff. But anyway, I was hoping this would go slightly faster. The voting process is probably pretty long to kind of want to just rerun it. There we go. Okay, so here's an example. Um, that first one, our classification was negative, but the confidence percent was Whoops, we we forgot to multiply by 100. Uh, basically 85 or 86%. Let's uh let's add our multiply by 100 real quick and then we'll run this one more time. Get a few more examples. But um you should be probably be getting the idea at this point. Paste, paste, paste, paste. Okay. So, uh with us at this point basically this last line here this classification negative confidence what um you should see this is the code right here that we will use to move forward. So testing set, remember testing set is merely a list of features and whether or not they contain those top 3,000 words and then that was trained against um you know whether those top 3,000 words are more commonly found in negative or positive reviews. So this is just a list of true and false. It's basically a set of you know true and actually it's a dictionary of the key being the word the value being true or false of its you know whether or not it's in this document. So what we can do moving forward is we could read really anything. We could read news articles, we could read tweets, we could read uh blog posts, we could read comments, anything. And as long as we pass the the words through what what do we have to pass those words through? We just have to pass them through find features. As long as we know about word features, we need to fih pass them through there as long as we've pickled um our algorithm. Otherwise, every time we'll have to kind of do this process. Luckily, this process isn't that long anyway. But you could pick the entire thing uh and save that. But I think you'll probably still want to tweak these parameters, maybe do some more training and stuff like that. Um it looks like we Yeah, we outputed this. So, let's just let's check this out real quick. So, here's we've got a one position positive, the confidence is 71%. Um this negative negative, everyone agrees. negative 71% negative everyone agrees. So uh as we start to go on you can kind of decide what confidence score are you willing to uh accept the answer. Otherwise you can just kind of toss it as bad data. So, um, is it better to have 80% accuracy but get literally, um, you know, half of the results or is it better to have 70% accuracy but get, you know, 80% of the data set, you know? So, that's kind of up to you. A lot of times you can kind of do a little bit more analysis to decide which one you want. But anyway, moving forward, this is basically all we have to do now. We just need to get a data source, whatever that data source is, and pass it through the find features. And as long as we have this word features either pickled or in our memory, we can start to generate this exact output on any body of text we feed through it. So that's what we're going to start working on now is kind of feeding any body of text. So probably take to Twitter, maybe connect to a Twitter feed and start outputting some uh analysis on, you know, some sort of keyword. So anyways, stay tuned for that. If you have any questions or comments up to this point, please feel free to leave them below. Otherwise, as always, thanks for watching. Thanks for all the support and subscriptions. Until next time.