Creating a Reddit Bot to Detect Spam - Python Reddit API Wrapper (PRAW) tutorial p.4

Key Takeaways

what's going on everybody and welcome to a creating a reddit bot tutorial in Python so what we're gonna be doing specifically is creating a reddit bot that goes through and detects reddit spam and affiliate links specifically in this case to udemy but we could very easily modify this to detect quite a bit of spam so over like the last year I've noticed this is crazy sharp incline in the amount of just udemy affiliate link spam coming in to Reddit and it's hard I think to catch because they're not necessarily directly linking to to udemy they're kind of linking to some other tracker site that then links to udemy so a lot of times they link to a Twitter post that links to udemy or links to a tracker site that links to udemy or they link to a medium post and so on now that seems to be on the decline and now they're actually not even doing that extra step they're they're actually just linking direct to to the tracker site then which goes to udemy and they're not even being kind of hidden about it most of the time they're saying udemy and all this kind of stuff so anyways I thought about writing a bot a long time ago to do this and detect this kind of spam but there are just so many other things I wanted to spend my time on I didn't really feel like doing it that was until a few days ago I found some reddit spam to you to udemy course that was actually a pirated course of my courses a direct rip from YouTube even the author's bio on udemy was a rip from H Kingsley com so just a total pirated course and this was being spammed on reddit so why don't you know I found some time I found some motivation and I wrote the bot so here it is I'm gonna run through it with you guys we're not gonna be writing it line by line I'm gonna kind of copy the the method I did with the latest one of the latest tutorials if you want to learn more about working with Priscilla go to the project Oriole series like this is just going to be part four of that series so if you want to learn more about interacting with the the reddit objects and stuff like that check that out this is really just a application I mean just kind of showing you it going to so the first thing that we're gonna want to do obviously have a reddit account suck the API again go through the process if you don't know how to do that but basically make an account go to your account preferences app create the app make sure it's a script app that's about it really fill out the form so once you've done that you'll have your credentials client ID secret password user agent username I'm putting these into a prom on your score creds dot PI this is a copy of that because the real one actually has my real values and I don't wanna show it to you so anyway and then I've got a to catch a press to catch a scammer dot PI script so with that let's go ahead and get started the first things we're going to need to do is import that proc creds the client ID oh by the way you don't have to write all this out I have a link in the description to this tutorial also the code is on github so if you want to contribute or check out the code or whatever it's it's up there I'll put a link in the description but it's just Ram spam reddit spam detector bot okay so once we've got that the next thing that I want to do is basically we create the reddit object itself pass through all the credential stuff and all that and then we kind of have to figure out okay how are we gonna catch these people how are we gonna figure out who's a spammer who's not because just because someone's posting a link that does go to udemy it doesn't necessarily mean that they're a spammer I mean they're probably a spammer but we can't just assume at that point that they're a spammer so what we actually need to do is probably go through all of the posts but then like go through posts that are linking to udemy stuff but then also go to that username and then check out that username or all of their posts to udemy junk or is just some of it or you know whatever cuz sometimes like I thought it was probably the course creators just the course creator that created the course and then they went to you read it to kind of spam it out but no these are definitely massive spam rings so I found so much like they're huge I didn't realize how big this problem was until I I really started looking into it's pretty it's pretty nuts anyway so what I've done here is I'll pull up so for example let me pull up my little boy guy so this is my spam detector bot and basically what he does is he just you know he's just going to these threads and just post this message basically so I get in this case this guy I don't even know this name it's just a spam name we're just saying hey you know six a third of these 33 submissions from this user appear to be udemy affiliate links and the reason we can know that is if we actually go to this user account we just click on it and we can see like look at all of these courses really a hundred percent of these are courses in my case we're just looking for the term udemy but in time I plan to expand that out because obviously especially once somebody figures out that I'm looking for the term udemy they'll just remove the term udemy so so that'll that'll change for sure we'll come up with better words and stuff and ways to match courses there's lots of little things I'm not sure I'm going to go over all of them I don't want to give away the tricks to the trade but anyways yeah so it's pretty quick and easy to go to these people's page and figure out this is a spam bot or a spammer I'm actually not confident these are BOTS I think it might actually be a human behind them anyways we're gonna move these aside now so that's what we need to do we need to get to this user and look through his content so so we can do all of that with the Python reddit API wrapper so what we're going to do first is find a just bind spam by a username so the function that I wrote for that or find spam and then return the user names so basically what we're gonna do here is we're just going to go through a search query and just log all of the authors right and then what we're gonna do here is let's just iterate through that real quick just to show you an example of it running so paste and then I'll run that so here we have just the list basically just ran through you know the posts and then here are the users so all these people are suspicious at least but again these are just people who have posted about udemy we don't know anything more about them so we really kind of want to dig a little deeper into these users to really determine whether or not they're there spammer or they're just a regular person sharing a link to their course so now I'm going to go ahead and do is just overwrite this kind of loop here and so instead what we're going to do is we're gonna search for query and right now we're just gonna random choice for udemy again later I would add many many more terms to that because again if we're just looking for udemy if these spammers were to find out that that's the one thing that is the crux to this entire code they'll just stop posting udemy they'll they'll just stop using that word because it's really not essential so anyways once we find that we're gonna have spam content trashy users and then basically smelly authors we don't know if these authors are trashy but they smell bad so we're gonna figure out a little bit more about them so also we're using random here so let's go ahead and at this stage we need to import random okay so once we have these smelly authors we want to start iterating through the smelly authors to figure out what their deal is so come down here pay so make sure we're not having a tab issue good and let me make sure this is set right and then okay cool so what we're going to do is now we start iterating over those authors so for each author in smelly authors we're gonna say we're gonna start counting how many trashy URLs do they have how many submissions did they make and how many of those submissions are dirty so then what we want to have is something that is like common spammy words so I'm gonna actually come up to the top here and just paste that in so common spam you weren't worms udemy course save coupon free discount as a matter of fact really a lot of these things like for sure course coupon discount chances are if someone has these things in there it's probably spam anyway and then if the user continues like again if we just if we search for spammy words and then we only matched these spammy words if like more than 50 percent of a user content is always this junk there a spammer so so anyway that's just again to point out you know the the udemy should not be the only word that you're going to use I'm just gonna do it for this tutorial I don't want to give away all the great words but we'll see I'm gonna have it up on github for a little bit and if people contribute then we could probably leave it open source and it won't be a big deal if people can stay ahead of it but this isn't exactly the most exciting task in the world so I'm not really sure how many people are gonna help but it's it's kind of fun to combat these guys so anyway the next thing that we're gonna do is basically we want to run through their submissions so we found the author now I want to iterate over that authors submission so we want to visit their profile so I'm gonna paste in that chunk of code let me zoom out a little bit there so it all fits so we want to throw a try and accept into here mostly because the post might have been made by that author but these authors are spam authors they're they're going to eventually be removed from reddit so when that happens sometimes these users will return error codes if you try to access them otherwise what we're going to do is basically we're going to say is for submission in that redditors and then string author because at this point author is actually an object it's an author object from the reddit instance so from whatever that authors user name is their submissions ordered by new we're gonna start iterating over them so we're gonna see where that submission links to at the moment I don't actually think I'm using this but this is another great thing to track to figure out if someone's a spammer right now they're using the same kind of websites so one you could figure out they're a spammer but two you could also link them to other spammers and this is what I started to do to identify the networks themselves so the people that are spamming to the same kind of trackers they're pretty much all connected and given the fact that they all have very similar usernames I'm gonna go ahead and say it's probably the same group that's doing this but anyways moving right along I'm not actually using that right now I don't think but it's still something good to track otherwise we take the ID so we can quickly get to it actually I'm not even sure we're tracking that anymore because I think we just get the URL to it but anyways we also want to track the subreddit again just in case there's like a udemy freebies subreddit that is basically all affiliate spam so I think everybody in that subreddit probably knows exactly what that subreddit is it's like literally probably for spammers so not really sure what to say about that honestly I feel like that sub should just be deleted whatever but then we're gonna say dirty equals false and then what we're gonna do is we're gonna iterate through the spammy words and then for each word in the submit title dot lower what we want to do is if I'm sorry if the word is in the submit title lower that way we lowercase everything if it is then we're going to say dirty is equal to true we probably should break at this point and not way soon it's not gonna I would break at this point but we're gonna continue on anyways and then if that junk is not already in the users trashy URLs let's go ahead and add some information the submission ID the title and the author's name into there and then if this is dirty what we're gonna do is we're gonna add two dirty count add to the sub count and we're just it's just basically a counter a counter for that now what we want to do is if if the trashy score is above something basically so we want to generate this kind of track oh we don't have a trashy score so let me copy and paste this new block because actually this one's going to change so really we're just adding a little bit under here but basically what we're gonna say is so like what we did up to this point is like right here right that's what we've just covered here what we're gonna say is we're gonna try to say the trashy score is equal to however many dirty posts as person made over the total that they have if we fail for whatever reason for example sometimes the sub count will be zero again some sort of weird error you'll get a Divya so we're just gonna say trashy score is zero otherwise it's just the percentage of trashy posts so we'll just print this out you could leave this or or get rid of it honestly this won't take too much of your time so you could just leave it anyway if the trashy score right now is above 50% we're gonna add this user to the trash users and then for each trash in trashy or user trashy URLs we're gonna add that to the spam content in general so once we get to this point what we could do let's go ahead and run this and make sure this actually works so at this point we were just returning you users let's go ahead and run this real quick and make sure we don't have errors okay so we're already running through it we've found online free courses their trash D score is basically 92% we've already run through tools break that anyway so clearly this guy's a spammer this guy's a spammer this guy's probably a spammer for some reason his ratio is at forty four we've this is actually the guy shows you initially I think so anyway probably we should lower it maybe to 30% I mean if more than 30% of a user submissions or spam maybe that's worth pointing out I don't know I can't decide on what a good number is but I feel like definitely more than 50% they're a piece of junk so so once we've identified who's a piece of junk what we want to do is then iterate over that and then post of it because now we know it is true the junk so every time we make a post at least on a brand new account we have like a timer it's like a 10 minute or 8 minutes or something like that so every time we make a post we also want to sleep so we're gonna import time here and then I'm gonna scroll on down and now what we want to do is basically just iterate over the whatever we have now in spam content so I'm just going to come down here and I'm gonna paste this in I'm gonna scroll out and make sure we did that right let me confirm that real quick yeah so once we get to this point let me pull this back down zoom in we're gonna start iterating over the spam content so we're gonna identify the ID the user the submission itself when was that submission created if that was less than a day ago that's just 60 times 60 times 24 if that was less than a day ago then the full link would be to reddit.com slash submission I just want to save this for myself at least initially when I was writing this I just I really wanted to make sure I wasn't accusing someone spam that wasn't spamming so I've been I'm gonna save all the links basically then this is our message we're just be Boop we're a bot we're identifying this idiot they're a spammer get rid of them that kind of stuff and then we're just using string formatting passing it on in there and letting them know so once we've we've got that message what we're gonna go ahead and do is first we're gonna check the post of URLs text this is just a quick file that we're just gonna check and see if we've already posted to that file if we have then we're gonna skip it but if we haven't so if that links not already in the already posted what we're gonna want to do is let's print out the message let's go ahead and make a reply to that reddit post then we're going to print out that we've posted to and whatever the link is and then we're going to write that link to the post of URLs text file and then we're gonna sleep for 12 minutes and then we're gonna break because we don't want to necessarily iterate through through everything we'd rather like start clean and go with the newest submissions again so we'll break there other than that that's basically it so we could we can run this now I'm not sure if we'll actually find spam this time around but let's just see because I've been running this so I'm not really sure that we'll find any spam but we'll see what we get and if we don't see we return to 404 for some username I don't know who it was but anyway if we don't post what I can do is we can at least we can well we can go back further in history when we search for posters and then we can also go back further in history into the actual spammers like right now we say we're not we're not gonna post on anything older than a day we we could at least when I was writing this I didn't see any point to doing that especially if this is running live there's no reason to go back super 4 historically to do this but yeah it doesn't look like we're gonna find anything on this one but what I'll do I guess what I could do I'll pull this one over at least so at least here we figure it out this one was for online courses we found them we put it just says we've posted and then we're gonna sleep for 12 minutes if we do happen to find somebody what I'll do real quick just to see if we can get somebody else let me say I don't know 50 and then at the point where we wait area here so let's just say I don't know 10 let's try to rerun that and we'll see if we find anything I'm not really sure if we will but we'll try we fred anyway while we're waiting on that I'll just draw your attention one more time - this is on github if you want to make any contributions or support it or whatever feel free to take part other than that the codes also on Python programming at net running through it kind of like I did here so if you missed anything or whatever you can go over there yeah well get all these users we're fine probably shouldn't set that limit to 50 probably should've just done the historical - like 10 days or whatever this is taken forever okay so we found somebody so again it was online courses and it's on udemy freebies which is like just a total spam subreddit but let's go ahead and check them out anyway so we'll just pay copy over here we'll head to that and here we are so there it is so in action running and just in general being slightly annoying so far it looks like people have been relatively appreciative of of our BOTS but we'll see I wasn't really sure what people would think of it especially because like I said it like on on udemy freebies I'm pretty sure that's all for spam and I'm not sure like if people actually like that subreddit and find it useful because they get cheap courses I'm not really sure anyways that's it for now if you guys have questions comments concerns whatever feel free to even below otherwise I'll see you in another video