Determining insert - Creating a Chatbot with Deep Learning, Python, and TensorFlow p.4

Key Takeaways

what's going on everybody welcome to part four of our chat bot with Python in tensorflow tutorial series in this tutorial we're just gonna keep building on last one so we've begun kind of pulling out the data that we're interested in the other thing I wanted to do real quickly is bring up buff the buffer in parameter so I wrote buffer here in that file open it needs to be buffering and at this point just to make sure you got it right you can print the row so for row in a print row and if you run this you should see you know data flying by okay so make sure that's happening for you that's not happening something is wrong so I'm gonna get rid of that dose that's gonna be really slow especially with idle now what we're going to do is basically at this point we've got the information we're interested in inserting it into our database but should we so the next question is basically first of all like I was saying before I don't want to just insert every comment there's a lot of useless comments on Reddit who'd a thunk but so so basically were gonna have a threshold here we're gonna say if score is greater than or equal to two and you could pick something else I mean I feel like you know even like 5 or 15 especially if you've got the entire just torrent dump from reddit or you know if you're doing bigquery or something like that and especially with bigquery you could actually throw this into your bigquery all right before you even fold the data you can use a pretty substantial number here anyway I'm gonna go with 2 for now just something more than one so this just says basically somebody saw this comment in somebody somewhere was like yeah and they hit ok so yeah so if score is greater than or equal to 2 what we want to do is basically at that point we're thinking hmm we would like to insert this data into our database but what if we already have that a comment you know what if we already have you know basically what if you know that comment score is greater than 2 but it has a parent ID right what if we already have a reply to that parent ID like what if we don't need one and if we do already have a reply basically what we're gonna want to know is like okay well what's the existing score of that comment does this score or this comment have a better score so so we need to find that out so we're gonna say if score is greater than or equal to two we're gonna say existing underscore comments underscore score equals find existing score now we're going to search for the existing score by parent ID so we're going to search for the parent ID and see if is there a comment and reply to this parent ID that already has a score greater than greater than our current score if not we don't care if so we're gonna go ahead and update that row with a new reply in the new score and all the other new data okay so so yeah we want to do that now what we want to do is actually create this find existing score function so I'll copy that I'll come up here define find existing score and basically again we're gonna search for that by parent ID so what we're gonna do is let's just listen I like the idea of try and accept and return false if if all else fails so let's go ahead and paste that that lovely logic in and honestly a lot of this stuff is gonna remain the same so let's do yeah this is like almost an identical function honestly so so so in this case we want to not select comment but we want to select score from parent reply where not comment ID this time we want to do parent ID where parent ID is equal to blank limit one format parent ID see that execute SQL result fetch one none return zero otherwise we return false and then return false yeah I think that's right I hope that logics right we'll find out anyway cool so once we've done that awesome so then we're gonna ask you know if existing comments score so basically if it has any value besides false this is going to be true so if existing score then our question is well is our current score greater than the existing comments score question mark and then if it is now we're thinking okay we're ready to possibly insert this data and in this case we would actually be updating a row rather than just a straight-up insert now the other thing that I want to go ahead and do is create the acceptable data so because the other thing we're gonna ask before we do an insert is is this even an acceptable statement and in theory actually we probably should ask that even like up here like before we consider anything else it needs to be acceptable but let's go ahead and build it real quick so we're gonna do is define acceptable it's gonna take in data which is going to be the comment basically and then if Len of data split and we're split by space but we're just gonna tokenize it real quick and see is that length greater than so that we're going to be doing multiple models here but one of them is going to require that we buck it to a maximum length of about 50 50 words so I'm gonna say 50 here but later we might want to modify that a little bit but anyway for the Len of [Music] data is less than 1 so like let's just say that was like an empty comment for whatever reason maybe someone edited or something like that what we want to do in this case is return false Elif the Len of the data for whatever reason is greater than like a thousand so it's got more than a thousand characters chances are that's something we don't really want so we're gonna go ahead and return false there as well Elif data equals and here are two versions of comments being removed or deleted there's deleted elif data equals deleted or data equals [Music] removed we want to go ahead and return false else let's return true so again I'm not really sure why but when Chi maybe it's like when a user deletes verses when a mod deletes or an admin deletes I don't know but they these two things exist to me they mean the exact same thing okay anyway I think this is a good stopping point we've been building a lot of stuff here so I'm trying not to keep keep my videos from being 20 minutes long each so so we'll go ahead and stop here and in the next tutorial we'll continue kind of building this out will actually ask if something's acceptable and then we'll probably in the next tutorial be able to actually finish this and press go and start populating a database so that's what you guys have to look forward to if you have questions comments concerns will whatever feel free to leave them below otherwise I will see you in the next tutorial