Direct Input - Python plays Grand Theft Auto 5 p.3

Key Takeaways

So I was pretty confident that we would be able to just simply use pi autogeuey. I thought surely but and so you don't need to install py auto guey but um import pi autogeuey. Um I'm going to just comment all that out and then what I'm going to do is we're just going to run some pi autogy stuff. You can probably guess what's about to happen. Um, but yeah. So, what I'm gonna do is, um, I'm actually just gonna copy and paste this code because it's unnecessary for us to waste time on it. Basically, what it does, this is a brilliant countdown script. Um, it just counts down. Um, so we have time to get to the game, unpause the game, and accept input. Um, so, uh, let me go ahead and run [Music] this. Should start printing any Yep, there it goes. So, we're waiting for it. It's supposed to press W, which should go forward. It doesn't. And then it tries to go up and it doesn't. Okay. So, obviously nothing happened. Now, um what we can do is run that one more time, though. Um and then I'll just put my mouse here. So, this is a key down. So, we can see a couple things. One is it pushes the key down. But what happens on your keyboard when you actually push that key down, right? It goes like that. So, we already know the input is a little different. Um, but for whatever reason, it's definitely not working on Grand Theft Auto. I spent a long time trying to see like maybe it's just too quick. Like maybe the input was like too fast. So, I tried to put like many key downs and then I tried well maybe it's still too fast. I'm not seeing it. But if you press like if you press the turn keys, the wheels will turn all the way no matter how short you press them. So, I thought, okay, let's try that. So, send key A or whatever. It just didn't work. So, I did some research on that and again, we're just I'm able to cut this time down just simply because I did this prior. Um, but I did some research on that and I came to many threads, but this was the one that I got an an a good answer on. Um, this guy's trying to control like HalfLife in Minecraft and he just simply can't do it. He needs trying to use, you know, this code here, but then you can come down here. Uh, actually, let me see. Is it this one? Yeah, it's this one. Okay. Anyway, the main issue is you need to send keys via right you you the solution is to use scan codes instead of VKS. Someone can post below what the f that is but I don't know what that is. Um but anyway it's just a different way of I guess sending the keys and the reason is um from my understanding is because games are using DirectX. So you need to send keys how that expects the keys. Anyway, um regardless, take this code. This is your new code. So, we take this code, copy this code. Um and then also just note there's the scan the full scan codes are here. So, cuz like this is a W. You would never know. So, so anyway, um take understand that's a W. So, then we took I took that. Um we you guys aren't a part of this yet. Um, let me do this. And what I'm going to do is just copy and paste here. Let's do direct keys. And open direct keys here. And I need to recopy this because I copied that other thing. Oh, you can't really see that I'm doing that. There we go. I'm copying this whole thing. All right. So, we take that, paste, save. Um, and then this, just for the record, and so is actually some of this. This was code that I took from Stack Overflow. So, it's actually a Creative Commons license. I'll have to update that on the Stack Overflow as well. Uh, anyway, [Music] um, blank. Okay. So what we need to do is find where I just pasted that. Where did that go? Direct keys. Did I close it? No, I didn't close it. Um, so probably, you know, you can keep this, but then we want to define some some constants. So some of the constants that we have like bas basically for now we just want wd. Eventually we might want quite a few more um like mouse movements, click or clicks rather, maybe mouse movements one day. Um, even in a car the space bar is the E bra. So stuff like that. But for now, let's just save a few. So w is equal to zero. Actually, it should be probably lowerase. Um 0x11. A = 0x1 capital E. S = 0x1 capital F and D = 0x 20. This way we can just import this and we don't need to type those and it's a little more clear. And then we can use the send key which is down here. Make sure you can see that. Um here we can literally use press and release. That's fine. Um I'm going to leave this here just for my notes, but I'm just going to if name equals main it. Cool. Okay. So we'll save that. Fantastic. Close. We'll come over here. Um, actually we can leave we'll leave this code here. We're no longer going to be using pi autogoy because it failed us. And now instead what we're going to do is we're going to say from direct keys because we're just we just called that script direct keys. We're going to import um press key. We should press key. Let me think here actually because I think we need two things again. I've already, this won't be the case always, but I have already tested this and I swear I didn't need to use release key h let's um let's leave it there. Let's let's try just press key and not release key because I don't think I needed it. Um anyway, press key and then W A S D. Um okay. And then rather than pi out of gooey key down and uh key up, we're just going to do press key. So press key. Press key. Save that. And now let's run this one more time. Hopefully we'll get what did we not do? S. What did we do? Or maybe we didn't save it. No, because we got W and A. Let's go to direct keys. Where are you? W A. Oh, we lowercased it. Okay. Save. Let's try again. Um, and I'm actually going to close this so we can see the countdown so everybody can see it. Four. Three. Oh, this will only do it. What? What? right key. Sorry. An integer is Oh, this is confusing. Like I said, I've already done this. Why are you fooling with me right now? Oh, God. Harrison, come on, man. All right, save that. I didn't make this mistake originally. Okay, so I use strings here. These need to be Oh, you can't see it. Glad I caught that. Anyway, sorry. Um, yeah, that was press key was in string W form. What we actually need is full uppercase, you know, W what we imported. So, it's that little code. Try again. This better work. Oh, he doesn't ever release. That's weird. I swear I Oh, boy. We're going to be in trouble soon. Um, interesting. Um, that's really weird because I really, like I said, I don't remember needing to do the release key, but clearly we do. And that makes sense, but that's just weird that I don't recall doing that. But anyway, um, so yeah, release key. So instead, we can release key W. And then we're basically at the end of this tutorial. I just want to confirm that that works before we go to the next one. Settle down. All right. Forward for three seconds and then he's hopefully stops. Release key. Not fun. Did we not? Well, we didn't import it. Ah, Harrison. Release key. One more attempt. Surely this will work. I have faith. Just I'm just orienting it. Okay. All right, we're done. Okay, so at this point, I'm just going to move. Um, at this point, we have um confirmed that we can um take input. Well, we haven't fully confirmed, but I mean, we we should be pretty confident at this point, I think. Um, but what we can do is come down here, paste. We'll cut this. Cut this. Or Oh, you can't see what I'm doing. God dang it. Anyway, I moved the code that was up here to down here. I cut the actual um uh movement stuff and I'm going to tab it over. So now this is hard hard to keep all this stuff on one screen. Uh let me do that. I suppose I don't know. There we go. That's There we go. All right. So process image. I just moved this stuff down here. I've got this like little countdown here. The time starts. And then just down in here, I'm pressing and releasing the key. And now we can we'll confirm that we can actually um run this pressing keys and uh doing the open CV. Obviously the I can't imagine a reason why this wouldn't work but we just do kind of want to be 100% I suppose. Um the only pro well actually here one problem is um so that does bring up a good question actually. So good thing we ran it. One problem is with the time.leep sleep that we're throwing in here, it doesn't update the actual CV2 window. So, when you press and release that key, you'd probably need to use some other form of logic rather than sleep, which is totally fine. We would rather than just sleeping randomly, our logic would actually be press key and then we would have some sort of event that, you know, we're about to cross the lane, let's press this other key. Once we're content that we've made enough, we release the key. So, you wouldn't be using a time sleep anyway, but if you do throw in a time sleep, it's obviously going to 3 seconds every time you update that window. Anyway, cool. In the next tutorial, um, we're probably going to leave the whole key presses behind, um, and work on simply trying to actually detect lanes so that we can create something that can at least in a rudimentary form drive or some way of actually just automatically detecting whether or not an AI that's just simply exploring the world is within the bounds of the lanes. So, um, and now we're kind of getting into something that is completely unplanned at this point. If you have suggestions for improvement or ideas or things you want to test, uh, leave them below because, uh, this can be totally shaped by you guys. So, anyway, that's it. Questions, comments, concerns, whatever, feel free to leave them below. Otherwise, I will see you in another tutorial.