Self-driving cars with Python and TensorFlow update v0.04-v0.06

Key Takeaways

what's going on everybody and welcome to an update to the python self-driving car and really plays GTA series in this video I'm going to be talking about some of the changes that have been made recently and what's coming for the future so first of all some of the updates that we've made basically since the last video I think I had 0.03 in the last video 0.04 was just more waypoints to 0.03 just to see if that would continue on and then 0.05 was the implementation of Xbox 360 controller emulation so basically what happened there is I used V joy to emulate just a simple joystick in Python and then I used a a program called like X 360 C II which could take your simple joystick so if you bought like just a cheap joystick and you wanted to use it with something that required an Xbox 360 controller you could so that's you can basically map pretty much anything to an Xbox 360 controller so with those two things with VJ plus X 360 C II is able to emulate next box 360 controller with Python so that's what I did with 0.05 so basically the way that works is the output of the neural network is just key presses right it was either WASD or combinations like W a or W D or si and SD and so on so take those take that output but the output was never totally binary right it was almost like it was like a percentage it would be like 60 percent forward 10% left 0% right 5% slowed down and so on down the whole list of key presses that were possible so what I do is I take that output and then map that to the controller so let's say the turn is 50% left 10% right well take 50% - that 10% right that's 40% left so let's take a 40% left turn so set that joy 40% over that's basically hit there's there's actually an algorithm we used to pick the right values but pretty much that so that was version 0.05 and actually that made some some pretty significant improvements let me make sure the sound is muted here and what I'll do I might wind up being too loud but I pretty sure this is zero zero five yeah zero point zero five here so what it can do is actually make much more fine-tuned motion like whereas before it would if it said okay the the largest out you know the largest or basically what is the output node is that happened to be a left or right and it would be full left full right whereas here what you can see happening is it's got you know some throttles happening some turn is happening but it's so much more subtle it can do a full turn if the output layer is really like you know go all out but for a more fine tune required turning it can now actually do that whereas you know before it could weave but it was very violent weaving whereas what you just saw there was very subtle to get it through what it needed to get through and then this will be another example of just fine-tuned steering I'm not sure this is weaving I think this is just actually taking turns right especially this term up here was always really really difficult still kind of difficult obviously you just took a shortcut there he's kind of cheating but a lot of it to just correcting from terms and stuff like that like you just couldn't do also going in between two vehicles was always really really hard for it cuz it can never get perfect because it again would always press a full on key which was much more violent and I think this is just driving in the hills basically and these are just examples again as always you can go to the stream watch it 24/7 because it doesn't drive this perfectly all the time spoil alert but normally just especially in the hills the hills always gave it a really hard time just because there's always stuff to get stuck on but again as you can see he's able just much more finely tuned steering its everything's just much more smooth with the Xbox 360 controller emulation like I said smooth look at them I'm taking this position anyway okay so there's that let's see oh and this also you know one other thing that comes in is being able to control a throttle so for example if here's a real tight turn and hard to tell but he actually pulls back on the throttle significantly there normally he would have just smashed into that building like the older model every time it came through this area would fit right in there where this one actually can control the throttle right it's not actually a hundred either a hundred percent full throttle or not just like the turns it can be very granular so that has helped significantly and then here's some more examples I guess or this is one with I guess throttle and I don't know if it's going to turn or wet here it's been a while since I posted this one just barely made that exit yeah normally like if he hadn't slowed down for those turns normally it would have just blasted into that wall same thing there he just slowed down quite a bit okay so then 0.06 is the introduction of a speedometer so actually 0.05 had two million labeled frames and it drove really well arguably I'd say 0.05 actually drove better than possibly 0.06 I think it's just because it had a lot more data but with 0.06 started getting curious about adding the speedometer so I went ahead and added a speedometer I can't remember I think in the recordings these are like the sound is really loud so if it's hard to hear me I apologize anyway so with the the speedometer it seems like at least to me it does appear that it at least starts is starting to understand how fast it's going it seems to slow down from a high rate of speed much better than it did before and everything seems to be a lot more smooth now we really want to you know visualize the actual layers to be a hundred percent certain that it's actually seeing the speedometer I was mostly just kind of toying with it just to see like does this work at all like is there any hope for a speedometer being useful but it does seem to me that it is useful now the problem is with visualizing the layers like with like something like deep dream the only way I know how to do it is with raw tensorflow not with TF learn because you have to get like the actual like the graph file and through TF learning do get a graph file but it's not like the same format and it's just it's just a but simply above my head for how do we get what we want for the deep dream so you know how to make D dream with TF learn let me know anyway um I see some of the issues I don't know if he's actually going to run directly into something here or what oh no this is oh the tunnels so one of the main issues here is since we're using waypoints like waypoints came from 0.03 a lot when he enters a tunnel it's just kind of a feature of GTA to simulate the loss of GPS so so the waypoint disappears okay so then he's got to navigate the tunnel so as you can see that case he actually made the tunnel op many times he'll go through that exact same tunnel and he just loses his stuff because he just doesn't know what to do without that lovely Purple Line oh yeah and then we gave him a ramp buggy this is just interesting so one of the problems still is that he just Rams right into other vehicles again I think it's just because he's got so much of a focus on on that mini-map basically in that purple line so he still runs into vehicles in this case it's not a big deal because he's driving the ramp buggy so he just kind of goes right but anyway that's that so onto the actual stream now quite a few changes that happened to the stream as you can see we actually have a motorcycle working now it works okay in third-person you definitely don't to watch it in first-person but I think just because it's such a small thing on screen that just doesn't seem to bother too much if you drive like a car and third-person though it just constantly like trying to avoid itself because it just sees the car whereas the motorcycle he clearly just does not care if there's a motorcycle so so anyway some of the other things that we've added or as you can kind of see yeah you can see the chat is we've added some commands so the boost command just gives them like a boost of speed just really short term just in case you stuck or whatever you can give them a boost also rockets shoots rockets I'm not sure if anybody's going to do it so I'll just do rockets times two and then it takes a second so there's just like a stream delay by a twitch so it's like something like six and eight seconds usually hopefully we'll get somebody here direct hit anyway so does a few chat interactions there you can think the mod Daniel for that one you can also do stuck to get him to teleport back to the road if he happens to get stuck and you don't want to wait for the actual automated mod to teleport him back and then hopefully in the future we'll have some other internet you chat interactions like turning on and off police maybe changing weather and that kind of stuff somebody got happy there also you can change the car so as you can see here the car changes available someone's about to change it it should change to ramp buggy here in a moment again because of the stream delay there's really nothing I can do about that it's just it is what it is so now he's driving a ramp buggy which is just basically a giant ramp so if he does happen to hit anything it doesn't really matter it just goes flying so anyway also if you if you like to you can actually subscribe to the actual story looks like someone did it boost at some point there if you have Amazon Prime you can actually subscribe to the stream completely for free you can subscribe to one stream a month if you have Amazon Prime and it's connected to your twitch account so to me it's the same thing as any other sub so it's like you know free money so um so yeah there's that trying to think if there's anything else you know for coming changes for things that I'd like to change in as far as the AI is concerned oh nice so it's not just stuff so when you step you get some fireworks in game as well see if we can get some sad nice anyway um so let me meet this again I like I said I think when it records sound records actually from the desktop and it will come through the microphone as well but I think it's pretty loud if you let it come through the desktop anyway some of the changes that I'd like to make in the future obviously I'm still working on like memory and reinforcement learning reinforcement learning is actually already done the problem is it's really slow at learning so it would take probably months to see like noticeable changes with what I have going I'm sure there are people smarter than me that could make it happen much faster it's just I can't figure it out so anyway there's just so many variables in Grand Theft Auto but basically the way I'm doing it is as you can see he's following waypoints and then we can take the way that we can generate a score is how long does it take him between two exact waypoints so I have a list of like 30 total waypoints that's a lot of combinations I could make that list much smaller to get those combinations more frequently but basically how much time does it take in between those two waypoints given that we don't use a teleport in any way so it's just purely just getting there not getting stuck and stuff along the way and then we can use that time to generate a score of how to get there now there's so many variables especially with changing vehicles the only vehicle that's really truly unfair is probably the ramp buggy other than that maybe the motorcycles because they're much skinnier than normal but otherwise all the vehicles are relatively equal I mean some of them are much faster than others for example the monster truck is kind of let's be serious now so it's like I would throw out that data but we can use that but the bottom line is it's just very slow to learn it's just a very complicated task so there's that also I'd love to add memories still working on memory I've done it done recurrent layers on this data it just didn't perform any better than the pure convolutional neural network and yet it costs like 10 times in processing and maybe that's why I didn't do any better right because it had less frames per second that it could go through so it's reaction time was just simply worse and I've already seen that if I can improve this frame rate like right now it reads and understands at about 60 to 90 frames a second maybe on this machine actually more like 6070 but for some reason on my main computer which is actually on lesser Hardware if this machine process is about 80 to 90 frames per second and this in running Charles on this machine he does way better so I'd love to eat out more frames per second so anyway maybe that's all it was with the recurrent layers and if we could get 80 to 90 frames a second with the recurrent layers then it would be just like so much more better but I really don't know at this stage so there's that also um like a non you know self driving change I'd like to make is one of the things that's really interested me about this project is from the very beginning when I put it up on a stream was that people were being entertained by an AI we have you know well initially we had thousands of computers because of all the news that it got but lately it probably like the last month we've had a consistent summer between 20 and 60 viewers at any given time just simply watching an AI play a game that's pretty interesting to me so anyway one of the things I'd like to also do is get a chatbot going for Charles so bring Charles in as a chatbot and interact with people in the actual chat and try to make a full-on actual streamer and all that at least for the initial stuff I probably will have some how to create a chat bot with deep learning I'm not sure I'll cover the whole project but definitely something out there for that if I do get around to doing that there's anything else that I wanted to cover in this one I think that's that's basically it oh the other thing the other thing besides doing reinforcement learning and possibly memory I also chose grant to follow because there's other tasks that you can do so obviously you could I suppose do missions and stuff but I don't really think I'm going to go that route I don't even know how I would around while we're doing is driving I would really like to get Charles out of the car as well so actually I would like to be able to have him both drive and not drive I'd like him to actually be able to commit GTA but alright that would be that's basically the epitome of Grand Theft Auto right you got to be able to steal cars so anyway so I might add that as well later on I still want the driving to improve the driving is actually better than I ever thought I would be at this point but it definitely has a lot of room for improvement so but I'd also like Charles to be able to do other things besides just drive a car I think that'd be make for more interesting stream as well anyway a lot of information there hopefully I'm not leaving anything out a lot of changes have happened also just to give credit where it's due the chat mod daniel i'm not going to pronounce his last name this guy is responsible for all of the twitch interactions so shooting rockets changing cars the fireworks when you subscribe and donate and all that kind of stuff the ability to drive a ramp buggy because by default the ramp buggy actually only has a cockpit view it doesn't have a hood view so he actually customized that the hood view also the fact that we can drive third person motorcycles such as auto switches third person all that stuff is thanks to him so if you're enjoying those those changes make sure you you highlight them and say thanks in the in the chat anyway that's all for now if you want to check out the stream it's twitch TV slash semtex again you can subscribe totally for free if you have Amazon Prime you can also donate on the stream or go to the patreon page or whatever this is running 24/7 so come check us out otherwise questions comments below see you in another video