Build a Self Driving Car in 5 Min - Fresh Machine Learning #6

Key Takeaways

I'm glad I can book a self-driving Uber now hey this car isn't self-driving and it will never [Music] be hello world it's SJ and in this episode we're going to talk about how self-driving cars work then Implement our own self-driving car in a simulated environment self-driving cars aren't in the realm of Science Fiction anymore real companies like Toyota and Ford have millions of dollars in R&D pouring into this technology services like uber and lyt thatr pay human drivers will soon deploy entire fleets of self-driving cars so prepare for Skynet just kidding in 2 or 3 years we're going to start seeing hundreds of thousands of self-driving cars being sold to regular consumers get your self-driving trucks right here but how do they work well when we humans are in the driver's seat we're observing our environment by receiving an input of our surroundings and simultaneously processing it in order to make a decision of which way to move the steering wheel this can be translated into a machine problem known as slam slam or simultaneous localization and mapping and it's something all self-driving cars do a self-driving car is usually outfitted with a GPS unit an inertial navigation system and a range of sensors it uses the positional information from the GPS and navigation system to localize itself and the sensor data to build an internal map of its environment once it has its position in its internal map of the world it can use that map to find the optimal path to its destination that avoids any kind of obstacles be that dead babies or Pokémon once the car has determined the optimal path to take that decision is then broken down into a series of motor commands which are fed into the car's actuators that's a highle description of how they work but roads are complex it's not just about avoiding obstacles there are weather conditions that require changes in the way you accelerate different types of road signs and situations that you probably couldn't ever predict a recent paper came out just this year called long-term planning for short-term prediction these guys proposed a planning algorithm for self-driving cars specifically one that would be able to make immediate actions so as to optimize a long-term objective an example they used was a roundabout when a car tries to merge in a roundabout it should decide on an immediate acceleration or breaking command while the long-term effect of the command is a success or failure of the merge traditionally planning for self-driving cars is done via reinforcement learning the car learns to continuously correct its driving capability over time through trial and error when training the car or agent observes a state s that is a scene that it observes and takes an action a and depending on whether whether or not the action was good however we Define good it can receive a reward R then it moves to the next state s and the process repeats the goal is to maximize the reward and that depends on a policy which Maps state to action that it learns over time the state action value function is called q and it helps find the optimal policy but it can be super hard to learn Q in an environment as dynamic as roads with multiple cars it's not just a problem of predicting your own car's actions you have to be able to predict other cars actions as well so to solve this problem of learning que they used a deep recurrent neural network to learn a policy and the input to the neural net was a vector that contained both the predictable part the speed of the car and an unpredictable part the speed of other cars so it could learn from both they applied it to just two features adaptive cruise control and merging roundabouts but is there a way to make one learning algorithm that can learn everything from the ground up the technical term for this is end to end and an even fresher paper that was released about 3 months ago tried it a team from Nvidia put three cameras on a car windshield to receive input data fed this video data to a convolutional neural network and features were learned by themselves they didn't explicitly decompos the problem into subm modules for different scenarios their CNN mapped what it saw from the input directly to steering commands it was first trained in a simulation with pre-recorded video then trained by a human driver they got great results but it was hard for the paper authors to differentiate the feature extractor part of the neural network from the controller part so it was difficult to test each that's why most real world car manufacturers have decided it's not yet possible to test and verify an endtoend system they end up just making software where each module is separate and can be tested on its own a hacker named George Hots built a self-driving car in his garage with just a couple of cell phone cameras and the total cost turned out to be just 1,000 bucks let's train our own self-driving car using q-learning to drive itself without running into obstacles after we declare our Imports let's write our training function for our car first it'll take in a neural net with a set of hyper M as the parameters then we'll Define some variables for the number of frames we want to observe for both training and testing we'll then Define our positional variables for localization we'll create a new game instance and get the first state of the game instance we'll also set a timer for tracking purposes then when we start building experience replay we'll update our positional variables then choose an action depending on the state randomly if the random variable is outside of our constraints we'll get the Q values for each action to help us find the optimal policy we'll take that action and if it is valid we will get a reward once it's done observing the game and building experience replay we'll start training sampling The Experience replay memory and getting the training values it'll then train the model on this batch that is a neural network then update the starting State and if the car dies log the distance and reset the car's life finally we want to save the model every 25,000 frames in the weights file let's see how it looks in a simulated environment it constantly tries to avoid obstacles through a mix of reinforcement learning and a neural net once you've got it work working in the simulator you could Port it to a real RC car and have it self drive all over your room links down below for more info definitely subscribe for more ml videos I've got to go descend some gradients so thanks for watching