Military Robots

Key Takeaways

Oh world it's Suraj and lo robots we're gonna learn a bit about how military robots work and then teach a simulated robot how to walk using a reinforcement learning technique called proximal policy optimization military warfare is as old as mankind and the better the technology the more likely the chance of winning is so it makes sense that militaries want the most advanced weaponry they can get their hands on by whatever means necessary ideally militaries can minimize deaths of their own soldiers while maximizing their effectiveness that means removing the human from the loop as much as possible which is where the field of military robotics comes in the first wheel advances their team from none other than electrical wizard Nikola Tesla in 1898 Tesla demoed a controlled motor boat to a government representative it laid the foundation for the first unmanned vehicles and weapons as World War 1 began Germany used radio-controlled motor boats to ram into enemy ships containing more than 300 pounds of explosives during both world wars inspired by Tesla's initial invention using radio waves as a communication signal was a revolutionary idea the Germans use it to deploy the first remotely piloted aerial drone called the fritz they dropped a device at a high altitude from a bomber then Bombardier would steer the fritz via a radio link using a joystick and it wasn't just the Germans getting in on the radio-controlled fun Americans dropped more than 450 radio remote-controlled glide bombs during World War two remote-controlled vehicles on land air and sea have improved in the decades since World War two as more than 40 countries have developed technological breakthroughs in that regard there are three important trends happening right now that are causing rapid advancement in military technology namely the plummeting costs and soaring performance of computer hardware the rise of cloud network robots and of course advances in machine learning put together these trends mean it's getting easier to create more powerful robots soldiers no longer need to aim at targets if they can just use a convolutional network to detect enemy's commnets our neural networks that can learn the features of any type of object its trained on be that a certain type of vehicle or a type of soldier using a pre trained network soldiers can Don a set of goggles that show a computed bounding box around our targets or even feed the computed coordinate to a weapon and have its position adjusted accordingly and fire automatically effectively using auto aim reinforcement learning has been an effective tool to get robots to learn how to navigate virtual environments some soldiers go through extensive simulated training before being deployed to the real world and an RL agent could prove useful as an unpredictable and adaptive enemy during combat simulation for a range of soldiers from fighter pilots to ground soldiers these agents could use techniques like q-learning or policy gradients or actor critic to learn how best to avoid enemy fire and even create effective strategies to confuse the enemy before attacking AI can also be used to change the medium with which soldiers control their weapons they can issue orders via voice command instead of having to use a keyboard in intense situations thanks to developments in natural language processing speech recognition is getting really good and recurrent networks helped make this possible if we have some labeled audio data set we can train a recurrent Network by slicing the audio up into small chunks feeding those chunks into the RNN sequentially and generating a prediction for the next chunk computing an error back propagated the error gradients and repeating that process lots of times since we're current networks are made for sequential data audio is a perfect use case after training they can recognize human speech with incredible accuracy and this essential for mission-critical applications governments have mountains of data from surveillance systems and satellites of all kinds finding disturbances using unsupervised learning techniques like anomaly detection via auto-encoders war doesn't just have to happen in real life it can happen online as well today the process of finding and countering bugs hacks and all sorts of cyber infection vectors is still effectively artisinal professional security experts search millions of lines of code to find and fix vulnerabilities that could be taken advantage of by users with ulterior motives all of this could be automated using deep learning to find the features of vulnerable code and learning to patch bugs by itself so rule-based intelligence can only do so much but by integrating learning algorithms machines can become far more capable when a human is in the loop they are able to make the final call the highest level decision that's the one that the human makes but as AI gets better its able to make more decisions for itself so how many decisions should we delegate to AI how much is too much take the idea of a swarm for example any aircraft manned or unmanned can be brought down by a single missile but a swarm can take multiple hits and keep going it's a collection of drones that can coordinate together as a single entity one could take out enemy systems while the other creates a distraction while the other picks up important cargo evolutionary techniques like particle swarm optimization can make this happen the US Department of Defense recently released a document called a human systems roadmap review that reveals a plan to use AI to create autonomous weapons using social media analytics to make decisions on lethal force with minimal human involvement it shows that while having a human in the loop is necessary for the near-term therefore turn vision is self-aware systems they define this as systems that have perception reasoning and intelligence allowing for entities to have existence intent relationships and understanding in the battle space relative to a mission that means drones that coordinate amongst themselves make decisions for themselves prioritize by themselves and of course decide who to kill and when to kill them by themselves that's kind of scary and reminiscent of the infamous Skynet system from the movie Terminator in the movie the military system becomes self-aware and decides to kill all humans and no one can shut it down we know of one system that can't be shut down and that's Bitcoin because the blockchain miners must verify transactions using the proof-of-work algorithm it would require more computing power than the 500 fastest supercomputers in the world combined to shut it down and no one has that much computing power an AI agent that lives on this kind of blockchain couldn't be shut down and if its objective was to say destroy humanity and it has access to weapons of mass destruction that's how Skynet happens IRL so how do we stop this we've got to spread awareness and educate ourselves on how AI works the more people that understand the power of AI the less likely it is that governments will abuse this power and the more likely it will be used for beneficial purposes more research into creating explainable AI systems that give us a detailed list of why an AI made a certain decision will help unravel the black box of deep learning and help fuel a debate for policy makers as they regulate this technology so to give an example of how a robot can learn to walk we can use a reinforcement learning technique called proximal policy optimization this is two randomly initialized neural networks and a teacher that rewards forward progress the policy gradient technique takes steps in the direction that improves the policy a similar technique is called trust region policy optimization or TRP Oh the idea is to take steps in the direction that improves the policy while simultaneously not straying too far from the old policy that's the difference making too big a change from the previous policy in high dimensional environments can lead to a dramatic decrease in performance a little forward lean can help running speed but too much causes a crash a naive solution is to take minuscule policy steps but the question then becomes how small a step TR Pio takes a principled approach to controlling the rate of policy change the algorithm places our constraint on the average KL divergence between the new and old policy after each update so proximal policy optimization is an implementation of TR Pio that adds the KL divergence term to training a loss function with this loss function in place we can train the policy with gradient descent like a typical neural network in our PPO algorithm we capture sequences of states actions and rewards from our environment and added to our data batch line 10 adds value estimates to each visited state from the rollouts with predicted state values in hand we calculate the advantages and add these to the data set in line 11 the advantage of a state action is how much better or worse an action performs than the expectation of present policy from the same state we update the policy in line 13 finally we update our value function to reflect our latest data in 114 we use the present data batch and the previous data batch to smooth changes in the value function for our policy update function we store the old policy and compute the KL divergence as we make policy gradient updates and after only 25,000 training episodes our humanoid will start learning how to walk it's pretty hilarious to watch the progress in the meantime all right so three ending points here militaries can use AI to create autonomous weapon systems and that means less and less of a need for humans in the loop a Skynet's like scenario can occur the public isn't made aware of AI dangers and governments go unchecked and proximal policy optimization uses two neural nets and the teacher to forward progress to train an AI to complete an objective this week's coding challenge is to use the PPO technique on a game of your choice details are in the readme poster github links in the comments section and winners will be announced next week please subscribe for more programming videos and for now I've got to go prevent Skynet so thanks for watching