PyTorch in Robotics - Yisong Yue, Caltech

Key Takeaways

[Music] for those of you don't know me my name is Z song you I'm a faculty member at Cal Tech I've been at Cal Tech for about five years studying all aspects of machine learning today I want to talk about some of our work at the intersection of learning and robotics and control and in particular we'll be talking about risk aware machine learning for dynamic robot control okay so that's a bit of a mouthful so let's unpack what these words mean so dynamic robot controls so what our dynamic robots well in my definition dynamic robots are robots that need to exhibit some level of agility so here's one example where the robot a drone must interact with the ground and fly really close to the ground but not touch it so this is a level of agility that can be quite hard to achieve with your off-the-shelf consumer drones and yet we were able to demonstrate how to do this and I'll talk a little bit about this in detail later on this presentation another example is of a dynamic boundary condition so the ground is a static boundary condition and a dynamic boundary condition you can think of as multiple drones interacting with each other so for example when one drone was on top of our second drone it creates what's called a downwash effect that can actually cause the lower drone to crash and so if you can learn how to control against that then you could perform very agile and graceful dynamic formations and that were previously not possible or very hard to engineer using classical control approaches other examples that will you do at Caltech include things like exoskeletons and prosthetics so you know how do you design exoskeleton gates that are optimized for the comfort of each individual person let's say an injured person while being stable safe upright and all those other factors simultaneously so these are just some of the examples that we've been working on at Caltech where we really need to be able to perform agile or dynamic robot control and so our goals are to have robots in these agile and dynamic settings that exhibit of course agility safety comfort aesthetics power consumption efficiency so on and so forth and it turns out that if you talk to a classical controls person or a roboticist leaving some of these dynamic behaviors can be very challenging because the environment is not perfectly modeled if you don't model the affordances you don't model the the boundary conditions if you don't model some of the stresses and deformations if the robot is as it's undergoing stress if you don't model some of these things very accurately then your simple models of the world will cause the robot to let's say fall over or crash and so that's where machine learning comes into play how do we use machine learning to learn all of these extra high order effects that are not model using nominal models of control in order to achieve these types of agility and aesthetics and comfort and so that's what we're doing at Caltech through a new center called cast cast stands for the center for autonomous systems and technology it was founded about two years ago and it basically is a center that allows faculty from electrode engineering mechanical engineering aerospace and computer science to come together and work on these problems very synergistically so if you're interested in learning more the URL is very easy cast at Caltech that edu invites you to visit our website and learn more it's ok so this is gonna be a machine learning talk and we're gonna talk about how to use machine learning to put it into a safety-critical real-world robot that's performing agile maneuvers by learning all these high order effects that we cannot model you by hand one thing that immediately jump jumps to mind is the fact that ok these robots because they're performing these potentially unsafe actions we need guarantees on safety stability on comfort when you fluent in airplanes for those of you traveled when you flew in an airplane last night to come to San Francisco that airplane has been certified by the FAA and by other governing bodies and by various companies to satisfy certain safety standards and a lot of that comes with theorems and proofs but learning as we use it today is largely a black box deep neural Nets to a first-order approximation is basically a black box certainly from the perspective of let's say certifications at that level that we will be required for these types of safety critical systems so how do we actually bridge this gap how do we get the power of machine learning get the power of deep neural Nets and yet get the safety guarantees that we need for safety-critical robotic systems and so at Caltech we've been studying this from a variety of perspectives as you can there's no silver bullet solution and so we can dichotomize or or categorize the various approaches along what is needed and what is useful and so my personal research has sort of decomposed these area into four philosophies if you will on the top left is what I call a compositional a hierarchical blending where you have a rule-based system you have a model basis then let's say a differential equation of gravity and and so forth but we blend learning as a component into this model on the top right we flip the script it's also compositional blending but rather than blending the learning into the model we blend various models within a larger learning based framework so you could think of this as a for example a neural net with model-based components as part of the neural architecture so this is of course something that's very exciting and a lot of people are thinking about on the bottom right you see an example where we have a planner but actually coming up with a plan that's probably safe could be lets say np-hard and so how do we use learning to learn accelerated solvers to solve these kind of problems potentially in real time so this is reminiscent of learning for accelerated research which you've seen to be successful in playing games so it's a variant of that and on the bottom left you see a additive blending where we blend learning based and model-based approaches additively of course in I don't have all day so in the interest of time it's gonna give you one quick example in compositional blending we're blend learning into an existing rule-based system and do so with theoretical guarantees and so the example is the neural Lander and so here the goal is to be able to gracefully and safely land a drone touching the ground in a very agile and provably safe manner for those of you don't know many drone controllers that you buy off-the-shelf they have trouble fully landing with the drone typically it gets close to the ground and then you have to just have to cut the power and the drone sort of falls awkwardly to the ground and the reason why that's the case is because the ground acts as a boundary condition which creates what's called the ground effect and the ground effect actually disrupts the nominal aerodynamics models that people use to create drone controllers and so whenever the drone gets close to any boundary condition has trouble actually controlling itself and so let's use learning to get around that and so this is what I call compositional blending so what you see on this slide and again you know if you're not an expert in control you this may be a little bit foreign to you but just keep in mind that most of this is very standard this is a standard drone controller that is implemented in almost all commercial drones or a variant thereof and there's one thing that we added and that's F that's F sub a and that is the ground effect classically it's hard to model this ground effect and so people just ignore it it's just always zero and we're gonna use learning to learn that one component and integrate it into a conventional controller that's the basic idea and so here's an example so at caltech we flew such a drone we try to predict how the aerodynamics evolve as we fly too close to the ground and the aerodynamics are wrong and we basically do least squares regression on the residual at in a nutshell that's what we do and we fit innernet to this residual to predict the residual ground effect from flying close to the ground and what you see in the green dots are the actual ground truth measurements what you see in the orange lines are classical models that people in control theory in aerodynamics Theory sort of developed by hand so equations of motion and aerodynamics that they built by hand and you can see that there are a very poor model of actually capturing real ground effects and yet the neuro net basically learns it perfectly or within a very very small error tolerance and this is particularly useful because now we can just plug this in to our nominal control model - in order to achieve agile drone control so here's an example on the left you see our neural Lander approach on the right is a standard base line without the residual dynamics learned and you can see that the nominal controller cannot properly touch the ground and typically what people do is just cut the power in Atlanta awkwardly and yet our controller can gracefully and safely to land and we can now even get it to land at very high speeds although we don't have a cool video for that yes yet so why don't we watch this one more time and so I hope this gives you a taste of how it pi torch and deep learning can be used to integrate with modern robotics to build agile safe dynamic robotic controllers that can perform interesting maneuvers interacting with the environments close to boundary conditions this is in many ways one of the next frontiers in robotics and control and so I like to thank my collaborators there of course too many people to thank most notably the students but here are the faculty that we span electrical engineering mechanical engineering aerospace and computer science and we work together through casts and I hope that if you're interested you come check us out so thank you all for your attention [Applause]