SLAM Robot Mapping - Computerphile

Key Takeaways

and basically I'm just going to do essentially what we did with the the handheld version just now that it's on the rot today we are looking at the frontier device it has an embedded computer a multicamera system a lar and inside there is hidden an inertial measurement unit the cameras work uh pretty much as our eyes uh the lighter basically allows you to get a point cloud of the environment by firing lasers at the environment and by measuring the return time it can get you a metric measurement of what's around you and the IMU works pretty much as our internal year it gives us uh an idea of where gravity is and how fast we are rotating all of these sensors can be fused together to answer the main question in robotics called uh slam simultaneous localization and mapping so the localization bit is knowing where you are and mapping is what's around you slam is uh pretty much like a chicken egg problem uh you don't know where you are you want to know where you are in the environment but at the same time you also want to know how the environment looks like if you walk around you're going to see um an orange line that shows where we were before so if I keep walking around and I come back to the same place I was before I can show you the main ingredient of Slum which is Loop closure there you go so here you see a red line this red line means that the system who recorded all of our previous position recognized if we were back to the same place and in this way we can correct all of the error we have accumulated this far and have a better Mar by adjusting it knowing that we were there before the screen was just to Quick visualization but all of the problem is being solved in 3D so here is the exact same problem we have solved before so here you see the table we see all of the mapping 3D being built and the loop closure is right here so after just a very quick walk we have a complete 3D map of the environment and we also have our pass trajectory the map is mainly built using the lier because it gives you a continuous stream of Point clouds but to accurately know where we were uh we also need to fuse the IMU because there is physics inside so the IMU tells us your acceleration for instance IM is IMU inertial measurement unit yes if for instance there is an error of registration between two point clouds uh and we don't use the IMU uh the system might just think we teleported somewhere else but the AMU is telling us no your acceleration was like that so your velocity should have been like that and you should be here instead of somewhere else so if they use the IMU on its own it'd be kind of dead reckoning it would be guessing yes so there is a component of that droning but this is fused together with uh the other sensors so if we were doing that droning all of the time then we would end up on the moon because there are many smaller errors that gets integrated over time so the IMU is just giving us an idea of where we should be between two camera images or between two lighter uh Point clouds and then the measurement from the point clouds and the cameras uh uh kind of correct those and are merged together so we can have a walk and go downstair in the lab so we unofficially call it the bat lab and the reason for that is is because we have a huge elevator that can carry an entire car here we are going to show the quadri section of our lab what's funny is the camera can see some of the lidar yeah yes because it's um it's near infrared and Camera don't have infrared filters there you go so here we have different quadrupeds today we're going to see the spot one this is all the way from where we we were before here with the the round table the loop closures and we went downstairs all of the compostion is inside here but for E of visualization we have uh some interfaces on the phone and on the computer most of our software is compatible with Ross which is a framework that is widely used in robotics Community there are many libraries and programs ready to use to accelerate robotics uh this one in particular is called Ros board and it's running a little server on the computer and on the other side there is a client that gets the outputs from our algorithms as um messages and these messages are then uh relayed and visualized in the in in the web page web browser interface so on the phone that's just basically running yeah I mean the phone is yeah it's just a an optional that we we use especially when we do handheld uh missions we want to have an immediate feedback of what's going on everything is going all right so we have developed this interface slam is very useful for robotics in particular because it allows us obviously to know where the robot is and what the map around it looks like so um just going to drive around the lab a little bit um and then we'll be able to see kind of uh what the lab looks like [Music] and now that we're kind of done again you can see we have a map of the environment we have a loop closure and slam is essentially a key component in enabling robot auton so a lot of the work that we're doing in kind of remote inspection Mission planning all of those kind of things relies on the ability of the robot to know where it is and the structure of the map to actually do useful things um you know nuclear decommissioning those kind of things is what we're um looking at the handheld has its uses but also if we can do this with a robot obviously we don't have to send people in to the same places sometimes it's easier to access places with humans so the handheld device is useful but other times we'd prefer to use a robot so we have both options this is the one we saw in bristles yes on the front we've got what we call a frontier device it's running the exact same mapping software but we're just not running the autonomy part of the system right now what I'm going to show is the algorithm that is running inside the frontier and the main technique that is using it which is called Factor graph so a factor graph is a way to solve uh this slam problem uh by modeling the unknowns as nodes so we might have uh our position X1 and then position X2 position X3 and so on and this is what we want to know in addition to uh landmarks which are basically points in the environment which eventually will make up our map so we can think of the the landmarks as L L1 L2 2 L3 and so on and the measurements that come from camera or IMU or any other sensor create relationship between these unknowns the IMU might just tell us okay so between X1 and X2 your position should have incremented by this amount because your acceleration was this much over this period therefore velocity and therefore position and at the same time you can have you know multiple IM measurements so you connect this and from X1 you might just see also things from the camera features in the environment for instance a corner uh that you can track over time so this creates also some other relationships between our position and the landmark and you can see the same landmarks over different times so you can create this sort of relationships and if you keep adding all of these constraints then you have a strong idea of where you should have been and what was around you because all of these relationship are merging to an optimization problem where their unknowns are exactly X1 X2 X3 and L1 L2 L3 and in this way the algorithm will find the best solution that uh makes sense on all of this relationship together and then the last bit comes to solve the slam problem imagine that we we walked a very very long way so imagine I've made this chain very long and at some point we are at time x50 and in reality we should be in X2 because we have kept track of all of the measurement we have collected so far and so we have a map of them and we uh close the loop by thinking okay I'm exactly in the same place as X2 so I can create a relationship between X2 and x50 in this way all of the arrow we have accumulated up to x50 gets corrected the map gets warped to match reality and we have solved theam problem it's oversimplified if I say Islam problem is solved many many Professor will become angry yeah you have solved this some problem in this specific situation yes you did but not generally solved it no yeah so in practice is yet not solved because there are many problems related to uh noise in the sensors and and um inaccuracy in the calibration so there are techniques to make this uh solution more robust for instance so if you have a slum system working then you know where you are and what's around you and also robots do know where they are and what's around them so we can make do autonomous things Jane Street is a research-driven trading firm with offices in New York Hong Kong and London full of impossibly creative and clever people they like to think the next star individuals that may be working for them could be watching this computer file video right now that's why they're hoping you might peruse their website including this section for internships are these the sorts of jobs that might launch your career check out Jane Street there's a link in the video description and our thanks to them for supporting this episode of computer far