Google Tango - Computerphile

Key Takeaways

AI is becoming quite prevalent these days. I mean, we see mobile apps telling you to point your phone at your can of Coke at a magazine or something like that. All these solutions are basically using computer vision and your phone camera. All they're really doing is recognizing a uh picture. So, ideally a 2D image, but they can extrapolate from that. Once they recognize the image, they can get its position in space and its orientation and then just tie virtual content to it. Right now, the problem with that is it looks pretty impressive, but they don't actually these systems understand the environment around them. It doesn't know that this is a card. It doesn't know that I'm holding it in my hand or anything like that. So, if I put my hand in front of it, if it can still see part of the card or anything like that, it will keep drawing the content above my hand. The illusion is very easy to break with these things. And that's presumably the same as the depth problem that we've seen in computer graphics before. Exactly. Yes. And also it limits the kinds of things you can do. If all it really understands is, okay, I've seen this thing. I know my position in space, it quickly limits the the things you can do with physics or moving about, you're tied at looking at that marker and maybe moving a little bit off it with optical flow. As we discuss another time thing is that there are already solutions to try and deal with this. For example, there is the uh Google Tango project which what they did was basically they looked at the connect and says okay what does the connect do? It has a depth camera. It reads the environment around it. Well, it was designed to re uh to look at humans and register what they're doing. But it turned out was pretty good at 3D scanning stuff. So they said, "All right, let's strap one of those to a back of the tablet." And they came up with this. This is the original Google Tango development device. It has on the back of it a laser sensor, depth cameras, and what have you. I believe it's built as the spatially aware tablet. So, it has a very good understanding of where it is, its orientation, what it's looking at, and the motion. So, uh what this does is it creates a point cloud in pretty much well real time of the environment around it. Unlike the previous AI, that is where we start reading the reaching the point where with an understanding of the environment and understanding of the motion of the tablet, we can start tying virtual content to things we couldn't before. So, I can put a mesh on this floor and it will stay there. Or I can put a piece of sound on the corner of this cabinet and whenever I come close, I'll get 3D sound. Uh, but this is now an old device. I believe it's about 3 4 years old and it was only a developer device. There is actually the first commercial device, the Lenovo Fab 2 Pro. I believe it is now looking like a proper commercial device. Horrible case non-withstanding. And just like the developer device, it does have an array of sensors over here. So, what can this do? Let's have a look. Here's probably my favorite example, although it's not very exciting. We have this plane that is reading the environment pretty well and telling me what plane I'm looking at. Is that the green little dot? So the green little dot is basically the edge detection and this plane over here is aligning itself to the real world. So in this case the table there is a a white plane uh a surface basically that is aligning itself to the real world. I can see that the the edges aligned up with the tabledeed. And if you put it near the wall what happens? And as we go over the floor, see it's it's turning around and aligning itself to any surface. Okay. Yeah. Okay. Yeah. So, that's all fair and well, but what does this actually mean? Well, it means that I can go and understand the real world. So, there we go. A Let's measure the size of the door. So, you're drawing lines on that. Yeah. I'm basically using a measuring tape. And there we have it. So there are centimeter markings on there. So you can see how with an understanding of the space around it, it can actually start measuring as well. So we could go around this room taking uh measurements and end up with an architectural floor plan. I mean that is one of the uh use cases. Um so they'll sell this to estate agents, real estate. Uh yeah, it has already they we've seen them using it to create uh basic floor plans and 3D models that they'll just chuck on a website. With a better understanding of what is what the environment you're in is populated with and how the camera is moving about, you can start doing slightly more interesting things such as populating the environment around you with your own virtual content or whatever you might want to see. So this is the uh augmented camera application and what it is is a normal camera application but again with that whole understanding of the environment thing we can start putting virtual content inside it. So in this case we can put a cat where would you normally have a cat? Let's see. Can hear it. Oh. Oh there it is. There's a lot of reflection. It's on the floor there. And there we have a cat that is standing on the floor playing on the floor with seeing as the device has an understanding of what a floor is. And I guess we can also use a laser pointer. Or will the cat follow that? Is that It should when it sees it. Oh, there we are. It's definitely not acting like any cat eye now to be honest. But put down a fish. So that's fine. It's recognized the floor and we have a cat playing on the floor. But maybe we can get it up onto another surface that is recognized like say the table. and it gets the difference between the floor down there and the top of the table. Well, and it's jumped off the table. With a traditional AR application that was just using a marker to do that, it wouldn't have an understanding of that. It would just be drawing everything in relation to that marker. Whether we had it on the floor, on the table, it wouldn't matter. This now is we're actually having well, I don't want to say intelligent content in this case, uh, populating our space, our real space, the objects that we use every day. You can see how this could end up being something a bit more exciting. And in fact, we do have something to show for that. One of these. So, this is the Microsoft Hollow Lens. Same principle, but strapped to your face. So, as you can see here, has again, it's pretty much a connect all across the top over here. Right. This is a proper heads-up display un say some earlier endeavors like the uh Google Glass which is simply just a screen off to the side which didn't uh cover your normal vision with augmented content. But this is dare I say the real deal. This is the one. Uh I don't want to make that claim, but I have to say from a lot of the stuff I've tried, this is probably the one that gets it closest. In fact, we can show you a bit more about