Motion capture workflow implementation with Stable diffusion

Key Takeaways

hey everyone if you click this video you probably want to see some cool new AI Tech what I got showing for you today is motion capture in stable Fusion let's Dive Right into that [Music] all right so here we are in the I am at Sasha rubitsky's YouTube channel and as you can see this guy has 27 subscribers so we'll see if we can help him with that a little bit hey and Sasha here now he's a PhD student at the nanyang Technological University of Singapore and he's doing some some creative stuff with computer graphics and video and what am I about to show you here is motion capture video so let's dive straight into that stable Fusion Mo cap visualizer so the project represents an approach to a stable diffusion implementation and a motion capture adding pipeline activity to be able to render human motion data in different styles plask which is a software is used for motion capturing from plain video footage and we're going to see that in a second and from there the data exported to Houdini which is a three animation software you can rig the models prefer the cleanup and editing within Houdini stable diffusion is integrated into the procedural dependency graph PDG cameras can get it freely created and Tangled from the mocha video footage and be used to rendering with disabled Fusion yeah okay let's go uh okay so here we have a video input of a person dancing and with plask and together with the plastic software this is a rigged SSR as a model [Music] and then this is imported into Houdini which is the 3D animation software preparing a 3D animation of this and that's where stable Fusion is going to start to start working so here we can see stable Fusion working together with this 3D model you can see here in this example the coherence is is superb even I mean the faces is not easy creating a coherent animation in the Forum right now what we're seeing here is amazing to be honest let's keep watching a little bit it's a very short video just two minutes so we've already looked at more than half of it you see this person dancing here I hear someone walking stairs I believe you can see the face is actually holding together even parts of you so you can see that pockets on the pants still there the hair is keeping its shape most of the time [Music] [Music] very cool let's look at the guy here again what was that let's look at this part here you can actually see the face and the hair the flowers in their hair it's keeping pretty good consistent I mean it's still it's still a frame by frame kind of thing snow is not perfect but compared to what we have out there right now this is really good I like it and the same here with this guy great coherence yeah so this was just a quick look at motion capture in stable Fusion if you like this please like And subscribe because that's gonna help me a lot and it'll show the algorithm that you like this content and show you more of it so whenever I find something new I'm gonna try and make a quick video out of it it doesn't have to be a 10 minute thing every time maybe just a short update on what's going on have a good one see ya