Machine Learning for Jiu Jitsu

Key Takeaways

in this video I'm going to share how I tried using machine learning to get better at Jiu Jitsu now Jiu Jitsu is a martial art that's been getting a lot of popularity over the last few years due to its applicability in real world combat as well as providing a lot of health benefits now I've been practicing it for over 10 years and it's been one of the greatest sources of enjoyment in my life along with doing machine learning so I decided to investigate if I could use my machine learning skills to get better at Jiu Jitsu now what did I do to understand that we first need to understand what pose estimation is and also what Jiu Jitsu is let's start with pose estimation pose estimation or Post tracking is a technique of detecting and tracking a person's body by detecting certain key points that match the joints of the body now you can use techniques like these to analyze movement in real time okay cool but what about Jujitsu Jiu Jitsu is a martial arts center around the idea of subduing an opponent through a series of pins joint blocks and chokes focuses on grappling and ground fighting techniques and its essential principle is that a weaker person can subdue a larger opponent by using leverage and technique okay so why use Post tracking for jiu jitsu one key aspect of getting better at Jiu Jitsu regards being able to execute perfect technique and perfect technique requires a lot of repetition and consistent appropriate feedback however it's often the case that people might not have access to an expert and a student can get the specific feedback it needs to get better at a certain technique it might be hard for them to learn and develop and it's in this gap of feedback that I think techniques like post-tracking could play a very important role the idea being that you can execute a movement in front of the camera and then use Post tracking to give you some sort of feedback about the level of correctness of the technique you're trying to learn okay but why did I do this to understand that we have to understand one thing about Jiu-Jitsu there's a common trend of people leaning towards their preferences in the beginning of their Journey which can hugely impact them in other areas of their Jiu Jitsu game if they get stuck executing the same strategy over and over in a way that's what happened to me I used to fight a lot as a guard player which has to do with a predominant culture that incentivizes starting your grappling bouts from your knees or sitting down because we built an entire sport around conceding bottom position now being a predominantly guard player negatively impacted my development in Jiu Jitsu because I lacked knowledge on how to take people to the ground and that ignited a fire in me to start studying and practicing Judo and wrestling techniques however I don't know any Judo or wrestling experts nor do I live close to a high level Judo or wrestling gym but I really wanted to learn foundational movements in Judo in wrestling that's where Post tracking comes into play I decided to use something called media pipe which is Google's open source project for facilitating the application of machine learning to live in streaming media to analyze my execution of a specific movement in Judo called the uchimata in essence here's what I did first I created some videos with the pose estimation overlay in order to get a sense for how well the model was capturing my movement then I created real-time plots of the XYZ coordinates of my feed while executing the movement then I created traces that represented the execution of a move at a given time these traces gave me sort of a signature for how my feet moves when I'm executing the movement and then I compare the traces with the traces of the elite player executing the same movement here we can start seeing some differences between the reference Trace representing the elite player movement and the trace is generated by me here we see while the elite player does a straight step into a turn generating an almost half circle with the speed I on the other hand have this curvature appearance to my initial step inside and do not create this half circle when throwing my leg into the air can be associated with flexibility and just overall lack of knowledge for how to execute movement properly now while the Elite player generates a Wide Circle when moving his leg up I create a shallow Circle almost like an ellipse I found these preliminary results to be quite nice because they indicated that despite the limitations of the comparison wonking Dodge differences between the signature shape of the movement's execution just by observing traces like these besides that I wanted to see if I could make comparisons regarding the speed with which the moves were performed to analyze that I visualized the real-time motion of the coordinates of the body in real time putting the plots of me and the expert side by side to see how off was my timing or to see what kind of differences I could Dodge the issue here is that we have two temporal sequences with different speeds and with different duration so we needed first to align these shoe sequences I wasn't sure what technique to use here so I talked to Aaron a buddy of mine that works at the Chevalier Neuroscience Institute in Lisbon and he eliminated an option for me Dynamic time warping Dynamic time warping is a technique used to measure the similarity between two temporal sequences that have different lengths the essential idea is that you have two time series they have some pattern you wish to analyze so you attempt to align them by applying a few rules that allow you to calculate the optimal match between the two sequences to use Dynamic time warping I did the following first I normalize the values to have them in the same range rather than in pixel resolution and then I use the python implementation of the DTW algorithm the outputs I get here are the distance which refers to the euclidean distance between the two temporal sequences and the path which is a mapping between the indexes of the two temporal sequences after being aligned now I can use the output in the path variable to create a plot with both sequences aligned I can say this plot specifically gave me a lot of insight but it did help emphasize the difference between the shapes of the true movements second thing I did with this was to try to use the euclidean distance from the DTW algorithm as a metric for the success of an attempt to execute the movement I wanted something that could represent being closer and closer to the correct execution of the move so here I'm showing the clips I generated from the videos where I try to execute a movement each of these traces represent a single execution of a move which now can be compared to a reference Trace that I created in a similar fashion so now I can Loop over these tracings representing my attempts to execute the move and see how they compare to the reference Trace across a few training sessions now the first thing I noticed here is this up and down pattern of the metric which I could only explain by the fact that some of the tracings that I obtained refer to the foot coming down rather than up while executing the movement however the cool thing about this is that the scores for the tracings even seem to improve a little bit or at least stay consistent at around 20 which isn't the measure for neocleading distance between the two temporal sequences now I can say I could conclusively interpret these numbers but I found quite interesting that an approach like this will be converted into a measurable metric that compares the quality of the move with respect to another if I can make the condition of the recording a bit more uniform they also improve how I collect the tracings for the execution of individual moves now right now my showings are a bit mixed on the one hand I think this approach is promising because these traces clearly capture some meaningful information about the execution of a move however there are many challenges to make this approach fully automated there are also challenges regarding how to provide proper feedback when correcting moves that are as complex as the move that I try to analyze in this video however by analyzing the traces and using techniques like the dynamic time warping to align true temporal sequences that have different durations I was able to manually gather some insight about how to improve the movement of by feet when executing this particular move so overall I think this approach was kind of cool although I would need to work a bit more on it in order to get something that's actually useful however I do think this is an interesting direction from martial arts in general to start exploring techniques like Post tracking boost estimation as good complementary tools to help with development of systems that can provide useful feedback to people trying to learn certain types of techniques if you like this video don't forget to like And subscribe and see you next time cheers