Visually Indicated Sounds | Two Minute Papers #79

Key Takeaways

dear fellow Scholars this is 2minute papers with Caro this name is not getting any easier is it it used to be car which was hard enough and now this anyway let's get started this technique simulates how different objects in a video sound when struck we have showcased some marvelous previous techniques that were mostly limited to wooden and plastic materials needless to say there are links to these episodes in the video description box a convolutional neural network takes care of understanding what is seen in the video this technique is known to be particularly suited to processing image and video content and it works by looking at the Silent Video directly and trying to understand what is going on just like a human would we train these networks with input and output pairs the input is a video of us beating the hell out of some object with a drumstick the choice of research and the output is the sound this object emits however the output sound is something that changes in time it is a sequence therefore it cannot be handled by a simple classical neural network it is learned by a recurrent neural network that can take care of learning such sequences if you haven't heard these terms before no worries we have previous episodes on all of them in the video description box make sure to check them out this piece of work is a nice Showcase of combining two quite powerful techniques the convolutional neural network tries to understand what happens in the input video and the recurrent neural network seals the deal by learning and guessing the correct sound that objects shown in the video would emit when struck the synthesized outputs were compared to real word results both mathematically and by asking humans to try to tell from the two samples which one the real deal is these people were fooled by the algorithm around 40% of the time which I find to be a really amazing result considering two things first the Baseline is not 50% but 0% because people don't pick choices at random we cannot reasonably expect a synthesized sound to fool humans at any time like nice little neural networks we've been trained to recognize these sounds all our lives after all and second this is one of the first papers from a machine learning angle on sound synthesis before reading the paper I expected at most 10 or 20% if that the tital wave of machine learning runs through a number of different scientific Fields will deep learning techniques establish Supremacy in these areas hard to say yet but what we know for sure is that great strides are made literally every week there are so many works out there sometimes I don't even know where to start Good Times indeed before we go some delightful news for you fellow Scholars the scholarly 2minute paper store is now open there are two different kinds of men's t-shirts available and a nice Sleek design version that we made for the fellow scholar ladies out there we also have the scholarly mug to get your day started in the most scientific way possible we have tested the quality of these products and we're really happy with what we got if you ordered anything please provide us feedback on how you like the quality of the delivery and the products themselves if you can send us an image of yourself wearing or using any of these would love to have a look just leave them in the comment section or tweet at us if you don't like what you get within 30 days you can exchange it or get your product cost refunded thanks for for watching and for your generous support and I'll see you next time