Generate Rap Lyrics - Fresh Machine Learning #4

Key Takeaways

You suck. Rhymes like mine can't be beat. That hurts my feelings. Oh, sorry. Do you want to play Pokémon Go instead? Sure. Yes. [Music] Hello world. It's Suriraj and I am a big fan of rap music. There's plenty of rap that degrades women, promotes violence, and glorifies the gangster lifestyle. But for all of its negatives, rap is also a medium for some incredible storytelling. It's a glimpse into real human suffering in a hopeless place. A cry for justice, a distillation of the human spirit of rebellion. And it's not just the content of rap that's compelling. It's the style. Let's not even think about the subject matter for a second. Rap gives you a certain lyrical freedom that nearly no other musical genre can. It's all about the rhymes. There's the perfect rhyme, words that end with the same sound. There's also the asinance rhyme, where only vowel sounds are shared, like And so because there's so much creative potential in the rap game, we might need to introduce a digital MC to the scene. Am I right? What if we could get a machine to understand rap lyrics or even write them? It's not like this stuff hasn't been tried before. It has. Several attempts have been made to try and understand lyrics using machine learning. But one attempt in particular got some really great results. A pair of researchers at Hong Kong University decided to download the lyrics to 52,000 rap songs, then trained a model so that given a novel song, the model could identify its rhyme scheme. So, you wait a big mac, your breath is whack, you need a tic tac, take the whole pack is an example of the aa rhyme scheme since all the ending words rhyme. Whereas, life is a dream, the future is now, I eat ice cream, then I take a bow is an example of abab since every other line rhymes. The model they used to train on the lyrics was called a hidden marov model. Let's remove the word hidden for a second and talk about how a plain old Marov model works. Let's say we want to predict the weather. And let's also say that the weather can only be one of three states, either sunny, cloudy, or rainy. For 100 days, we record the weather and record the transition between each day. Whenever we want to find the probability of the weather being a certain state after a given day, we can just tally the number of transitions of that type and divide the number of days by 100. That's how Marov models work. They help us predict the likelihood of a future state. Pretty useful, right? But what about a hidden Marov model? Well, suppose we can't directly observe the weather. So, we can't really calculate the transition probabilities. The model is hidden from us, but we can observe related phenomena like the number of people with umbrellas on a given day because Rihanna said so. So, using one of many techniques, we can still find ways to calculate the transition probabilities using these related variables. So hidden marov models are pretty cool and there are entire textbooks devoted to how they work. In fact, they can be used not just to classify lyrics but generate them as well. That same pair of researchers published a later paper. You know how during a cipher one MC challenges another with a verse? Then the opponent is supposed to spit back some sick rhymes. They trained an HMM to do this. The results were interesting, but let's just say their digital MC wasn't exactly the second coming of Tupac. Let's take a look at a fresh approach. A group of researchers published a paper just last month called Dope Learning, a computational approach to rap lyrics generation. Legendary. They used an algorithm called Rank SPM, which was partially powered by a deep neural network. And they fed it a data set of all the songs from the top 100 English-speaking rap artists. No idea how Lil Wayne got on that list. So, how does this algorithm work? Well, first they needed to extract features from their rap corpus to feed to their model. And they wanted features that represented three metrics. Rhyming, structural similarity, and semantic similarity. Of those three metrics, the semantic similarity was the one that required the use of a deep recurrent neural network. The neural net did what it does best. It found vector representations of words, lines, and groups of lines. Once the features were calculated, they were input into the rank SVM model. Rank SVM is a support vector machine, by the way, which is just a type of linear classifier. And the SVM eventually learned to predict the next line once it was trained on the input features. They also wanted to find a way to quantify how good their algorithm was compared to human MC's. And so they calculated something called rhyme density, which is the average length of the longest rhyme per word. Using rhyme density as a metric, they found that the algorithms generated lyrics had a 21% higher density than the most rhyme dense human artist on their list named Inspector Deck. Wait, who? Let's write our own rap lyrics generator using a hidden markup model in Python. We only need two dependencies here. Random, which helps generate random numbers, and RE, which helps deal with text formatting. Our highest level method is called test markup. In it, we'll initialize an empty array called raplive. Then add all of our lyrics to the rap library and return the rap generated from the rap library using the stop word that was input by the user. The generated rap will always start with the stop word. Let's take a look at the add to live function. It opens the lyrics file, then constructs an hmm. It iterates through every word and checks its record to see if it's a new word or sequence. If a word or sequence has appeared before, it won't re-record it. Then it changes each count to a percentage value or a transition probability. So once we've constructed our model, we can run the make wrap function. It'll take both the set of lyrics and the start word as the parameters. It'll continuously generate words via the marov next function for up to 50 words. The marov next function either returns a random word if the word is novel or finds a word from the model probabilistically. Let's test this out and see what it generates. I'll start off my rap with homie. Homie Grows Punani likely I'm toteen inspired enough. Basically boy coming period. Damn. So dope. Check out the links down below for more info and please subscribe for more ML videos. For now, I've got to go fix a malformed request. So thanks for watching.