How Does AI Generated Songs Sound Like? [OpenAI Jukebox]

Key Takeaways

this is earth radio and now here's human music it's Christmas time we do know what that we oh the music that you are currently listening is a piece of an AI generated song which is composed by this AI called jukebox a generative model for music published by open AI on April 30th 2009 it has the ability to generate music based on artists and genres all the way from hip hop to country music the close part about this AI is that it has the ability to learn the artists style and singing any has never been seen done so well before also in the song I've just played we can clearly perceive the phonemes or even lyrics coming from this music piece and this is made possible with three separate models of vqv AE and sparse transformers to explain it briefly vqv AE is an autoencoder where it takes in the original audio [Music] and reconstruct it on the other side which each layer have a different level of details which are focused on when reconstructing the audio on the bottom level we have a very detailed decode of the compressed audio [Music] as it focuses on music and sound clarity in a very short time range which results in the high audio quality on the middle level the auto encoder focuses on the short term consistency of the song like a sentence or a chorus page contains a bit of loss in quality [Music] on the top level it decodes into the lowest quality but contains high-level semantics like the singing and the melodies [Music] this lets the song that's being generated learned long-term patterns and structures of the original song these three layers basically covers all the aspect that needs to be considered rendering a piece of music all three of these tracks are then composed together with the mono sparse Transformers by sampling each layer resulting in a more considerate music generation originally the lyrics pose a problem the AI does not know where they should be located in the piece of soft and generating them randomly is like feeding subtitles randomly into a movie but surprisingly jukebox overcame this problem by having an encoder also being trained to align lyrics against the songs the more defined the line is the more certainty AI is about the presence of the lyrics at that certain time frame this also extended into having a lyrics generating model within the jukebox AI as it generates lyrics and combine it with a song without lyrics which creates the same song with new lyrics which is really fascinating there are also two other functions along with generating unseen lyrics one is rear-end ich UNS where the AI generates completely different instrumentals or the music but with the same lyrics off a song that was chained upon and another one is called completions where it completes the song based on the first 12 seconds of the original and here are some examples of all three different types of generations within different genres [Music] from just a tiny bubble scarf when my twin ships with my hair assisted fire [Music] [Applause] when is my [Music] [Music] this moment [Music] [Music] the games even early Oh in the end as amazing as this AI is it is still far from perfect competing against human produced songs and then sometimes the bots will probably decide to give you some your rape I don't know what was dying in the background but that was pretty horrific also this jukebox AI requires a very long render time like average around 9 hours per 1 minute of audio which is crazy so be prepared if you want to play it around yourself well that's it for today thank you guys for watching all of me on Twitter if you haven't and I'll see you guys next time [Music] you [Music]