Applied Deep Learning 2024 - Lecture 4 - Recurrent Neural Networks

About this lesson

In this lecture, we're looking at Recurrent Neural Networks (RNNs), and how they can be used to process sequences such as text, or audio. We also take a brief tour through LSTMs and the Connectionist Temporal Classification (CTC) Loss for training on unaligned data like an audio file and its transcription. Complete Playlist: https://www.youtube.com/playlist?list=PLNsFwZQ_pkE8tSQuU3jN71fmmGFFCi7Dc == Literature == 1. Deep Learning: http://www.deeplearningbook.org/ 2. Karpathy on RNNs: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ 3. Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions: https://arxiv.org/abs/1909.00590v1 4. Simple OCR: http://cs231n.github.io/assignments2019/assignment3/ 5. Understanding LSTMs (Blog): https://colah.github.io/posts/2015-08-Understanding-LSTMs/ 6. RNN-Walkthrough: https://github.com/gabrielloye/RNN-walkthrough 7. Neural End-To-End OMR: http://ismir2018.ircam.fr/doc/pdfs/33_Paper.pdf 8. Recognizing Speech Commands: https://towardsdatascience.com/recognizing-speech-commands-using-recurrent-neural-networks-with-attention-c2b2ba17c837 9. Understanding CTC: https://towardsdatascience.com/intuitively-understanding-connectionist-temporal-classification-3797e43a86c 10. Neural Machine Translation with Attention mechanism: https://arxiv.org/abs/1409.0473 11. https://towardsdatascience.com/a-comprehensive-guide-on-activation-functions-b45ed37a4fa5 12. https://machinelearningmastery.com/rectified-linear-activation-function-for-deep-learning-neural-networks/

Original Description