[MINI] Long Short Term Memory

Data Skeptic · Intermediate ·🧬 Deep Learning ·8y ago

Key Takeaways

The video discusses Long Short Term Memory (LSTM) units in Recurrent Neural Networks (RNNs), explaining how they store information for longer periods of time without using activation functions within their recurrent components. This allows the network to remember values for both long and short time periods, mitigating the vanishing gradient problem during backpropagation through time.

Original Description

Thanks to our sponsor brilliant.org/dataskeptics A Long Short Term Memory (LSTM) is a neural unit, often used in Recurrent Neural Network (RNN) which attempts to provide the network the capacity to store information for longer periods of time. An LSTM unit remembers values for either long or short time periods. The key to this ability is that it uses no activation function within its recurrent components. Thus, the stored value is not iteratively modified and the gradient does not tend to vanish when trained with backpropagation through time.
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Data Skeptic · Data Skeptic · 0 of 60

← Previous Next →
1 Data Skeptic book giveaway contest winner selection
Data Skeptic book giveaway contest winner selection
Data Skeptic
 2 OpenHouse - Front end and API overview
OpenHouse - Front end and API overview
Data Skeptic
 3 OpenHouse Crawling with AWS Lambda
OpenHouse Crawling with AWS Lambda
Data Skeptic
 4 [MINI] Logistic Regression on Audio Data
[MINI] Logistic Regression on Audio Data
Data Skeptic
 5 Data Provenance and Reproducibility with Pachyderm
Data Provenance and Reproducibility with Pachyderm
Data Skeptic
 6 [MINI] Primer on Deep Learning
[MINI] Primer on Deep Learning
Data Skeptic
 7 Big Data Tools and Trends
Big Data Tools and Trends
Data Skeptic
 8 [MINI] Automated Feature Engineering
[MINI] Automated Feature Engineering
Data Skeptic
 9 The Data Refuge Project
The Data Refuge Project
Data Skeptic
 10 [MINI] The Perceptron
[MINI] The Perceptron
Data Skeptic
 11 [MINI] Feed Forward Neural Networks
[MINI] Feed Forward Neural Networks
Data Skeptic
 12 Data Science at Patreon
Data Science at Patreon
Data Skeptic
 13 [MINI] Backpropagation
[MINI] Backpropagation
Data Skeptic
 14 [MINI] GPU CPU
[MINI] GPU CPU
Data Skeptic
 15 OpenHouse
OpenHouse
Data Skeptic
 16 [MINI] Generative Adversarial Networks
[MINI] Generative Adversarial Networks
Data Skeptic
 17 [MINI] AdaBoost
[MINI] AdaBoost
Data Skeptic
 18 [MINI] The Bootstrap
[MINI] The Bootstrap
Data Skeptic
 19 [MINI] Dropout
[MINI] Dropout
Data Skeptic
 20 [MINI] Gini Coefficients
[MINI] Gini Coefficients
Data Skeptic
 21 [MINI] Random Forest
[MINI] Random Forest
Data Skeptic
 22 [MINI] Heteroskedasticity
[MINI] Heteroskedasticity
Data Skeptic
 23 [MINI] ANOVA
[MINI] ANOVA
Data Skeptic
 24 Urban Congestion
Urban Congestion
Data Skeptic
 25 [MINI] The CAP Theorem
[MINI] The CAP Theorem
Data Skeptic
 26 Unstructured Data for Finance
Unstructured Data for Finance
Data Skeptic
 27 Detecting Terrorists with Facial Recognition?
Detecting Terrorists with Facial Recognition?
Data Skeptic
 28 Predictive Models on Random Data
Predictive Models on Random Data
Data Skeptic
 29 [MINI] Entropy
[MINI] Entropy
Data Skeptic
 30 [MINI] F1 Score
[MINI] F1 Score
Data Skeptic
 31 Causal Impact
Causal Impact
Data Skeptic
 32 Machine Learning on Images with Noisy Human-centric Labels
Machine Learning on Images with Noisy Human-centric Labels
Data Skeptic
 33 The Library Problem
The Library Problem
Data Skeptic
 34 Stealing Models from the Cloud
Stealing Models from the Cloud
Data Skeptic
 35 Data Science at eHarmony
Data Science at eHarmony
Data Skeptic
 36 Multiple Comparisons and Conversion Optimization
Multiple Comparisons and Conversion Optimization
Data Skeptic
 37 Election Predictions
Election Predictions
Data Skeptic
 38 [MINI] Calculating Feature Importance
[MINI] Calculating Feature Importance
Data Skeptic
 39 MS Connect Conference
MS Connect Conference
Data Skeptic
 40 Music21
Music21
Data Skeptic
 41 The Police Data and the Data Driven Justice Initiatives
The Police Data and the Data Driven Justice Initiatives
Data Skeptic
 42 Studying Competition and Gender Through Chess
Studying Competition and Gender Through Chess
Data Skeptic
 43 [MINI] Goodhart's Law
[MINI] Goodhart's Law
Data Skeptic
 44 Trusting Machine Learning Models with LIME
Trusting Machine Learning Models with LIME
Data Skeptic
 45 [MINI] Leakage
[MINI] Leakage
Data Skeptic
 46 Predictive Policing
Predictive Policing
Data Skeptic
 47 Mutli-Agent Diverse Generative Adversarial Networks
Mutli-Agent Diverse Generative Adversarial Networks
Data Skeptic
 48 [MINI] Convolutional Neural Networks
[MINI] Convolutional Neural Networks
Data Skeptic
 49 Unsupervised Depth Perception
Unsupervised Depth Perception
Data Skeptic
 50 [MINI] Max-pooling
[MINI] Max-pooling
Data Skeptic
 51 MS Build 2017
MS Build 2017
Data Skeptic
 52 Activation Functions
Activation Functions
Data Skeptic
 53 Doctor AI
Doctor AI
Data Skeptic
 54 [MINI] The Vanishing Gradient
[MINI] The Vanishing Gradient
Data Skeptic
 55 CosmosDB
CosmosDB
Data Skeptic
 56 Estimating Sheep Pain with Facial Recognition
Estimating Sheep Pain with Facial Recognition
Data Skeptic
 57 [MINI] Conditional Independence
[MINI] Conditional Independence
Data Skeptic
 58 MINI: Bayesian Belief Networks
MINI: Bayesian Belief Networks
Data Skeptic
 59 Project Common Voice
Project Common Voice
Data Skeptic
 60 [MINI] Recurrent Neural Networks
[MINI] Recurrent Neural Networks
Data Skeptic

This video teaches the basics of Long Short Term Memory (LSTM) units in Recurrent Neural Networks (RNNs), including their ability to store information for longer periods of time and mitigate vanishing gradients. Understanding LSTMs is crucial for building effective deep learning models, particularly those involving sequential data. By the end of this video, viewers will be able to implement LSTMs in their own RNNs and appreciate the importance of careful neural network design.

Key Takeaways
  1. Understand the basics of Recurrent Neural Networks (RNNs)
  2. Learn how LSTMs store information for longer periods of time
  3. Implement LSTMs in an RNN using a deep learning framework
  4. Train the model using backpropagation through time
  5. Evaluate the model's performance on a sequential data task
💡 The key to LSTMs' ability to store information for longer periods of time is that they use no activation function within their recurrent components, allowing the stored value to remain unmodified and the gradient to remain intact during backpropagation.

Related Reads

📰
Want to get started with deep learning
Get started with deep learning by leveraging resources like Andrew Karpathy's playlist and frameworks such as TensorFlow or PyTorch
Reddit r/deeplearning
 📰
Building a Deepfake Detector From Scratch — What Nobody Tells You
Learn to build a deepfake detector from scratch and understand the challenges involved in detecting AI-generated fake media
Medium · Deep Learning
 📰
Unfolding the Meandering Path: High-Dimensional Invariance and the Flat 2D Plane of Neural…
Learn about high-dimensional invariance and its relation to the flat 2D plane of neural networks, and how to apply these concepts to improve model performance
Medium · Deep Learning
 📰
Implementing Neural Style Transfer from Scratch: The Project That Started It All
Learn to implement Neural Style Transfer from scratch and understand its significance in deep learning
Medium · Deep Learning
Up next
Image Classification with ml5.js
The Coding Train
Watch →