ELMo vs BERT

Key Takeaways

Elmo versus Bert Elmo is a bi-directional lstm network while Bert is a stack of Transformer encoders Elmo is trained to be a language model while Bert is trained on two tasks masked language modeling and next sentence prediction Elmo is slow to train as it relies on the back propagation through time to learn as it consists of lstm cells Transformers are quicker to train as they make use of parallelization Elmo may not understand true context as it learns a forward and backward context and then concatenates them Bert on the other hand is deeply bi-directional as it uses attention to learn both forward and backward context simultaneously check the channel for more NLP content