Generalized Language Models
📰 Lilian Weng's Blog
Generalized language models achieve state-of-the-art results on various language tasks through contextualized word vectors and unsupervised pre-training
Action Steps
- Learn about word embeddings and contextualized word vectors
- Explore large unsupervised pre-trained language models such as ULMFiT, GPT-2, and ALBERT
- Apply these models to various language tasks to achieve state-of-the-art results
Who Needs to Know This
NLP researchers and AI engineers can benefit from understanding generalized language models to improve their language tasks, while product managers can leverage these advancements to develop more accurate language-based products
Key Insight
💡 Contextualized word vectors and unsupervised pre-training are key to achieving state-of-the-art results in language tasks
Share This
🤖 Generalized language models are achieving SOTA results on language tasks! #NLP #LLMs
Key Takeaways
Generalized language models achieve state-of-the-art results on various language tasks through contextualized word vectors and unsupervised pre-training
Full Article
<!-- As a follow up of word embedding post, we will discuss the models on learning contextualized word vectors, as well as the new trend in large unsupervised pre-trained language models which have achieved amazing SOTA results on a variety of language tasks. --> <p><span class="update">[Updated on 2019-02-14: add <a href="#ulmfit">ULMFiT</a> and <a href="#gpt-2">GPT-2</a>.]</span><br/> <span class="update">[Updated on 2020-02-29: add <a href="#albert">ALBERT</a>.]</span><br/> <span class="updat
DeepCamp AI