Machine learning won't solve natural language understanding
📰 Hacker News · andreyk
Machine learning may not be the solution to natural language understanding due to its limitations in capturing the complexities of human language, and a more nuanced approach is needed.
Action Steps
- Read the article 'Machine Learning Won't Solve Natural Language Understanding' to understand the limitations of empirical methods in NLP
- Evaluate the role of large language models in NLP and their potential drawbacks
- Consider alternative approaches to NLP that incorporate linguistic and cognitive insights, such as symbolic and logical methods
- Research the history of NLP and the development of empirical methods to gain a deeper understanding of the field
- Apply critical thinking to the use of machine learning in NLP and consider the potential consequences of relying solely on data-driven methods
Who Needs to Know This
NLP researchers and engineers can benefit from understanding the limitations of machine learning in achieving true natural language understanding, and consider alternative approaches that incorporate linguistic and cognitive insights.
Key Insight
💡 The reliance on machine learning and large language models may not be sufficient to achieve true natural language understanding, and a more nuanced approach is needed.
Share This
🚨 Machine learning may not be the solution to natural language understanding 🚨 Consider alternative approaches that incorporate linguistic and cognitive insights #NLP #AI
Key Takeaways
Machine learning may not be the solution to natural language understanding due to its limitations in capturing the complexities of human language, and a more nuanced approach is needed.
Full Article
Title: Machine Learning Won't Solve Natural Language Understanding
URL Source: https://thegradient.pub/machine-learning-wont-solve-the-natural-language-understanding-challenge/
Published Time: 2021-08-07T17:36:55.000Z
Markdown Content:
[](https://thegradient.pub/)
* [About](https://thegradient.pub/about/)
* [Articles](https://thegradient.pub/)
* [Newsletter](https://thegradientpub.substack.com/s/the-update)
* [Podcast](https://thegradientpub.substack.com/s/podcast)
* [Subscribe](https://thegradientpub.substack.com/)
* [Write](https://thegradient.pub/write/)
* [Support](https://thegradient.pub/support/)
# Machine Learning Won't Solve Natural Language Understanding
07.Aug.2021 . 15 min read
_This piece was a runner-up for the inaugural [Gradient Prize](https://thegradient.pub/2021-gradient-prize/)._
#### **The Empirical and Data-Driven Revolution**
In the early 1990s a statistical revolution overtook artificial intelligence (AI) by a storm – a revolution that culminated by the 2000’s in the triumphant return of neural networks with their modern-day deep learning (DL) reincarnation. This empiricist turn engulfed all subfields of AI although the most controversial employment of this technology has been in natural language processing (NLP) – a subfield of AI that has proven to be a lot more difficult than any of the AI pioneers had imagined. The widespread use of data-driven empirical methods in NLP has the following genesis: the failure of the symbolic and logical methods to produce scalable NLP systems after three decades of supremacy led to the rise of what are called empirical methods in NLP (EMNLP) – a phrase that I use here to collectively refer to data-driven, corpus-based, statistical and machine learning (ML) methods.
The motivation behind this shift to empiricism was quite simple: until we gain some insights in how language works and how language is related to our knowledge of the world we talk about in ordinary spoken language, empirical and data-driven methods might be useful in building some practical text processing applications. As Kenneth Church, one of the pioneers of EMNLP explains, the advocates of the data-driven and statistical approaches to NLP were interested in solving simple language tasks – the motivation was never to suggest that this is how language works, but that “it is better to do something simple than nothing at all”. The cry of the day was: “let’s go pick up some low-hanging fruit”. In a must-read essay appropriately entitled “A Pendulum Swung Too Far”, however, Church (2007) argues that the motivation of this shift have been grossly misunderstood. As McShane (2017) also notes, subsequent generations misunderstood this empirical trend that was motivated by finding practical solutions to simple tasks by assuming that this [Probably Approximately Correct](https://en.wikipedia.org/wiki/Probably_approximately_correct_learning) (PAC) paradigm will scale into full natural language understanding (NLU). As she puts it: “How these beliefs attained quasi-axiomatic status among the NLP community is a fascinating question, answered in part by one of Church’s observations: that recent and current generations of NLPers have received an insufficiently broad education in linguistics and the history of NLP and, therefore, lack the impetus to even scratch that surface.”
This _misguided_ trend has resulted, in our opinion, in an unfortunate state of affairs: an insistence on building NLP systems using ‘large language models’ (LLM) that require massive computing power in a futile attempt at trying to _approximate_ the infinite object we call natural language by trying to memorize massive amounts of data. In our opinion this pseudo-scientific meth
URL Source: https://thegradient.pub/machine-learning-wont-solve-the-natural-language-understanding-challenge/
Published Time: 2021-08-07T17:36:55.000Z
Markdown Content:
[](https://thegradient.pub/)
* [About](https://thegradient.pub/about/)
* [Articles](https://thegradient.pub/)
* [Newsletter](https://thegradientpub.substack.com/s/the-update)
* [Podcast](https://thegradientpub.substack.com/s/podcast)
* [Subscribe](https://thegradientpub.substack.com/)
* [Write](https://thegradient.pub/write/)
* [Support](https://thegradient.pub/support/)
# Machine Learning Won't Solve Natural Language Understanding
07.Aug.2021 . 15 min read
_This piece was a runner-up for the inaugural [Gradient Prize](https://thegradient.pub/2021-gradient-prize/)._
#### **The Empirical and Data-Driven Revolution**
In the early 1990s a statistical revolution overtook artificial intelligence (AI) by a storm – a revolution that culminated by the 2000’s in the triumphant return of neural networks with their modern-day deep learning (DL) reincarnation. This empiricist turn engulfed all subfields of AI although the most controversial employment of this technology has been in natural language processing (NLP) – a subfield of AI that has proven to be a lot more difficult than any of the AI pioneers had imagined. The widespread use of data-driven empirical methods in NLP has the following genesis: the failure of the symbolic and logical methods to produce scalable NLP systems after three decades of supremacy led to the rise of what are called empirical methods in NLP (EMNLP) – a phrase that I use here to collectively refer to data-driven, corpus-based, statistical and machine learning (ML) methods.
The motivation behind this shift to empiricism was quite simple: until we gain some insights in how language works and how language is related to our knowledge of the world we talk about in ordinary spoken language, empirical and data-driven methods might be useful in building some practical text processing applications. As Kenneth Church, one of the pioneers of EMNLP explains, the advocates of the data-driven and statistical approaches to NLP were interested in solving simple language tasks – the motivation was never to suggest that this is how language works, but that “it is better to do something simple than nothing at all”. The cry of the day was: “let’s go pick up some low-hanging fruit”. In a must-read essay appropriately entitled “A Pendulum Swung Too Far”, however, Church (2007) argues that the motivation of this shift have been grossly misunderstood. As McShane (2017) also notes, subsequent generations misunderstood this empirical trend that was motivated by finding practical solutions to simple tasks by assuming that this [Probably Approximately Correct](https://en.wikipedia.org/wiki/Probably_approximately_correct_learning) (PAC) paradigm will scale into full natural language understanding (NLU). As she puts it: “How these beliefs attained quasi-axiomatic status among the NLP community is a fascinating question, answered in part by one of Church’s observations: that recent and current generations of NLPers have received an insufficiently broad education in linguistics and the history of NLP and, therefore, lack the impetus to even scratch that surface.”
This _misguided_ trend has resulted, in our opinion, in an unfortunate state of affairs: an insistence on building NLP systems using ‘large language models’ (LLM) that require massive computing power in a futile attempt at trying to _approximate_ the infinite object we call natural language by trying to memorize massive amounts of data. In our opinion this pseudo-scientific meth
DeepCamp AI