Underspecification Presents Challenges for Credibility in Modern Machine Learning (Paper Explained)

#ai #research #machinelearning Deep Learning models are often overparameterized and have many degrees of freedom, which leads to many local minima that all perform equally well on the test set. But it turns out that even though they all generalize in-distribution, the performance of these models can be drastically different when tested out-of-distribution. Notably, in many cases, a good model can actually be found among all these candidates, but it seems impossible to select it. This paper describes this problem, which it calls underspecification, and gives several theoretical and practical examples. OUTLINE: 0:00 - Into & Overview 2:00 - Underspecification of ML Pipelines 11:15 - Stress Tests 12:40 - Epidemiological Example 20:45 - Theoretical Model 26:55 - Example from Medical Genomics 34:00 - ImageNet-C Example 36:50 - BERT Models 56:55 - Conclusion & Comments Paper: https://arxiv.org/abs/2011.03395 Abstract: ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains. This ambiguity can lead to instability and poor model behavior in practice, and is a distinct failure mode from previously identified issues arising from structural mismatch between training and deployment domains. We show that this problem appears in a wide variety of practical ML pipelines, using examples from computer vision, medical imaging, natural language processing, clinical risk prediction based on electronic health records, and medical genomics. Our