Deep Networks Favor Simple Data

📰 ArXiv cs.AI

Deep networks assign higher density to simpler out-of-distribution data than in-distribution test data, known as the OOD anomaly

advanced Published 2 Apr 2026

Action Steps

Investigate the OOD anomaly in various architectures and detectors to understand its prevalence
Analyze the estimated density of in-distribution and out-of-distribution data to identify patterns and trends
Develop new methods to mitigate the OOD anomaly and improve model performance on complex data
Evaluate the impact of the OOD anomaly on downstream tasks and applications

Who Needs to Know This

ML researchers and engineers benefit from understanding this phenomenon to improve model performance and robustness, while data scientists can apply this knowledge to better evaluate model results

Key Insight

💡 Deep networks' tendency to favor simple data can lead to poor performance on complex data and out-of-distribution samples

Key Takeaways

Deep networks assign higher density to simpler out-of-distribution data than in-distribution test data, known as the OOD anomaly

Full Article

Title: Deep Networks Favor Simple Data

Abstract:
arXiv:2604.00394v1 Announce Type: cross Abstract: Estimated density is often interpreted as indicating how typical a sample is under a model. Yet deep models trained on one dataset can assign \emph{higher} density to simpler out-of-distribution (OOD) data than to in-distribution test data. We refer to this behavior as the OOD anomaly. Prior work typically studies this phenomenon within a single architecture, detector, or benchmark, implicitly assuming certain canonical densities. We instead sepa

Read full paper → ← Back to Reads