Generative vs Discriminative Models - Explained

What's the real difference between a generative classifier like Naive Bayes and a discriminative one like Logistic Regression? It comes down to a single choice: do you model the whole world, or do you just draw the border? A generative model learns the joint probability P(x, y), the full story of what each class looks like, so it can even generate brand-new data. A discriminative model learns the conditional P(y|x) directly, modeling only where the boundary sits. We build the contrast from the ground up: joint versus conditional, fitting a density per class versus shaping a single boundary, and the famous tradeoff from Ng & Jordan where the two learning curves cross. The generative model learns fast from little data but plateaus at a higher error; the discriminative model needs more data but settles lower. By the end you'll see why Naive Bayes and Logistic Regression are secretly a matched pair, reaching the same logistic form two completely different ways. *Related Videos* ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Naive Bayes - Explained: https://youtu.be/Kstjz91Ks4U Logistic Regression - Explained: https://youtu.be/FiDWEnve9go Bayes' Theorem - Explained: https://youtu.be/dd5KU9VzwWo Multivariate Normal (Gaussian) Distribution Explained: https://youtu.be/UVvuwv-ne1I Maximum Likelihood - Explained: https://youtu.be/Pk7kDdWuG1Q Support Vector Machines (SVMs) - Explained: https://youtu.be/K1EcCjDD_q4 Frequentist vs Bayesian Thinking: https://youtu.be/zIyMz5YUdcY Bayesian Linear Regression - Explained: https://youtu.be/lzXltSCF4A8 *Contents* ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 00:00 - Two Ways to Classify 00:40 - Joint vs Conditional: P(x,y) vs P(y|x) 01:42 - The Generative Side (Naive Bayes) 02:31 - The Discriminative Side (Logistic Regression) 03:17 - The Tradeoff: Learning Curves Cross 04:02 - Same Pair, Two Philosophies *Follow Me* ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 🐦 X: @datamlistic https://x.com/datamlistic 📸 Instagram: @datamlistic https://www.instagram.com/datamlistic 📱 TikTok: @datamlistic