What is machine learning, and how does it work?

Key Takeaways

have you heard about this concept called machine learning and you're trying to figure out exactly what that means or maybe you've checked out a few machine learning competitions on kaggle.com but you don't know how to get started if so I'm here to help my name is Kevin Markham and I'm a data science instructor in Washington DC this is my brand new video series about how to use the p learn library in Python for machine learning this is material that I love to teach and I can't wait to share with you in this series I'm going to cover s kit learn from the basics all the way through Advanced Techniques I'm not going to presume any familiarity with machine learning and in fact we're going to spend the next few videos talking about machine learning before we write any code the reason being there's really no point to using scikit learn if you don't know how to do proper machine learning you will need to have at least minimal experience with the Python programming language but I'll suggest some resources in the next video if you don't yet know python so with that let's get started in this video I'll be covering the following topics what is machine learning what are the two main categories of machine learning what are some examples of machine learning and how does machine learning work so what exactly is machine learning there's no Universal definition but at a high level I would Define machine learning as the semi-automated extraction of Knowledge from data let's break that down into three component parts first machine learning always starts with data and your goal is to extract knowledge or Insight from that data you have a question you're trying to answer and you hypothesize that your question might be answerable using the data second machine learning involves some amount of automation rather than trying to gather your insights from the data manually you're applying some process or algorithm to the data using a computer so that the computer can help to provide the Insight third machine learning is not a fully automated process as any practitioner can tell you machine learning requires you to make many smart decisions in order for the process to be successful we'll cover many of those decisions throughout this video series next let's talk about the two main categories of machine learning which are supervised learning and unsupervised learning supervised learning also known as predictive modeling is the process of making predictions using data for example if my data set is a series of email messages my supervised learning task might be to predict whether each email message is Spam or non-spam which is also known as ham this is supervised learning because there is a specific outcome we are trying to predict namely ham or spam in contrast unsupervised learning is the process of extracting structure from data or learning how to best represent data for example if my data set was the characteristics and purchasing behavior of Shoppers at a grocery store my unsupervised learning task might be to segment the Shoppers into groups or clusters that exhibit similar behaviors I might find that college students parents with young children and older adults have characteristic shopping behaviors that are similar within each group but dissimilar from the other two groups this is an unsupervised learning task because there is no right or wrong answer about how many clusters can be found in the data which people belong in which cluster or even how to describe each cluster let's do a quick quiz this is the kaggle website which is a popular platform for machine learning competitions this is their well-known Titanic competition and the goal is to predict which passengers survived the tragic sinking of the Titanic is this supervised or unsupervised learning this is supervised learning because your goal is to predict a specific outcome namely survival for each passenger in this video series I'm going to primarily Focus on supervised learning though I may cover unsupervised learning in later videos we've talked about what supervised learning is but we haven't yet talked about how it works so how does it actually work at a very high level here are the two main steps of supervised learning first you train a machine learning model using your existing labeled data labeled data is data which has been labeled with the outcome which in the case of the email example is whether each message is ham or spam this is called Model training because the model is learning the relationship between the attributes of the data and the outcome these attributes might include the message text the number of embedded links LS the length of the message and so on second you make predictions on new data for which you don't know the true outcome in other words when a new email message arrives you want your trained model to accurately predict whether the email is ham or spam without a human examining it to summarize these two steps you could say that the model is learning from past examples made up of inputs and outputs and then applying what it is learned to Future inputs in order to predict future outputs because you're making predictions on unseen data which is data that was not used to train the model it is often said that the primary goal of supervised learning is to build models that generalize in other words you want to build machine learning models that accurately predict the labels of your future emails rather than accurately predicting the labels of emails you've already received this simplified description of machine learning might raise some questions in your mind such as how do I choose which attributes of my data to include in the model how do I choose which model to use how do I optimize this model for best performance how do I ensure that I'm building a model that will generalize to unseen data can I estimate how well my model is likely to perform on unseen data these are excellent questions and hint at the complexity of doing effective machine learning all of these issues will be addressed later in the video series if you'd like a more in-depth introduction to machine learning there are two resources that I recommend that I've linked to below the video the First Resource is my favorite book on machine learning an introduction to statistical learning by Trevor Hasty and Rob tipani it's available as a free pdf download and section 2.1 introduces machine learning in a thorough yet accessible way the second resource I recommend is a 13-minute video from caltech's learning from data course which uses some excellent examples to compare supervised and unsupervised learning and also introduces another type of machine learning called reinforcement learning in the next video in the series I'll be covering the benefits and the drawbacks of psychic learn as well as my recommended way to set up python for machine learning in the meantime I'd love to hear from you in the YouTube comments if you have a question about machine learning or if you just have a cool example of machine learning that you like to share please do subscribe on YouTube if you'd like to hear the moment my next video comes out thanks for watching and I'll see you soon