1.2 What is Machine Learning (L01: What is Machine Learning)

Key Takeaways

okay now after we covered the logistics of this course let's get to the more interesting parts and talk about machine learning and learn what machine learning is so what is machine learning for that let's look at some i would say interesting quotes so one quote is by john hennessy a former president of stanford and he basically said machine learning is the hot new thing so it sounds like machining is something cool but of course it doesn't really tell us what machine ring is so let's see what bill gates has to say about that so bill gates said a breakthrough in machine learning would be worth 10 microsoft's so based on that statement it sounds like machine learning is something pretty important so what is it really i really like this quote by rana l kalubi and uh yeah she's a co-founder at effectiva and ceo of effectiva which is a company focusing on recognizing emotions from images and videos i agree i mean it can be a little bit of a controversial topic like to recognize emotions from face images and things like that but uh yeah if you're interested search for some of her interviews and she makes a pretty good and compelling point why this is actually sometimes really useful so anyways what she said is learning is a subcategory within the field of computer science which allows you to implement artificial intelligence so what this basically says is um machine learning helps us to create artificial intelligence so actually then says so it's kind of a mechanism to get you artificial intelligence so machine learning and um machine learning and artificial intelligence they are closely related and in fact machine learning was originally created with a desire to create artificial intelligence so artificial intelligence is much older it's a older field and in order to implement artificial intelligence researchers developed a couple of approaches and one of the promising approaches was using machine learning so here based on this slide we kind of learned as a relationship between machine learning and artificial intelligence today so back then i would draw the relationship like this where we have artificial intelligence and machine learning as a subfield nowadays i would say it's rather like this where we have um the big field of ai and machine learning is uh intersecting with ai because there are many machine learning approaches applications that i would say are not necessarily ai and we will see some of them also in this class or in this course so nowadays i think the fields are of course very closely related but not um i would say not all of machine learning is ai but a large a very large portion of machine learning is actually ai in any case let's get a little bit uh more precise now what machine learning actually is so in order to explain machine learning i uh really like this quote this is actually a classic that you can find also in most online material or other materials about machine learning and i guess it's so often used because it's actually really concise and really captures the idea of machine learning so what arthur l samuel said is that machine learning is the field of study that gives computers the ability to learn without being explicitly programmed so what we have here the key words are the ability to learn without being explicitly programmed so there's also an aspect mentioning programmed and here with machine learning we want to achieve something without having something be programmed explicitly so what does that mean here so in order to understand the previous quote i have a flow chart or a diagram here to illustrate the traditional programming paradigm so with traditional i mean just a yeah i would say regular software development cycle so that's for the sake of the example to make this a little bit more i would say um yeah to visualize this with a concrete example let's assume we want our task is we want to implement an email spam filter let's say our task is uh implementing an email spam filter and we have an email program and we want to now develop a let's say a separate program that filters our spam and um in order to illustrate that let's walk through it step by step so if we have an email spam filter we have some inputs so in this case our inputs would be the emails that we want to filter or recognize a spam and not spam and the program that we want to develop this is our spam filter and the inputs in order to develop this pamphlet and the traditional paradigm we look at the email so we as the programmer we take a look at these emails and see okay what makes them different or what makes spam email different from non-spam email and how could we maybe implement some rules to distinguish between them and that takes actually a lot of work so we have to read ideally a large number of emails do a lot of thinking and comparisons to find out what makes them different and what do spam and non-spam email have in common so we don't accidentally classify something as spam and so forth but let's say we looked at 10 000 emails came up with some good um rules we can then write our program our spam filter which is basically a bunch of rules like in traditional programming a bunch of if else statements basically um and this is our spam folder that goes then to our computer so well it's implemented on our computer the computer itself executes this and then classifies the emails as oops classify spam or not spam so our outputs would be the labels spam and not spam so that's the traditional paradigm so how on how does machine learning now come into play so how is machine learning related to that traditional paradigm so machine learning helps us to do this more conveniently or automatically so the computer now has the role of developing the spam filter so before we as the programmer we developed the spam filter so again so this top part is just the same as i showed you on the previous slide so this is the traditional paradigm and here at the bottom so this part this is basically machine learning so what do we have here what's different between the traditional and the machine learning paradigm so here notice that we have now the outputs on the left hand side before they were on the right hand side and yeah that's basically the only difference so we have here a data set that is labeled later we will see not all machine learning algorithms are about labeled data sets some approaches exist also to work with unlabeled data the goals a little bit different but now let's say we have this approach where we have labeled data with that label data remember the outputs previously they were the spam and not spam so again here the outputs are spam and not spam these are our labels for our emails so we have now a data set consisting of emails and the correct classification the correct spam and not spam email and such a data set could be created for example by um yeah taking a look at your inbox and labeling the emails let's say you go through again 1000 emails you label them but now in contrast to the previous paradigm here let me use a different color here you develop the rules so here you developed rules as the program but now we let the computer develop the rules so we have the computer look at the labels the spam and not spam look at the emails and come up with a spam filter here and it's learning the spam photos so the computer doesn't does the learning work now so instead of us instead of us having a look here at all the emails the computer now looks at um labeled examples of emails and then develops the program which is so i should have um yeah so this is the process of learning the spam filter and what comes out of it is the spam filter so here the computer has now the role of learning from looking at examples that we provide so yeah that is uh machine learning in a nutshell i would say let me briefly follow up on the previous slide so there's a very i think important quote by faith lee so what she said is we will not only use the machines for the intelligence we will also collaborate with them in ways we cannot even imagine and i think the previous slide may may have made it look like that machine learning is there to replace humans but i think this is not really true i mean in some cases yes but it's not entirely replacing all kinds of human labor i mean there are cases where it makes sense to use machine learning and there are cases i think where it really doesn't make so much sense to use machine learning for the email example that i showed you on discussed in the previous slide i think it's really uh yeah still we are still developing programs that we like to use so you can maybe imagine in a distant future that some computer is developing or some machine learning algorithm is developing an email program but i think um nowadays we are not even close to that yet and we have people thinking about okay what is good usability what do i want what features do i want and we have programmers designing email applications or email interfaces however we can use machine learning to make the life of the programmer easier so instead of having the programmer to read through 50 000 emails and deciding what is spam and what is not spam maybe maybe maybe uh making bad choices we can have a computer doing maybe a better job at that and making the life of the programmer easier by developing the spam filter by learning by machine learning so that way we free up the time of the programmer to focus on other things and also don't have the programmer doing like boring things like reading through 50 000 emails and making decisions about spam and what's not spam so that way we can use machine learning to enhance certain workflows not really entirely replace them okay but now back to the definition of machine learning so if the previous slide was a little bit confusing where i was drawing the flow chart or the diagram maybe i can help you making this more confusing so um no i'm just kidding so i just want to offer an alternative definition and this is by tom mitchell so he described the learning process so the process of learning as follows he said um a computer program is set to learn from experience e so this is the experience with respect to some class of tasks t so we have tasks t and performance measure p if its performance at tasks in t as measured by p improves with experience e then this is the learning process so we have some tasks and we learn basically from the experience and we measure some performance p and we want to learn from the experience to become better at task t and this has to be some way we can tell that we make progress and that is the performance measure and there are many different types of tasks many different types of performance measures and the experience itself is basically in the case of machine learning like learning from the data so like i said this can maybe be a little bit more precise because we have now some yeah letters here so in mathematics you know natural symbol symbols help making things more precise i'm not sure if that's necessarily true for that quote but i think it really defines machine learning or the learning process nicely so in order to yeah kind of make this maybe more clear let me um give you an example a hands-on example so here that is a handwritten recognition example so what we have is the mnist data set it's uh m nist so that is a data set consisting of 50 000 images handwritten digits the digits are zero to nine and there are handwritings so there are if there are 10 classes and they are uniformly distributed in this case we have 5 000 images per digit so here on the right hand side i'm just showing you an example of each of each digit so this is an example of the letter zero one and so forth and for a human of course it's very easy to tell what digit that is but yeah for a machine we have to teach it to recognize what it is so now maybe as a optional exercise maybe pause the video and think about how this handwritten recognition example maps to the definition by tom mitchell where we have the task t the performance measure p and the training experience e so maybe pause the video and think about it a little bit and uh then continue okay after yeah pausing the video and thinking about it a little bit i hope you have some ideas of what the task t could be and it's actually just here the handwritten digit and recognition so here it would be the hand written digit classy classification um we can also say handwritten digit recognition but classification is the usually the more common term in the field of machine learning and we will also see exactly how this is defined in a few moments later in this uh or in the next video actually so next the training experience e so that would be the 50 000 images and they are labeled so they are labeled images labeled means again that we have the information available in the data set so we have the images and and also the labels 0 1 2 3 4 five six seven eight nine so that we can show the computer the images but also the computer has the information about the labels to learn uh to associate a given image consisting of pixels with a given class label so now the performance measure p what could that be and that is maybe something uh you or we could discuss in a discussion forum so maybe uh i suggest you open a new question on or new thread on the piazza forum on canvas and then we can uh have a discussion maybe i will let you go first you make some suggestions and other students can maybe chime in and discuss what the performance measure p could be that would be a good exercise to yeah double check whether yeah what your ideas are and what possible performance measures we could use and maybe what are the advantages and downsides of different performance measures so that is maybe a good question for piazza before i provide you with this information okay lastly some applications of machining we talked a little bit about the email spam filters so that would be um one application email spam versus uh none spam classy vacation uh many many other examples of machine learning uh in fact i would honestly have a hard time thinking about anything nowadays any like software or application that is not using machine learning so maybe another intuitive example would be sorting letters in the post office it's just related to the handwritten digit classification example from the previous slide so for example if we have a postcard with a classic post stamp then on the sender's address and then the address where it has to go there's usually a zip code for example i think the university's zip code would be five three zero no five three seven zero um four or something so let's say you have written the zip code on the letter at the post office um the they receive a lot of letters they go to different cities in different states so it would be kind of efficient to have an automatic sorting system and that is something where you have the handwritten digits could be computer printed but also handwritten and you have a machine learning algorithm yeah recognizing these digits and sorting them another classic example is you have your smartphone and you have like a little camera there and nowadays you don't have any buttons but you have just a screen usually the screen goes edge to edge so you just if i would draw a modern smartphone it would just be a square with a little circle here for the camera and it usually unlocks if it sees your face so there's some face recognition going on it's also one of the areas i'm working on my research areas i'm working on face recognition in any case that would be also a typical example of machine learning there are many many other examples um could go on and on and on but that is maybe also an interesting um yeah threat for piazza so maybe on piazza open a new discussion thread and have a discussion with the other students and maybe come up with some ideas um i will chime in then and maybe provide you with more ideas or also maybe just discuss some ideas you have related to machine learning whether it's a machine learning application or maybe not um yeah so maybe that's a cool question for piazza too okay so in the next video we will talk a little bit more about the different categories of machine learning