Live Day 4- Advance Statistics With Python In Data Science

Key Takeaways

hello guys i hope i am audible can you give me a quick confirmation hello hello so how are you all i hope everybody is doing well so i guess i know today's republic day but yesterday i could not take the session so thought of like completing it today and uh there's some more things that we really need to cover okay okay so we will try to finish off all those things but i hope you all are enjoying the session you are able to understand everything just give me a minute and then we will probably start okay okay shall we start can i get a go ad virender kumar you don't need to do standard scaling for decision tree so that is basically understandable problem the team like second wherever the standard deviation is more that that team have probably won you know based on the final question okay so let's start today what all things we are going to do first of all we are going to implement this iqr using python okay the second topic we are going to discuss about is probability the third thing that we are going to discuss about is something called as permutation and combination once we finish this up the fourth thing that we are going to discuss about is something called as confidence intervals okay so in confidence intervals then probably if we get time we will cover up p value and then we will start with hypothesis testing so these all things we are going to cover today and uh please make sure that you keep your laptop ready also write code parallelly so that you can practice all the things and uh with respect to all the materials that is uploaded in the community course which is basically given in the description of this particular video so go ahead over there okay and probably you can just have a look out to it okay so let's start first of all now what we are going to do first of all i am going to start with google collab you can also open google collab okay so i will just make a new notebook okay so first of all we'll try to implement zscore and try to find out iqr and with respect to that we will try to see what all things we can basically implement other distribution will also come don't worry bernoulli binomial distribution power law distribution everything will be discussed first let's go in some specific order i have actually decided and when that is those distribution will basically come we'll discuss about it okay okay so here you go so in this session we are going to first of all discuss about outlier okay as usual uh we have already seen the theoretical discussion with respect to yeah you can use jupiter notebook whatever things you require you can basically use okay and that is not a problem at all okay now first of all what i am actually going to do over here is that i am going to import some libraries import numpy as np okay import matplotlib dot pi plot that's plt and then i'm just going to import matplotlib inline okay so i'll be executing this now the next thing that probably we will be discussing about is that let's define our data set okay so here i'm going to just define our data set data set you can take up anything that you probably want okay you can just define your own data set whatever data set you like okay now for for my sake i have just created one data set over here so here you will be able to see that this is my data set can you say some numbers that are like kind of outliers in this yes it will be available videos are already available guys yeah so just tell me if you know some of the outliers and before come on guys i need to see more likes you know that will give me red bull like kind of thing okay so please go ahead with respect to the data sets uh tell me some of the outliers here what are the outliers do you think over here are present okay so uh now the first thing that we are probably going to do is that okay let's say that using z score i probably want to also find out some outliers okay now using z score how do you find out some outliers now let me just go and explain you over here let's say that you know about normal distribution till now you have discussed we have discussed so many things in normal distribution we know that this is the mean first standard deviation second standard division third standard deviation first second and third standard division to the left you know that 68 percentage of data 95 percentage of data 99.7 percentage of data can i consider that during some of the scenarios if my data is normally distributed after the third standard deviation probably the data are outliers yes or no yes after third standard deviation whatever data is basically present right data outliers yes or no just think over it most of the time if the values are you know after probably third standard deviation they are like kind of outliers yes so just think over it guys it can be treated as an outline right if if data is present after third standard deviation so first we'll try to implement this now what i am actually going to do over here is that first of all let me make a list okay so here i'm just saying it is outliers i'm going to basically create it as a list and put up all outliers inside this okay now the next thing let's define and how do you find out standard deviation or by using z score right we can definitely find out z score with the help of z how many uh data set or data points actually fall within the third standard deviation so here i'm actually going to create a function which says define detect underscore outliers so this will be my function and here i'm going to give my data okay now the first thing that i will create a threshold my threshold will basically be three standard deviation right anything that falls away from the three standard deviation i will basically be able to do it and i hope everybody remembers the formula the formula for z score is what if i go and probably define over here my z score formula is nothing but it is x of i minus mu divided by standard deviation we usually also write this formula by root n but i'll talk about it why specifically i'm not specifying root n over here uh over here i'll just try to use this formula okay so this is basically the z score formula okay so i have to implement this formula in python programming language okay so what i am actually going to do first of all obviously in in in this i need to compute mean i need to compute standard deviation you know how to compute mean right so here i will say mean is equal to np dot mean and here i can actually give my data points which will actually help me to find out mean then my standard deviation here i can basically write np dot standard deviation of that specific data i will be able to get the standard deviation right so i have got my mean and standard deviation now for each and every points inside my data set i will just apply the z score formula so i'll say for i in data okay i can say z score is equal to i i is my x of 5 points right i'll say x i minus mean right divided by standard deviation so this is my z score formula right and for every item i'm actually trying to find out the z score formula z score will basically give you how many standard deviation it is away from me okay so i can write one condition to check whether it falls below the third standard deviation or not so i can basically use nb dot absolute which will basically help us to round off the z score and i'll say z underscore score okay is greater than threshold if it is greater than threshold what does this basically mean let's let's define threshold over here i have already defined threshold right so if it is greater than threshold then what does this basically mean okay oh sorry it is data set i'm extremely sorry data set okay so let me zoom in a little bit more to you now tell me if np dot apps zco greater than threshold what should i do what does this basically mean green more clarity you want i think now it is fine right what what should we do in this this basically means that it is an outlier right because it is falling away from third standard deviation right so it is falling below or beyond the third standard deviation so what i can basically do is that i can just write something like this because i have created a list i'll say outlier dot append and i'm going to append that specific set score value okay so i hope it is fine i'm just going to append this z score value not z score i will append the i value because i in data set yes i am just going to append this i yeah outliers sorry it is outliers dot append of i and then finally what i'm actually going to do i'm just going to return the outliers return outlier let's see whether it will work or not i'm also trying it for the first time so this is my function that has got executed okay i will just execute one more code okay everybody clear with this code just give me a confirmation threshold three basically means this this defines our third standard deviation right below like beyond third standard deviation i can basically say that this actually falls on okay if you want to probably go and check how this distribution is so i can write plot.test on a specific dataset okay plt is not defined why okay this should be plt it's okay whether it is normally distributed or not but i am actually trying to see this okay there are some definite outliers okay but it's okay let's see that whether we will be able to do this or not what is which our past has changed data set data in for loop this is simple right guys this this function everybody understood or not oh sorry this should be data this data i'm actually passing over here fine perfect i think everybody is saying it it's fine okay see threshold threshold here is my third standard deviation third standard deviation okay if you want the data set i can paste this entirely and given the chart so this is my chat with respect to the data set i've already given it to you all okay so now let's go and execute it now i have executed this now what i am actually going to do over here i am just going to call detect underscore outliers and i am going to call this specific data set the data set dot apps nb dot apps basically means nb dot absolute okay absolute function now once i execute it here you will be seeing that it will be returning this three outline are these my outliers or not guys yes the for loop is very simple for i in data i'm finding for every data which is in the form of list all the z score and i'm comparing if the z score is greater than 3 or not if it is greater than 3 i am considering it as an outlier here you can see all the outliers are there okay outliers means a big number right if you have not attended the previous session guys see if you have not attended the previous session you can drop off okay because you'll not be able to understand this is a seven days live session right if you are not able to understand the previous one yeah this upper is really i'm going to put the username timeout okay okay now i have got the outliers right now this is one way how we can use z score so this was an example of actual z score so i'm just going to write it as z score right very simple okay z score computation and basically we have done it now let's go towards the iqr okay iqr basically means interquartile range so for interquartile range what type of code i will be writing always understand in iqr what are we discussing in iqr first of all we need to find out q1 q1 is 25 time then we have q3 q3 is 20 75 percentile okay then if i subtract 75 percentile minus 25 percentile i will basically get the iqr right and always understand in iqr what we do we basically find out what what do we do in iqr in iqr we basically find out the low the lower fence and higher bits that we really need to find out in case of iqr so how do i write the code because this theoretical is already explained so i'll write down all the steps that is required so the first step is that i want to arrange i want to sort the data let's say that i'm sorting the data okay this is the first step the second step is that i will calculate q1 and q3 right q1 and q3 is pretty much important in this particular case so i need to do it in this scenario i'll just move this up i'll copy and paste it over here okay so the first step is basically calculate sort the data and then calculate q1 and q3 then we need to find out iqr which is nothing but the third step which is nothing but the subtraction of q3 minus q1 okay then we need to find the lower fence find the lower pins now lower fence formula i hope everybody knows it so it is nothing but q1 q1 uh what is the formula guys tell me q1 plus or minus it is q1 minus 1.5 multiplied by okay i q r right this is the formula to basically find out the lower fence then find the upper fence here i will basically be using q3 plus 1.5 multiplied by iqr okay so these are the steps that we are probably going to do okay so these are my steps that i am actually going to plan for and based on the steps i will be implementing it okay so these are the steps that i will be performing in order to find the outliers with the help of iqr everybody clear with this yes everybody clear just give me a quick here so that we can continue the discussion right i hope screen is also visible now right now first of all if i really want to find out the sorted data set how do i find out the sorted data set sorted data set i will just say this will be my data set and i can use sorted function and in sorted function if i give you the data set this will basically be my sorted dataset so sorted is an inbuilt function which will actually help you to sort all the numbers okay okay sort all the numbers over here so right now i have actually created a data set which is completely sorted so my first step is done okay so i am done with my first step now second step i need to calculate q1 and q3 already in our previous session we have seen how to calculate q1 and q3 so i will say q1 comma q3 and here i will basically use np dot percentile i will give my data set over here along with this i'll give two values one is 25 comma 75 okay so once i execute it you can see that it has got executed now i am going to just print q1 comma q3 so here you can see which is my q1 q3 this is my 25 percentile this is my this is my what percentile this is my 75 clear everybody yes everybody clear with this i guess everybody is clear okay okay now once we have this now let's go ahead and compute the lower fence and the higher fits now in order to compute the lower fence and the higher pins here i'm just going to write the comment find the lower fence and higher pills okay the lower sense is equal to q1 right minus 1.5 multiplied by iqr right and before that i need to compute the iqr let's say iqr is equal to q3 minus q1 so if i go ahead and print iqr what is this error it is coming as okay now if i go and execute this you will be seeing that iqr is three right so this is my lower fence for the higher fence i will basically write higher pens is equal to q3 plus 1.5 multiplied by i cubed right so done once i execute it now i know my lower fence and higher so i'm going to print lower underscore pens comma higher underscore right so if i print it it is 7.5 to 19.5 everybody clear with this yes now the further part i think you can comfortably do it okay and uh based on this higher lower fence and higher pins you can write a condition and you can remove all the elements that is required clear everybody has some problem i don't know what is the problem with him so i'm just going to okay everybody clear with this yeah so now you can basically write don't worry whether the data is normally distributed or not here what we are doing is that whatever data set you are basically getting you are getting what you can actually do you can basically uh find the lower fence and higher fence and basically do these things okay now instead of doing all these things if i import c bond as sns okay and execute it and there is an option which is called as hist plot not sorry box plot we also saw how to create box plot right okay whatever asmita if the if the lower fence is negative then what you can do is that based on that condition any value lesser than that you can remove all those things right and here if i give my data set you will be able to see that this will be how a box plot will be created now this looks you see that there is a very big outlier right so that is the reason this same outlier we found out with the help of multiple things right and here also you can see 7.5 to 19.5 so most of your data points that will be lying over here will be based on that okay everybody clear if i probably remove those three elements and try to see that particular data set then this box plot will look bigger clear okay so i will just share this notebook to everyone i'll also upload it okay i'll share it here in the chat so you can basically find out this notebook okay and then you can basically click and check it out okay that's your full but what we have done with the help of easy python code we have done this okay everybody clear guys how to remove this you can write condition right any number that is greater than 7.5 just remove from the data set you cannot write see for i in data set write the condition no if i is less than this also i should write for you simple right if i is less than 7.5 and i is greater than just write the condition right that's it you will be able to do it okay fine let's go ahead and let's try to discuss about the next thing over here yes you can also use lambda function now let's go ahead and discuss about the next topic which is called as probability probability is super super important and in this session i will discuss major major important things in probability okay and we will try to see that what all things we can actually do with the help of probability probability is by default used in machine learning also in deep learning also many places let's say one example okay suppose i have two categories of data set like this right i have another category of data set if i try to create a best fit line you can see that let's say that this belongs to class a this belongs to class b okay now over here you will be able to see that if i talk about this right when i draw this linear line this is basically used in linear regression let's say linear regression now my question is that what probability of this particular point belongs to class a and what probability of this particular point belongs to class b because it is passing through the line so based on probability we can definitely get a lot of things in linear regression it is used in logistically it is used and all right so probability really focuses uh like base is basically used over there and different different things are used okay susha i'll come just wait right probability distribution will be explained everything will be explained don't worry right so let's go ahead and let's discuss about probability okay now uh let's understand what exactly is probability if you want to give a definition what exactly is a probability so here you can say that probability is a measure of the likelihood of an event okay probability is a measure of the likelihood of an event the reason why i am writing you this all definitions guys understand you really need to think you know what exactly is happening over here what is the definition you know if you can remember those definition in an easy way by example so that is the reason i also give you a lot of examples let's say that i am flipping a dice right i'm flipping a dice in a dice what are my possible sample events you know that it is one two three four five six now if i ask you a question what is the probability when i roll a dice or sorry roll a dice not flip flipping a coin it should be i'll say roll a dice okay so here i'm basically saying roll a dice so what is the probability of getting 6 if this is my question then how probability you will be able to calculate just tell me guys what should be the answer what should be the answer right what should be the answer in this particular case what is the probability of getting 6 what is the answer obviously you will say 1 by 6 right it's very simple so how do we define probability i'll say that number of ways number of ways an event can occur an event can occur divided by number of possible outcomes number of possible outcomes okay so this is the exact definition of this okay now in this particular scenario number of ways an event can occur over here i am trying to find out what is the probability when i roll a dice i get a six so how many events can occur it can only occur as one and what is the number of total possible outcomes it is six so this is how we basically find out right similarly if i give one more example okay let's say that i want to i want to toss a coin toss a coin right obviously i know what are my sample space head and tail what is the probability of getting head you'll just say that 1 by 2 because the sample space is 2 and one number of event that can occur is 1 by 2 so you basically say this as probability of header one by two this is very simple right this this uh you'll say that krish what is this man you're making us fool and sitting around over here uh please explain something else that we don't know right now let's go one step above probability which is called as additional rule now here is where you will probably discuss about something called as so let's let's go to the next topic over here i'm basically going to define as addition rule this is super important probably in your aptitudes you will be using this okay addition rule or we also say it as probability or or or or also you say it as like this or okay now in order to understand additional rule you need to understand about two things one is mutual exclusive events okay what is this mutual exclusive events anybody who can probably give the definition to me about mutual exclusive events it will be amazing and yes please keep on hitting the like okay what is mutual exclusive events okay so i can basically define two events are mutual exclusive two events are actually mutual exclusive if they cannot occur at the same time if they cannot occur at the same time okay let's see an example rolling a dice rolling a dice now when i roll a dice at a specific time i can either get 1 or i can either get 2 or i can either get 3 or i can either get 4 or i can either get 5 or 6 right you cannot get 1 and 2 at the same time or you can't get one two three four at the same time you will only get at one one probably one experiment or one event that you're probably rolling a dice at a single time you'll only be able to get one number you will not be able to get two numbers so this is specifically an example of mutual exclusive another example again uh tossing a coin in this particular case also right tossing a coin in this particular case also what happens you may either get head or tail you cannot get both right unless and until your coin is standing there like shown in the movies i hope which movie am i talking about which movies probably i'm talking about you can also consider you know good movies like show chole and surely only one type of event occurs at every time right so this is very much important to understand okay so for this kind of problem scenarios now let's let's discuss let's discuss about non-mutual exclusive obviously you understood that what is mutual exclusive now with respect to non-mutual exclusive obviously both the events can occur at the same time right multiple events can occur at the same time here i'll say that multiple events that can occur at the same time can occur at the same time right two or more events can occur at the same time let's let's say one example think of an example guys right think of an example whatever comes in your mind let's take a deck of cards let's take a deck of cards a very simple example with respect to this in deck of cards have you seen like what will happen in a deck of cards right two events let's consider that from a deck of cards when i pull out a card a king can also come okay or or let's say that a queen card can come along with the screen card a red color heart card can also come hard card can also come right so here multiple events are there right so this two cards are obviously not mutual exclusive right so here you can see that okay i can also pick up a king it can be in black color it can also be in red color right multiple things are basically so this is an perfect example of a non-mutual exclusive okay getting getting spades and s whatever you want you can basically do it right now based on this there is some amazing problem statements that you can basically solve okay and that is what i'm actually going to discuss now some very good problem statement okay uh let's let's consider uh with respect to mutual exclusive event first of all okay mutual exclusive suppose if i toss a coin so my first question is if i toss a coin toss a coin which is again a mutual exclusive event what is the probability probability of the coin of the coin landing on heads or tail now whenever you get this kind of problem statement first of all you really need to think that okay whether it is mutual exclusive or not yes obviously it is mutual exclusive mutual exclusive it is obviously mutual exclusive now i need to find out what is the probability of getting heads or tails right this is what i i need to find out right i need to find out what is the probability of getting heads or tails right from this specific event so i want to define a common definition probably for this we can write probability of a or b where a and b are events is equal to probability of a plus probability of b so whenever you have a mutual exclusive event at that point of time you can define this specific definition which is also called as additional rule which is also called as addition rule right for mutual exclusive now here what is probability of a you know that it is 1 by 2 plus 1 by 2 so the answer will be 1. so probability of a or b to come is basically one these are some very very important things in in exams also you will be getting this in aptitude also you will be getting it in multiple things you will basically be getting it right now i may also tell you okay let's take one more example okay what is the probability suppose if i roll at is what is the probability of getting one or three or six not do this everyone for me what do this everyone for me what is the probability of this i hope everybody is able to understand guys is the session going on good or not yes yes many people are saying it right it is one by two so here i will basically say that what is the pro this i can definitely say it as probability of 1 plus probability of 3 plus probability of 3 6. these all are 1 by 6 plus 1 by 6 plus 1 by 6 which is nothing but 3 by 6 which is nothing but 1 by 2 which is nothing but 0.5 okay so 0.5 is basically with respect to this and here you can easily solve it okay now this was with respect to mutual exclusive okay this is what we have discussed in mutual exclusive if i take the next problem statement for non-mutual exclusive so for non-mutual exclusive let me take a very good example again okay so the so the question is very much simple over here let's say that you are picking a card you are picking a card randomly this is the question from a deck so the question is what is the probability what is the probability of choosing a card that is queen or a heart so this is the question very simple obviously first of step you will see that whether it is mutually exclusive or non-mutual exclusive obviously you will say that in this particular scenario it is non-mutual exclusive or mutual exclusive it is non-mutual exclusive right because it can occur at the same time okay now let's go ahead towards the answer obviously you understood that this is non-mutual exclusive non-mutual exclusive right now how do you solve this specific problem now in this specific problem first of all you need to find out what all different things it is basically asked let's say that i have got probability of getting a queen what is probability of getting a queen guys just think over it how many cards how many queen card will be in deck of cards in the total deck of cards there are 52 cards right if none of you have played cards please go buy today and see okay i'm not saying you to put your money and you know waste your money or do gambling so the probability of getting queen is nothing but 4 by 52 because in every deck there will be four queen cards now the next thing probability of heart cards so what is this probability of heart how many heart cards will basically be there in a deck obviously there will be 13 cards so i'll say 13 by 52. now the next thing is that probability of queen and heart because this is also one one possibility right this is also one possibility how many queen and heart will be there there basically right it will be only 1 so here i will write 1 by 52 so these are the possible things that can occur right this is the thing now if i come to the formula and this is the addition rule for non-mutual exclusive events non-mutual i can write probability of a or b is equal to probability of a plus probability of b there will be one important thing which is this intersection which i have to basically separate it so it will be p probability minus p sorry minus probability of a intersection b a intersection b basically means a and b which is the possibility of both now my question is very much simple what is the probability of getting queen or hot i'll draw it with red color you have the answer with you this will be probability of queen plus probability of heart probability of heart minus probability of queen and heart okay so what is probability of q 4 by 52 what is probability of heart it is 13 by 52 and what is probability of queen and heart it is one micro q so here i am actually getting 52 this will be 17 minus 1 16 16 how much the 52 you can calculate this will basically be the probability clear everybody yeah 4 by 13. yeah okay now you have probably understood additional rule addition rule now we need to understand one more rule in probability see guys if you do this much i think you will be able to solve any problem statement that comes in your mind okay that is what i feel okay i don't know like what other skills or whether other will be able to do it i cannot guarantee whether you will be able to do it again okay so here was the problem statement that we did and this was specifically to something called as addition rule now coming to the third one which is called as multiplication rule so the third one is something called a multiplication but just tell me guys how is the session going on i hope you are liking it i hope none of you are scolding me khali billy galik you know if i teach also they will say krish okay you're teaching then also i'll scold you if you're not teaching then also i'll scold you okay so sometimes the energy becomes low okay so it is very difficult to satisfy everyone okay so multiplication rule now let's go ahead and discuss about the multiplication rule in multiplication rule you one thing you need to understand here we need to understand something called as independent independent events and non-independent events these are something very very important okay okay so non-independent events yeah okay so independent events and non-independent events sorry it should be and i said or hot okay independent events okay now in the case of independent events events what are specifically independent events let me talk about example okay like this example okay let's say that i am rolling a dice rolling a dice if i roll a dice i may get one two three four five six okay suppose for the first instance i got one right in the second instance it is possible i may get one in the third instance i may get two i may get any number so one event is not at all dependent on the other event right because anytime we roll every every possibilities or every outcomes has an equal probability to come right so over here what you can understand is that each and every events each and every events each and every events are independent okay one if one one comes or if two comes out if any events come it is not going to impact any other event every time you probably have to roll and everybody has an equal probability to come over here this is what is an independent event called as okay let me talk about non-independent event or i will also say it as non not non-independent but instead i'll say dependent events so i will talk about dependent events now independent events suppose let's say that i have a i have a bag in this bag let's say i have three red marbles and two green marbles okay now in the first instance if i pick out if i if i pick up a marble what is the probability of red marble in the first instance just think over it guys what is the probability of taking out a marble what is the probability of taking out the marble very simple you will be seeing that how many number of marbles are there there are total five marbles and how many number of red marbles are there there are three marbles right so you are basically able to write three by five right now let's consider in the first event you picked out a red marble so you picked out a red marble so i'll make it as red color okay red color marble okay now after taking out the red marble how many marbles are remaining so i will now update this okay so how i'll update this i'll update this bag now this bag will basically have two red marble and two green marks now if i try to go ahead and find out what is the probability of now taking out a green marble then how you will basically say how you will basically say you'll see that okay how many number of marbles are there two by four i'll say right so here what is happening after this particular event it has impacted this event right because the number of models are reduced right and finally you got 2 over here so this is a perfect example of a dependent area clear guys so multiplication rule basically says that in the case of an independent event we have to solve it in a different way in the case of a dependent event we have to solve in a different way because of this dependent event there is an amazing algorithm which is called as name bias have you heard of neighbors i think most of you have heard of right there is a topic which is called as conditional probability this is where conditional probability will come into existence okay so i will i will talk about it okay so let's go and solve some problems right so let's go and solve a problem till i hope everybody is liking it you have to keep on hitting like we have to make thousand likes then only i'll get red bull okay uh you know people over here will not give me red bull uh unless and until you give me a thousand likes okay so please help me out with this red book okay so let me just go ahead and talk about a problem statement first we will talk about independent events okay so independent events we are going to basically discuss about the problem okay so first thing the question is what is the probability of rolling a pipe and then a 4 in a dice so this is your question what it is saying what is the probability in the first event you have rolled a dias you are getting 5 and then again you rolled a dice then you got 4 so what is the probability of getting 5 and then 4 this is a simple question and for this this obviously is an independent event you know that right you know that right now how do we solve this particular problem so i'll say independent event i'll say here we'll apply the multiplication rule what is the multiplication what is the probability of a and b a and basically means first a event has occurred and then b event has occurred what is the probability of this so obviously here i'll define the formula over here first of all i'll say probability of a multiplied by probability of b a and then b okay so this is the usual formula that we use for an independent event in a multiplication rule okay so obviously you know what is probability of a so here i'll say probability of 5 and 4 you know probability of 5 probability of 5 is nothing but 1 by 2 multiplied by sorry 1 by 6 1 by 6 multiplied by 1 by 6 it is nothing but 1 by 36 right everybody clear with this yes everybody clear with this now let's take another example obviously because independent event looks very simple okay so here i'm basically going to take another example and this example will be of a dependent event okay so let's go ahead and let's try to solve a problem for this okay now what is the probability of drawing a queen and then a and then a aces from a deck of card see over here two events are actually happening okay so let's go ahead first of all again you need to find out whether this is an independent or dependent event obviously in this case this will be a dependent event because a deck of card will get reduced right so in this particular case i am saying what is the probability of a and b in the case of independent event so here i can basically write probability of a multiplied by probability of b given a now what does this mean this this term is basically called as conditional probability conditional probability let me show you an example with respect to the bags right so i have a bag over here let's say that i have three marbles two red marble okay now in the first instance i want to find out what is probability of what is probability of what is probability of green and then red marble okay now see over here how many marbles are there in the first instance if i'm taking out green obviously there is three by five right in the first instance when i took out the green marble after i take out the green marble my total number of marbles that will be remaining is four so the probability of red will be 2 by 4. now this term this term is basically probability of green and what is this term 2 by 4 this is nothing but this only right multiplied by probability of green given red sorry probability of red given green given green basically means this green event has already occurred this event has already occurred right so that is the reason the number of marbles has got reduced this is called as conditional probability and this is very very helpful in something called as nay bias or i'll also say it as bias theorem in bias theorem this will be very very important okay so we will probably in the further i'm also planning for a seven days live session on machine learning algorithms all the machine learning algorithms where probably i will be discussing about them where bias theorem when i talk about bias theorem conditional probability will also come okay so here what is probability of king sorry it is queen and king right queen and asus sorry so here what i'll do probability of queen multiplied by probability of asus given queen so what is probability of queen it is nothing but 4 by 52 multiplied by 4 by 51 so sorry 53 how many cards are there i forgot how many cards will be that in deck 53 right yeah no 52 only don't confuse me guys okay 4 by 52 multiplied by 4 by 50 so whatever answer you get over here this is basically your answers okay so clear everybody with this yes so we have finished up this topic this topic and now we will finish up this topic also okay good everyone happy happy then drink happy okay so why the likes are not going above thousand because they are less people okay now let's discuss about something called as permutation and combination a very small topic probably in five minutes i will be able to complete it okay clear everybody select continue okay now let's say that first of all let's discuss about permutation let's say that um um i have taken some students to a school trip and then we have gone to something like a chocolate factory in which many chocolates are basically they they they create a lot of chocolates they they okay so they they make a lot of chocolates okay so i i catch a word of a student and i say that okay i'll give you an assignment okay i'll give an assignment and let's say that in this chocolate factory six different types of chocolates are created like dairy milk right like five star milky bar okay and let's say eclairs okay jam how many one two three four five and one more chocolate uh normal toffee um hot chocolate i have not eaten chocolate from long time what is that chocolate name um i start dairy milk let's say one more category silk of dairy milk is there okay so okay so this many chocolates are basically there so i have given a student an assignment to that saying that okay there are six chocolates that are getting created in this factory let's create in your diary you write the first three chocolates whichever you see okay whichever chocolates you see once you enter into that factory whichever chocolate you probably see the top three the first three you just write that name and you come up come back to me so that student went inside the factory now in the first instance how many different options this particular student can have of seeing the chocolates he may definitely have six different options right now once he sees probably any one chocolate right he may have six options because six different any any chocolate he may see right so obviously he may have six options out of which he writes one name over here let's say in the next instance how many choices will remain total five will remain right so how many options he will have to write the name five he'll have the right to write the name of the chocolate then finally here you'll be seeing that when he comes and write the third name over there they'll be having four options now if i try to multiply this 6 multiplied by 5 multiplied by 4 it is nothing but 120 now 120 what it is it is all the possible permutations with respect to the chocolate name that he may see okay all the possible permutation like he may he may see in this way dairy milk gems milky bar he may also see in different way milky bar gem dairy milk so all the possible options that are possible is 120 okay now when i say 120 okay these are all the possible options now this is what permutation is okay but till now in school days you have learnt about permutation with the help of a formula but i wanted to show you how you are getting that okay let's see this is all things i started studying when i when i was preparing for data science okay before this visualized way i used to not study okay so permutation formula how do you write now let's go back to school days where directly used to ratify all the formulas npr is equal to n factorial divided by n minus r factorial over here n is nothing but the total number of chocolates r is nothing but how many names i have told that person to write right so here you will be seeing 6 factorial divided by 6 minus 3 factorial which is nothing but 6 into multiplied by 4 multiplied by 3 factorial divided by 3 factorial this and this will get cut so total answer is 120 very simple okay very very simple yes i hope everybody has understood it just give me a quick yes if you have understood till here okay now this is with respect to permutation now how does combination come into existence now and what is the difference between permutation and combination now in combination always understand permutation if i have the same element like this i have dairy milk i have gems okay i have gems i have probably eclairs if i've used this element once this combination i cannot use this same element and probably make a different combination right so combination will be unique with respect to the elements that is used okay if i have used derivative gem and eclair i cannot again re-swap it and make it as a different order okay so in the case of combination you have a other formula which will actually for help you to focus on the uniqueness of the objects that you are picking up okay so for this the formula is ncr which is nothing but n factorial divided by r factorial n minus r factorial what is n factorial you know that the 6 factorial what is r factorial 3 factorial and 6 minus 3 factorial so here you will basically say 5 move 4 and this will be divided by 3 factorial this i'll make it as 3 2 1 multiplied by 3 factorial this and this will get cut two ones are two twos are three ones are three two five twos are ten ten two twenty so 20 unique combinations you can basically have okay everybody clear so how was the session till now i hope now will be the most interesting thing that is that is confidence interval p value we will discuss about it clear everybody shall we now start something called as confidence interval this will be very much amazing okay okay so we have completed the third topic i will continue the session till 30 to 45 minutes and see that what all things can be completed okay let's say first of all the first topic that we are probably going to discuss about is something called as p value super super important topic many people gets confused gets confused in this okay now let's take one example everybody uses a laptop everybody uses a laptop let's say that this is my laptop this is my mouse pad okay right this is my mouse pack okay this is your right button to click this is your left button to click your laptop mouse pad over here you will move the fingers right here you will move the fingers okay now let's go ahead and let's understand don't you think most of the time when you're moving your fingers you will be moving in this specific region in this specific region right you will be moving your fingers in this specific reason not in the corner hardly you'll touch somewhere in the corner right hardly you'll touch somewhere in the corner okay now why i am specifically drawing this because this thing will basically specify your distribution of touches and most of the time your distribution of touches will be also looking something like this okay now understand one thing why this area is bulged this area is bald because most of the times you'll be touching here this area is less because over here hardly you will be touching over here okay now let's consider that i say my p value for this position is my p value for this position is 0.8 okay my p value for this particular position is 0.8 okay now here what i am actually going to do what does this point 8 basically means that let's say i am doing 100 times i am touching this mouse pad 100 times i am touching let's say that every 100 times every 100 times okay let's let's uh remove this i'll write in white color only every 100th time i touch the mouse pad 80 times out of this 100 80 times i touch this specific region okay i hope everybody understood this one every 100 times probably i touch this mousepad the probability of touching this region is 80 times that is 80 percentage similarly if i say my p value over here is 0.01 what does this mean now tell me what does this mean you you specify me the answer you specify me the answer right you specify me the answer if my p value is 0.01 how many times probably i am actually touching over there similarly you can consider any region this region is the the best like broadest right so this region may be p is equal to 0.9 that basically means out of every 100 touches i am basically touching 90 times over here this will be one time this will be only one time right so i hope you are getting the understanding of p value p value basically says most of the time what is the probability with respect to a p value for that specific experiment okay this is very much simple for u1 perfect now let's go ahead and let's understand something called as now i'm going to combine multiple topics the first topic that i am going to combine is something called as hypothesis testing in that i am going to combine confidence interval in that i am going to combine significance value in that i am going to combine many things okay so like this we will try to understand things this is super important guys if you have not probably attended the first hour session it's okay understand this this is super super important definitely helpful for your okay for your interviews okay now let's say i am solving a problem okay let's say that i'm solving a problem okay my problem is to i have a coin i want to test whether this coin is a fair coin or not okay simple problem statement i have a coin i want to test whether this coin is a fair coin or not by performing 100 tosses okay now we are entering into inferential statistics okay very important super important now when do you think a coin is a fair coin tell me obviously when the probability of heads should be 0.5 when the probability of tail should be 0.5 if you have this two condition definitely you will be saying that yes in this particular scenario obviously the coin will be a fair coin but if you have a chole coin if you have a sholey coin then what will happen if you have a sholey coin then probability of heads 100 so for this kind of things you'll definitely not say that it is a fair coin right now in order to support this i am performing 100 experiments 100 experiment basically means 100 tosses okay so 100 tosses i will be performed okay now inside this 100 tosses what i am going to do is that let's say that from this 100 tosses obviously what will be the mean let's say that i'm just focusing on probability of head i should basically get 50 times so from the 100 times from this 100 times if i'm performing 100 experiment i can definitely say that my probability of head or probably let's let's consider that forget about this probability of head the number of times i should get head is how much 50 right if i get 50 times head i can definitely say that this coin is the coin is fair yes yes yes or no the coin is fair i can definitely say this if the number of times after performing 100 experiment if i get 50 times head i can definitely say the coin is fair okay now very important first of all in this particular scenario we have to focus on something called as hypothesis testing we have to focus on hypothesis testing in hypothesis testing the first thing is that we need to define our null hypothesis the null hypothesis is usually given in the problem statement what is what is we want to test whether the coin is a fair coin or not so whatever the default question is i'm going to use it as a null hypothesis so here i'm saying that the coin is fair like one scenario you have right uh a person cannot be acquitted as a criminal unless and until it is proved right so the coin is fair now the second thing that we basically define is something called as alternate hypothesis here i'll say the coin is unfair okay the coin is unfair now the third step and always remember alternate hypothesis will be the opposite of null hypothesis whatever thing we are trying to put okay now the third thing is that we perform the experiments and the experiment can be anything it can be a z test t test whatever things you want you can do all this practical i will discuss it don't worry okay probably today some part will get covered tomorrow we'll start the practical part where i'll take a lot of problem statements and do it okay experiments now inside this experiment we see some values and based on that the fourth step that we do we reject or accept the null hypothesis null hypothesis these are the possible step these are the possible step of the hypothesis testing okay now let's define this guys let's say that my mean value is 50. i need to get at least 50 times that right i need to get 50 times head yes or no let's consider that this is my mean okay minimum 50 i'm not minimum but 50. 50 i should be getting in order to say that my coin is fair let's say that for this problem statement uh i'm just examining okay the standard deviation is 10 so it will come as 60 70 80 90 okay 40 30 20 10 okay right in this particular case it is there and probably if i if i know my mean and standard deviation i may draw a curve which looks like this everybody clear with this very simple not nothing so new i have taught you everything is same okay okay now what happens if i want to prove this now see this i'll perform the experiment let's say i have performed 100 times now just imagine i got 30 times head let's imagine i got 30 times ahead 30 times head is nothing but it is somewhere at this point at this point can i still say that this coin is fair or not can i say the coin is fair or not or coin is unfair can i say think over it if i am getting 30 times head can i say that this coil is unfair you tell me whether it should be fair or unfair tell me let's say that i have performed the experiment and i got 30 times head out of 100 so tell me whether this will be fair or no many people are saying no not fair fair fair not fair so for this to define it is always said that our experiment should be nearer to the mean okay nearer to the mean now how do we define that how far it can be away from the mean we need to define that how far it may be away from the mean so for that we use a very important property which is called as significance value is called as significance value now this significance value is basically given by alpha suppose let's consider that i am considering alpha as 0.05 okay 0.05 now this point 0 5 what exactly it is what exactly it actually means okay this means that if i do 1 minus 0.05 this answer let's say that this answer how much it will come it will basically come as okay let me just remove this once again okay this point zero five okay i've taken my significance value as point zero five when i convert this into percentage it will become five percent okay five percent so from my hundred percent if i subtract five percent this basically indicate that it is 95 confidence interval now what is this 95 confidence interval okay if i probably subtract from one my probably the 95 percent confidence interval is there okay now this 95 percent confidence interval is what part let's consider that i know my 2.5 is this part 2.5 is this part since this is a two-tailed test let i'll talk about two-tailed test also don't worry okay so let's consider at this part to this part okay this is my entire 95 confidence interval okay this is defined by a domain expert different defined by a domain expert let's consider that it has been defined by a domain expert now what does that 0.05 indicate i'm trying to show it to you when i probably divide this into two parts here my two point five percent will come here my two point five percent will come okay here my two point five percent will come now understand one thing very important over here now let's say that 30 i got 30 over here so this is my 30 right this is my 30. and i have also defined my confidence interval from this point to this point whenever we are coming inside this then we see we say that the coin is fair why because understand it is within this interval here we need to define because we don't know right what should be the number you said that when i got head 30 times many people is saying that not fair but who are we to decide domain the expert will decide and how will he decide with the help of this significance value suppose they say significance value is 0.05 that basically means that we the the experiment if it falls in this 95 confidence interval that time i will say that that coin is fair if it falls outside this confidence interval that time i will say that the coin is not fair now tell me let's say that this number that you are seeing is 20 let's say and this number that you are seeing is 75 20 to 75 is my confidence interval now i perform the experiment if i get 10 heads only out of 100 experiments should we accept or reject the null hypothesis the null hypothesis is basically the coin is fair the null alternate hypothesis coin is unfair so if i get 10 heads which region it is falling it will fall somewhere here it is not inside the confidence interval so we can definitely say that coin is not fair so for that particular case we reject the null hypothesis and we accept the alternate hypothesis i hope everybody is able to understand the terminologies that we are using over here i cannot teach you separate topics understand i have to combine these topics together to teach you how to do it right now if you have understood these things tomorrow i will show you how to find out the confidence interval suppose i have a problem statement how do i find out the confidence network if my alpha value is 0.05 what should be this value what should be this value that also i'll help you to do it exactly i hope you are understanding this guys or not yes or no so this is how we basically do it right so how did you define ci will be from 20 to 75 i have just assumed i have just assumed over here tomorrow i'll show you how you can find out confidence interval okay you have to do that particular part and for for that you need to know something called a z test t test and all okay till here revise the things but here in just five minutes i have smartly covered hypothesis testing confidence interval significance value and p value right significance value is not equal to p value understand one thing okay p value over say you are just saying the probability significance says that what should be within your confidence interval next time when you can actually perform experiment you can say that the p-value was less than 0.05 like that if it if you say p-value is less than 0.05 that basically means it is falling somewhere here right it is some falling somewhere here if you say p value is greater than 0.05 that basically means you are saying that it is falling here right so i hope everybody has understood this what if we if we have okay let's say that guys if you have 95 heads in those 100 experiments which region it will fall will it not fall in this region 95 is somewhere here here so should we accept the null hypothesis or reject the null hypothesis should we accept the null hypothesis or reject the null hypothesis tell me should we accept the null hypothesis or reject the null hypothesis we have to obviously reject the null hypothesis and alternate hypothesis will be accepted it's very simple i perform the experiment whatever value i get i go and check in this okay let me tell you now one more one more scenario okay here let's say that my domain expertise said that krish you are a fool okay and probably i will now use this is 50 60 70 80 90 okay let's say that krish you are a fool why have you taken alpha 0.05 okay i don't want that oh so let's say that your alpha is 0.20 now what will be your confidence interval what will be your confidence interval let's say that your confidence interval will be now 80 percent instead of 95 so now your graph will look somewhere here like this it will be still more in this side so this side will basically have point one zero this side will basically have point one zero and this all will be your point eighty percent when you combine all this when you add up all this it will be one right so at that point of time then you can go and find out your confidence interval this value will give you your lower confidence interval this value will be giving you a higher confidence interval you perform the experiment now just imagine you got 25 from that experiment whether you should reject or accept it origin origin roy i have just considered standard deviation as 10. i told you right i am assuming some values here the main is aim is to understand to make you find out the relationship between significance value confidence intervals what is hypothesis testing this all are integrated together right now tell me tell me one thing if your alpha value is 0.3 what is your confidence interval vishwasharam i just took it for heads only right so what is your confidence interval if your alpha value is 0.3 what is your confidence interval when your alpha value is 0.3 obviously you'll say that it is 0.7 that is 70 confidence interval so alpha significance value and confidence interval are reverse right they you need to calculate in that specific way right now usually when we say when we say like p value right suppose if it does not follows in the confidence interval i may say that the p value is less than 0.3 so because of that i have to reject the null hypothesis so my suggestion will be please go ahead and revise this session again okay when you revise this session you will be able to gain it tomorrow i'll tell you i will tell you how to find out confidence interval value with calculations that time you'll love it right now i've told you how this is related to each other okay how this is related to each other tomorrow first of all first topic that i will talk about is that how to find out confidence interval right because it is an experiment right it may it may be greater it may be max it may be less right because understand one thing that our mean our sample mean will always not be equal to population mean we are trying to find out an estimator so tomorrow i will be talking about confidence interval and how to calculate it okay so i hope you like the session yes tomorrow type 1 type 2 error will come tomorrow i'll just talk about all the topics that are going to come tomorrow first of all i'll try to show you how to calculate confidence interval then i'll be talking about type 1 type 2 errors one tail two tailed test then probably will be discussing about one sample z test one sample t test then we are going to discuss about uh dependent sample t test then uh z test for proportion then anova test then chi square test this is how things are going to go so in the upcoming days the session will be quite amazing i would suggest please revise all the things and come in the final day we will talk about all the distributions all the remaining distribution okay so i hope you have understood this guys yes how was the session would you like to rate anything out of 10 so i'm just going to save this community section 4 dot pdf it will get uploaded in just some time yeah so i hope you like this session i hope i was able to make you understand okay sir we don't usually reject null hypothesis it is based on the experiment okay okay perfect thank you okay guys so this was it from my side uh i hope you like this session i will see you all in tomorrow's session shark 7 pm so keep yourself ready tomorrow fifth day sixth day is very very super super important please everyone don't miss tomorrow's session right tomorrow will be like a eyes opening for you all right like that we'll try to learn things okay so thank you guys this was it from my side have a great day please make sure that you like and share with all your friends and yes tomorrow i will be joining the session thank you