Fairness Aware Outlier Detection

Key Takeaways

welcome to data skeptic pilot season i'm trying out a few new themes for the show as we transition out of data skeptic consensus and into whatever comes next this is one of probably two more we're gonna do before i ask everyone to vote so check out the recent run we've had and form your opinions today submitted for your approval data skeptic time series a delightfully broad topic today in particular what we're going to get into is outlier detection or more specifically fairness aware outlier detection after all what does it mean to detect an outlier to find something unusual to find a minority class and i mean that in the machine learning sense of that word but of course it could be used even inadvertently in a damaging way so it's great to see things like the fairness aware outlier detection methods discussed in the paper we're going to go over today hi my name is branchu and i'm a phd student in machine learning and public policy at carnegie mellon university i my research basically focuses on pattern mining and anomaly detection and decision support systems i think a lot of people have a vague conception of what anomaly detection is you're welcome to give a formal definition or an informal one but to you what is anomaly detection anomaly detection is typically like i'll give you a informal definition or informal sense of what anomaly protection is so anomaly is defined as anything which deviates from the normal so as you can see like the definition of normal is again dependent on the application or the domain where you are applying some concepts and so on so for example if you are looking at a classroom right so if you're looking at undergraduate classroom and if you're just recording ages of the people present in a classroom all the students typically would have age range between say 18 and 21 and maybe professor has an age range between 35 plus or something like i'm just just making this up right so in that sense the normal is defined because we are only recording age so in that sense normal is defined by maybe the ages around 20 so in that sense the teacher would be an anomaly in that sense because the teacher's age his or her age is like very different from the normal right so in that sense the teacher would be anomaly in this case so essentially anomaly detection is a field which tries to find unusual activity or unusual observations in a given domain and that's what anomaly direction deals with and since it's about finding unusual things you typically find applications of anomaly detection intrusion detection say fraud detection or even in medical domains like say epileptic caesar detection and so on i imagine there's a broad range of different methodologies one could try maybe some unsupervised for sure possibly some supervised what does the typical landscape and toolbox look like so anomaly direction by design is a ranking problem wherein it's an unsupervised task where given a collection of objects you assign some score to each of these objects in a rank order fashion where the object which receives highest score or lowest score a typically highest score is an anomalous point or is most anomalous compared to the group of or the collection of points that you have so anomaly direction is typically a ranking task where you rank most anomalous or most unusual point up to most normal point and so on so since anomaly reduction is about unusual activity detection typically most of the problems fall into unsupervised learning uh domain where you don't have a priory knowledge of class labels but there could be certain domains where you may have like supervised anomaly direction but then supervised anomaly direction would be very close to supervised learning problems with skewed class distribution so skewed here i mean by that like there would be only a few examples of unusual activity and rest everything would be usual activity so you could solve those problems by using supervised problems but typically then you will have to employ a techniques to deal with high skewness of the data because in any system number of anomalies would be somewhere around 0.1 or even less than that and that's very hard to train with when you have uh supervised spending problems so i think people typically utilize unsupervised methods which could be either pre-based or density based and so on yeah anything with outlier detection is inherently going to have that class imbalance problem where you have so few examples of the minority usually interesting class like fraud detection is a great example that you'd mentioned the people committing fraud are not going to be so kind as to reveal their nature they're going to try and trick you so it's got to be something new and novel you touched on a couple other domains and one that caught my attention was healthcare i was previously not aware of any real concrete use cases for anomaly detection and outlier detection healthcare could you go into a little bit more detail on how we benefit from the use cases there i'll just give you an example of one of the problems where i'm working on caesar detection in a medical domain so typically like a human brain would operate between certain ranges of voltages for each person that normal range would differ but typically it would exhibit certain behavior during normal periods but during seizures what happens is like caesar is essentially like either a very high activity in one part of the brain and almost no activity on the part of the brain and so on it could be either way typically very high activity or very high frequency activity going on in the brain so in that sense in that landscape so if you're only looking at voltages simplest case so you have a timeline of if you are observing a person over time and you see like typical voltages are say some one micro volt or two micro volt or something like that and suddenly you see spikes of like 10 micro volts 5 micro volts or even more than that so in that sense you have observed a voltage which deviates from your normal observations which is around 2 micro volts or something so in that sense you could easily detect seizures although in real world applications not that straightforward but the underlying idea is still the same so in a case like that it feels to me like it's all upside that these are you know additional automated steps that could benefit healthcare and doctors and maybe you know raise attention to catch things early or expand our ability to check patients so all positive things for sure even in fraud detection i don't necessarily see a downside because if the company makes a mistake and i'm a legitimate user and they turn away my business i guess i can go to a competitor but then i started thinking about what about anomaly detection and security cases lots of good examples but i would hate to think that at the airport tall people and maybe red-headed people and just rarer minority people would be flagged because they're outliers compared to the general public how can we introduce some notions of fairness into these ideas anomaly detection is about detecting unusualness but then unusualness in the statistical terms is basically low density region in a feature space right so what happens is that when you give an example of airport security and people with different types of attire or hairstyles and so on any person which has a different kind of hairstyle would stand out in this feature space and standing out means that it will be unlike any other point which means it will have low density and low density is so is statistically low density is our anomaly by definition so in that case like if say for example if a person has a raised collar of his or her jacket and we would maybe we will flag them as an anomalous person but this raised color if that doesn't correlate with the actual harm that that person can do then this system then becomes unfair in that sense unfair with respect to minorities so recently we worked on introducing notions of fairness into outlier detection which is very challenging because of two aspects one is like in supervised learning scenarios there have been numerous fairness metrics that have been proposed so it's very hard to figure out which one could be applicable to outlier detection so then we thought about introducing fairness into outlier detection the first problem that comes into outline detection introducing fairness notion to outlier detection is availability of so many choices for notions of fairness and their formulations most of these formulations come from supervised learning problems which have received unusual focus in terms of fairness in machine learning while there is like almost non-existent work and fairness in outlier detection so the problem becomes challenging as to which notion of fairness would work best for outlet detection the other thing is like as we talked about outlier detection is we only have unlabeled data or it's an unsupervised learning problem so absence of ground truth outlier labels makes this problem even harder to even post processing or some sort of processing that many of these supervised fairness algorithms applied right so to that and in our recent work which is called as fair od we have first of all designed uh decider to cater to this problem of fair outlet detection we also give a formalized problem definition for that and then we introduce a fairness criterion that could be incorporated into an outlier detection model when i think about your example of a classification case so not the unsupervised case you're working on it feels a little easier in some sense to me because intuitively if i look at my model after the fact i can test it and say well if it's a loan approval process you'd expect the same rates across different groups you know male versus female left-handed versus right-handed those should not be criteria that help you decide the outcome of a loan so if it shows up we can measure it as you were stating there's no ground truth so we need other criteria what do you lean into to find something useful here so then when we looked at different notions of fairness the first and foremost that comes to our analysis was something called as statistical parity so statistical parity is talking about the model's outcome should be independent of your membership to any group so here this statistical parity as we know it depends on the outcome of the model it doesn't depend on the ground truth so essentially all it says is statistical parity is talking about like i should match the ratios of number of people assigned label one so if i have flagged say 10 people right in total 10 people i have flagged as anomalous and there you have two groups with population eighty percent and twenty percent divide then i should have in those ten flags two from the minority group and eight from the majority group so as to maintain the ratio of outliers so that's what statistical parity is so this notion of fairness doesn't directly depend on the ground truth and which is where we include this notion of fairness into our model yeah i would refer you to the details if you want to look into details to the paper of fair od by us fairness aware outlier detection for more comprehensive understanding of what's going on there yeah we should drop the name of the paper again it's fair od fairness aware outlier detection i'll have links in the show notes great resource enjoyed reading through it and picking up a lot of the ideas you guys have presented here what was the state of the research before you came to it were you and your co-authors the first to take a look at the unsupervised case there was one word which was specifically into this direction but the literature that looked into this direction that was primarily focused on one particular outlier detector and even there what it was doing is there is some notion of disparate treatment so disparate treatment essentially tells you that when the decision time arrives your model will ask you your membership to your group right so then essentially your model will score you differently based on your membership to a group so essentially you are doing a disparate treatment based on your membership so the models that authors have suggested that suffers from this particular problem and if you know like in many of the domains especially in u.s so for example banking loans employment asking for your membership to a group such as your racial information or say income information is illegal i mean it's uh like people are protected under law to not disclose those information so if your model would have disparate treatment then that model would become useless in such domains but there hasn't been any other work apart from this and in this domain as such so this is like ours is like first work in the sense that first work to give you a decider to design your own outlier detector and following our own digital data we design an algorithm outline detection algorithm which is shown to be doing fairly well on synthetic as well as real world applications do you have to have some notion of the context for the group in order to do the statistical parity can i just use columns x y and z or do i have to really do a selection process of what is a potentially protected field or property of the values i have right so typically a protected field is something that is protected by law or something that is personally identifiable right so for example we you have to pre-specify what are your protected fields in general there could be ways to get that from the data but i'm not aware of that typically people have to pre-assign or have to know which field are being used as protected attribute or protected attribute is the attribute which decides your belongingness to a group majority or minority typically race gender income age these fall under this uh predicted attributes that people utilize for handling the notion of fairness in any algorithm there could be cases where you want to apply these techniques and different outliers will be found through the methodology right that's how it's kind of designed the question then becomes you know are these true outliers i don't imagine you'd say oh a bank should just automatically decline uh alone or maybe it should if they do high volumes of them or something but probably a researcher would take this as one consideration in a step or how do you see this fitting into the workflow of different projects and research going on typically the use of outlier detection is for two things one is for attention drawing to suspicious observations the other is like you could place some trust in your algorithm and you could create labeled data for your downstream applications and which you could use then to learn a classifier for flagging out like whether to give out loan or not so in that sense if your outlier detector kind of is being unfair towards a minority group then that would manifest even worse in the classifiers down the line that has few reasons why that happens so one thing is there is a feedback loop associated so like let's take an example in a city if there is a lot of law enforcement officers assigned to certain different areas based on some reasoning and whatever so maybe that comes from our initial outlier detector label so if outlet editor say these regions seem to have high crime rates because these reasons seem outlined and then you will place your law enforcement officers in those areas but then what happens is that as time progresses in subsequent iterations what you will notice is that where you had heavily and deployment of law enforcement officers they will start to report more like more incidents because you have every geographical area is kind of covered with them and there could be a large part of areas where you have very few law enforcement officers where many of these illegal activities may go unnoticed so what happens here is that then the in the subsequent iterations the data that you are generating from the initial deployment then you are basically are biased against a particular geographical area more and more and more so that food feedback loop could be a result of this biased outlier detector such problems may arise if we do not handle fairness in the outlier detection in general [Music] thanks to this week's sponsor linkedin jobs small businesses have always shown an incredible ability to adapt innovate and survive even more so this past year now another way you can adapt and grow is by finding the right people to help grow your business linkedin jobs helps you do that for free they provide you an all in concierge style web interface login post your job which again is free you'll start getting candidates immediately quality candidates we're better a place to learn about them than right there on linkedin messaging connect all in one seamless interface get started by posting your job and reach linkedin's network of 740 million professionals fill out targeted screening questions to get your role in front of the most qualified candidates with the experience skill and motivation you need when you need an exceptional professional to join your team start your search at linkedin.com slash data skeptic your first post is free you've got nothing to lose head over to linkedin.com datasceptic terms and conditions apply well i focused in on most of my questions on this notion of statistical parity because i feel like that's one of the meatiest aspects of the contribution but there are other proposed disarata coming from the paper could we do maybe a quick outline or some deep dives if you're interested on the other properties that you're proposing in the framework i would like to say the statistical parity is not the meatiest of this proposed work because statistical parity has been established earlier as well even i mean given the nature of the problem it's an unsupervised problem so statistical fairly fits this problem that we have at hand which is outlier detection but then even statistical parity suffers from a notion that we call in our paper as laziness which is or like it has been referred earlier also in literature as laziness where what happens is that statistical parity all it does is it cares about matching the fractions of your flag observations to their protected groups right so it could do so in such a way that it would flag say actual outliers from minority groups right and when it comes to majority groups you could just randomly sample some observations and say okay these are outliers so in that sense you would always be able to match the fractions but then you will not be actually flagging the real outliers from majority group in that sense statistical parity suffers from that problem which is why we require some additional constraints to this problem and which is where we introduce our decision data so our tested data is basically a five tuple where the first thing is like for any outlier detector it should be effective at detecting right so this destinator has nothing to do with fairness it's just talking about the effectiveness of an outlier detector so what do we mean by that is basically what you're saying is like the precision of the detector should be better than random right so essentially if i flag say 10 examples 10 observations as outliers probability that these flagged outliers are true outliers that should be bet greater than your base rate so base rate here is basically the proportion of your data which is outlier so if your proportion of data is like point one so if your point one percent of the data is outlier then your precision should be larger than that then only it makes sense to do outlier detection otherwise you would just randomly sample some people and then you will find who would represent as outliers and so on right so this is just talking about the effectiveness yeah necessary but not sufficient so what else right yeah next thing is we also require treatment parity so treatment priorities as i explained earlier that for in the domains like loans and employment one should not be asked to furnish their but protected attribute group membership the outlet detector should exhibit non-disparate treatment in the sense that the outcome that outlier detector gives you whether you furnish your membership or whether you're not financial membership the output should be the same so essentially it does not require a person or an observation to reveal its membership to protected group so that's another criteria that must be satisfied in outlier detection then we think about statistical parity so given that it's effective it has treatment parity then we talk about statistical parity which i think we have already covered in in detail to some extent like where what we want is that flag rate across two groups should be same so essentially we are flagging whenever i say flag basically flagging means that i am saying this observation is outlier so that is flagging so assigning a label one or assigning a label as anomaly is flagging so equal proportions from majority and minority groups should be flagged that's what the statistical parity is talking about but it suffers from this issue of randomly flagging some people from majority group but from minority group you flag through outliers in that sense your model is always biased against minority group in some sense so given this what we need to do is that to deal with this issue of statistical parity is laziness in the sense where majority group could be randomly sampled what we need to introduce something we call as group fidelity which is a property where we want that true outlier should be equally likely to be assigned higher scores regardless of their membership to the group that the points belong to as you see from this definition it immediately tries to address the shortcoming of statistical parity by introducing this notion that true outliers should be assigned higher scores of course we don't know what true outliers are so from this definition it seems that okay we need to have ground growth available so typically this definition is also known as equality of opportunity in uh supervised fairness problems but we do not have available labels associated with that so in this paper what we do is we propose an approximation to group fidelity where what we say that instead of looking at ground truth levels since outlier detection is a ranking problem what we say that we should be able to preserve the group level ranks from the base model so base model so here base model would be a model which does not incorporate any fairness criteria so what we are saying is a base model would typically be effective at detection but it discards any notion of any group membership and so on so it may be biased against minority group and so on but what we hypothesize is that if we look within group then even the base model within group would typically rate the more outlying points as uh assign more outlying points as high scores within each group so we try to preserve this notion that we have for each from the base detector into this criterion so essentially what we're saying is we should be the fair detector should preserve this group rank order so that way what we will do is even for the majority group we should be able to flag only those individuals who should be higher in the rank for the within their group and essentially giving rise to a rank list which is fair enough or fairer or approaching fairness the other thing is like one decidum that we propose but we cannot enforce or we could not enforce was something like base rate preservation so what we are saying is that so base rate is like your probability of being outlier given you belong to majority group so what we say that in the fair outline direction we should be able to preserve this base rate in the flag the population that we have so the flag population should be representative of true population and that we have the flag population are the individuals that we flag as being outlier or not being out at least flag for doing further policing or further investigation that should be representative of the true population because down the stream classifier should not be biased in the sense that they should learn from a true sample that represents true population but as we show we are various proofs that base rate preservation is not possible at least was not achievable within our limits so that is something we cannot maybe preserve at this point but at least i mean achieve statistical parity and we can address the shortcomings of statistical parity by introducing this notion of group fidelity so the meat of the work in this sense is the combination of statistical parity with group fidelity and and the other thing is basically this display item for designing any outlet detection now further down the road like if anybody wants to design a fair outlier detector they can get up this this data and they can try to ensure that each one of these designer item is being enforced in their detected detection algorithm and these different properties even though it may not be obvious from our audio conversation where they're intuitive to the mind they also have great statistical forms you can translate these into their mathematical versions which means we can use them in machine learning in some way can you talk a little bit about the experiments you guys did and how this is all implemented each of these properties that i mentioned each of these properties are essentially has a statistical identity associated with that which i think i will refer our listeners to read the papers that way it would be much better now once we established the notions of fairness and the criterions that a fair outlet direction should include then what we do is to demonstrate the effectiveness of this we start with a very simple outlier detector based on like reconstruction based outlier detector which is what we do is we use a deep auto encoder so auto encoder is you want to reconstruct the input features from a reduced dimension representation essentially so if you are not able to reconstruct back a certain example maybe that example deviates from the normal and then in that sense you could flag those examples as outliers so what we do is we start with very simple or deep auto encoder as our base outlier detector which does not include any notion of fairness and then we want to introduce notion of fairness into such outlier detector the reason for choosing this outlet was basically it is easy to optimize you could it's not easy to upgrade i mean to say that it is it can be optimized to some extent and any optimizable outline detection framework could work with this because the notions of fairness that we introduced they are statistical quantities which can be turned into loss functions and if you have any outlier detection algorithm which has a defined loss function then all you need to do is you need to include uh the notions of fairness the loss associated with those to your existing outlier direction laws to learn your model or learn the parameters of your model gotcha so fair od is not just like a way to score or give a report card this is like a regularization technique too is that right yes so it involves to some extent of regularization so it takes a base loss function and it introduces regularization with respect to the fairness constraints that we talked about gotcha so we want a result that's not only useful but also fair makes sense yes and let's get into some of the experiments what sort of data set and test can you run to evaluate the quality of these results to first of all like when we do experiments in such new things we don't really have benchmark data sets like where we just evaluate a given metric and then you're done with that right so here to demonstrate the effectiveness we started with a very simple synthetic data two-dimensional data where what we had is like we had features like features like two dimensions such a way that you have a feature which is highly correlated with true outlier-ness so in this since it's synthetic i will also have ground truth labels to evaluate the final result of the model so here we had like two features in one case it's highly correlated with the output which is like ground truth outlier and other case ah the other feature is independent of the output and then we have another data set synthetic with a two dimensional where we have features which are partially correlated with both the thing which is group membership as well as the ground truth outlawedness and the other one it was like one feature determines your group directly coded with a group one features directly determines your ground truth outlier nestle label although you don't know that right your algorithm doesn't know that to compare our experiments the first thing we need to establish are like on what criterion do we want to evaluate our models so the first thing is since we are talking about fairness we define fairness as basically the flag rate ratio probability of being flagged given that you belong to a certain so say majority group and the flag rate ratio is the ratio of majority to minority of this probability right so we take that as fairness measure and as you can see that for a fair outlier detector this ratio should be close to one the other thing is as i mentioned like so this basically captures your statistical parity that we talked about right if your model satisfies the statistical parity then your fairness score should be close to one the other thing we talked about was the issue with the statistical parity was that it doesn't can randomly flag some people from majority group right so in that case to measure that what we do is we create an evaluation metric which is called as group fidelity essentially as the way we designed it right so we are talking about like the group ranking should be preserved so essentially for any ranking evaluate any ranking you could use ndcg scores right so we are looking at basically that ndc score for majority group ndcg score for the minority group and we take harmonic mean of those to kind of compute this group fidelity so harmonic mean typically tilts toward the lower number between the two so if you have high group fidelity that means that you are preserving the base ranking and so on right so these are like the primary evaluation metric that we used and then we tested our algorithm on like four data sets i think these are like fairly popular datasets these days like adult dataset great default data these two are fairly popular in fairness related papers the other two data research that i used was one was abusive to it so i compute collected tweets from groups with a language as african american or mainstream the african-american dialect is encoding your racial dialect and the goal was to find out outliers so typically your outliers if you do not enforce fairness then what happens is that typically you will flag out most of the african-american tweets as outliers because they do not contain so they are minority number in first place and the vocabulary used is different from the mainstream vocabulary so how do you introduce fairness into such problems and so so these are data sets that we used and then we compared our models with the base model itself and we repost few of the pre-processing unsupervised fair representation learning methods as for doing outline fractions where we use those learned representations and we ran normal outlier deduction on those learned representation to see if on a learned representation do we get a detector which could be fair which passes on fairness from the occupied learning to the outlier detection and so on and then typically what we observe is that on fairness access we would be across data sets like including synthetic and the real world research we typically would be fairly close to fairness value equal to one which essentially tells that our method achieves statistical parity when we compare with preprocessing based methods although pre-processing based methods are also designed to honor statistical parity but we did not learn the representation keeping detection in mind so detection is a separate process from the learning representation part and when we do a detection on this learned representation within what we notice is that the statistical parity was not achieved on those methods like we have different types of methods over there like reinforcement learning based methods waste methods and so on but then they typically would not achieve statistical parity because these are these methods would try to enforce not enforce try to incorporate fairness independent of detection process which is uh so in our method since we introduced fairness notion as a integrated method we typically outperform most of the competitors on the fairness task and so on so other task was like looking at our top k rank agreement and so on so we observe that we also like notice that the fair outlier detector what you would expect so you would expect that you have a ranking from ranked list from an effective outline detection all you need to do is play around with this ranking in such a way that your top k ranking becomes fair or fair in the sense that it obeys certain notions of fairness that you have introduced so your rank list should not be drastically different it will just be some other permutation of this ranked list right your affair rank yes so in that sense uh we also as i mentioned like a score called group fidelity which measures like how well are we preserving the group rankings and again we based on our experiments we showed that here i think our method will be little biased in the sense that our method has a component to preserve that ranking so when we measure the ndcg we typically outperform all other methods because none of the methods are preserving the base ranking that a base model would learn on a given data set so so essentially what we show in our data set is like so in terms of notions of fairness which is group fidelity and statistical parity we achieve those and we are doing much better than competitors and when it comes to effectiveness we measure the effectiveness our method with respect to baseline base method because base is kind of optimized for doing detection and there we also notice that the performance like in terms of a average precision or precision doesn't deviate much from the base model right so or it deviates least from the base model compared to any other competitor so here competitors are not true computer they are like pre-processing based methods as i mentioned earlier so in that sense what we show is that by following our designer data and by incorporating the notions of fairness that we have introduced we are able to achieve a very nice balance between fairness as well as performance in fact one thing to note here is that in many of the machine learning papers you would notice that with the notion of fairness introduced you sometimes have to trade off performance but in the case of outlier detection what we observe is like that's not always true in fact that's not in most cases what you see that we improve the performance when we incorporate fairness and the reason for that as i mentioned earlier was because we enforce that the top ranks from the majority groups should be aligned with the true outlier-ness and hence what we do is when we introduce fairness we improve the top key ranking as well when we introduce fairness with our group fidelity as well as our statistical parity what we have is we refine our top k rank so as to have a better performance sometimes that's an interesting thing that you could take a look at in the paper as well yeah that's interesting it's almost like a free lunch then yeah right yeah so in some sense yes uh because as i said like when you introduce fairness you push down the false positives yeah i think that's even if for someone not that there are people out there who don't care about fairness but even if that's not your priority you're getting a net benefit in this case then which is interesting yes that's true this will not always be the case because depending on like how your top rank list is right but there may be cases where you will push down the false positives so in one or two data sets we observe that thing and that's where we have a better performance over there just to clarify that it's not always the case that's what i wanted to say what's next for you in your research is this a step in one direction or is this just part of your overall research interests it's part of my overall research interest because my research interests are mainly around focused around anomaly detection for policy problems or social problems in general so this fits in there like i typically work in medical as well as insurance domains and the problems over there are of interest to society at large so this particular fairness based outlier detection also fits into that a story where you want to have a fair outline detector for doing insurance fraud claim and so finding fraudulent insurance claims and so on that kind of goes into the overall picture where my phd has been focused on anomaly detection for societal problems this definitely forms a big part of that absolutely where can people follow you online best way would be my website if you just search my name i should get there i'm slowly getting active on twitter so you can look me up on twitter awesome i'll have a link to both in the show notes well she brought you this was really great thank you so much for taking the time to come on and share your expertise thank you so much kyle for inviting me here and i i'm really happy to share my thoughts and i'll also be happy to help you with any follow-up questions that your listeners may have thanks [Music] you