Measuring Fairness in Machine Learning Systems

Key Takeaways

hi my name is Melissa Hall and I work on a team whose mission is to ensure the responsible use of AI at Facebook at Facebook we believe that it's essential to evaluate machine learning nai systems to ensure that they are not biased against users this presentation gives an overview of algorithmic fairness including an explanation of the intricacies within large-scale systems that make them vulnerable to biases and the multiple ways of measuring fairness I'll focus is talk on developing ethical AI systems with a focus on safety and equity for users I'll begin with a bird's eye view of machine learning systems and discuss the ways that fairness concerns could manifest throughout the development process I'll share a commonly cited fairness case study to highlight biases in machine learning systems I'll then give an overview of the multiple conflicting methods of measuring bias and models I'll conclude by sharing additional points of consideration when building machine learning systems that have human impact to begin before we can understand whether a system is fair it's important to have a clear sense of what's at stake fairness is not just an engineering problem but it is crucial for engineers to think about the ways their products interact with society when someone interacts with a given system what are they getting out of it what are the benefits it's also important to understand the risks and in particular a few kinds of potential harms consider does the system restrict access to resources or opportunities for certain subgroups does the system reinforce harmful stereotypes does it perform reasonably well for everyone and beyond the impact to individual end-users it's also important to ask what kinds of unintended impacts might this system have on society as a whole fairness is a process as I said before not just an engineering problem it requires different kinds of expertise as well as input from a range of stakeholders if we zoom out this is what the process looks like first it's important to understand the product goals policy and implementation within the broader social texts where the system will be used next hold conversations with stakeholders to reach consensus to outline measurement and mitigation plans how is the system performing and why does it perform that way to measure fairness with robustness it's important to look at different points in the process of developing a machine learning system started the process of defining the policy ground truth with the intention of understanding whether this policy has been designed such that specific groups are systematically favored or disfavored in the process of collecting labels the developer needs to look at how the human labeling along the policy lines matches the actual content it's important to understand whether the human labeling program is robust against labelers who might be inclined to introduce their own social or ideological biases then of course there's a step of an understanding whether the AI systems introduce fairness concerns finally it's important to examine the intervention step I'll focus this talk on the ways of measuring fairness and the algorithmic step of the workflow but there are a lot of metrics that people have proposed for measuring fairness and it can get pretty contentious let's talk about an example a commonly cited case study I Pro Publica researchers and 2016 focused on the compass recidivism risk assessment which is developed by a company called North Point this software is used by US courts to determine the likelihood that a defendant recidivate meaning the likelihood that a defendant relapses into behavior that led them to court in the first place at the time of the study defendants would complete a compass questionnaire that would then be fed into the compass software to determine prediction scores for things like the risk of recidivism and risk of violent recidivism which were later surfaced to judges Republicans analysis found evidence of bias against black defendants in the compass system specifically they claim that among defendants who did not go on to reoffended pendants were often predicted to be at a higher risk of recidivism than they actually were all white defendants were often predicted to be less risky than they were according to Republican work black defendants who did not reoffended likely as their white counterparts to be misclassified is a higher risk this demonstrates the concern for equality of outcome Northpoint the company that created the compass assessment responded Republicans concerns by claiming that for any given score on its ten-point scale why am black defendants were just as likely to reoffend as each other they're essentially saying that their system demonstrated a quality of treat there a variety of metrics used to measure fairness and bias in machine learning systems in this case república used the false positive rate across black and white defendants to determine that they compass assessment was unfair while Northpoint used calibration measurements to claim that this system was there it's worth also pointing out that the data they trained on may embed biases the likelihood of recidivism in the training data was possibly the outcome of a biased justice system and our framework that would be considered label bias but for the rest of this presentation we'll be focusing on the question of model bias and assuming that we're discussing systems with correct labels across academia industry there are multiple and oftentimes conflicting notions of fairness for example one fairness goal could be a quality of treatment where similar samples from two different groups are treated the same on the other hand a fairness goal could be to ensure equality of outcome which means that samples from different groups have the same likelihood of a certain outcome this might require an equal treatment since oftentimes their external confounding factors such as historical biases that mean correcting for there are a few ways we could compare how a model performs for different groups the first is demographic parity meaning that the proportion of positive decisions should be the same across all groups when applied to our pretrial example demographic parity means that the rate of labeling a defendant as high-risk is equal across both black and white defendants however when the true underlying distribution of risk varies across groups differences in group level error rates can happen when algorithms accurately captured each individual's risk attempts to adjust for these differences often require implicitly or explicitly misclassifying low-risk members of one group as high risk and high risk members of another is lowest potentially harming members of all groups in the process another metric is a concept of equal opportunity which focuses on the advantaged outcome this would mean that the compass assessment would correctly classify defendants who don't really risk at equal rates for both black and white defendants in other settings in which the advantage outcome is a positive classification they'd need to be equal to positive rates there's also equalized odds which is a stricter fairness condition than equal opportunity in equalized odds for the compass assessment both the false negative rates and the true negative rate should be equal across groups this punishes models that perform only well in the majority like with equal opportunity the choice to require equality of false and true positive versus negative rates is dependent on the advantaged classification in your setting note that a drawback of equalised on to equal opportunity but that it's possible to gain these rates by introducing more examples or spores are extreme further warm equalized odds is usually possible by introducing randomness into the decision-making procedure finally there's calibration calibration means that regardless of the group that a sample is in the models predicted probability for the sample represents the true probability of occurrence in the context of the compass risk assessment calibration means that among the defendants with a given risks for the proportion that would be offended for at least is the same for both groups while calibration is generally desirable it has been shown to prove only a weak provide only a weak guarantee of equity in particular it is often straightforward to satisfy calibration also teaches we miss classifying individuals who order to discriminate now of course it would be great if we could simultaneously satisfy all fairness definitions however this is only possible under certain highly unlikely circumstances like when there's a perfect predictor or equivalent base rate across subgroups therefore it's important for developers and organizations to understand the different ways to measure fairness and the inherent trade-offs they have the mathematical conceptions of parents are grounded in contextual and philosophical perspectives that are often unique to specific individuals it's important to consider these perspectives among builders and stakeholders when establishing definitions and metrics for measuring fairness some questions to think about fairness could include what is a person's source of value from the decision but can we expect a system to guarantee for its users what groups matter when measuring and ensuring the fairness of a system what information about systematic differences between groups should be available to the model and what should be excluded what privacy trade-off do these decisions imply sometimes and may be more useful to revisit what the model is designed to do in other words to interrogate the target metric product goals and policy it's a system something that should exist at all and she can see measuring and evaluating fairness even at just the model stage it's not an easy task fairness is a process that requires systematic thinking and in every step of the implementation of the system we must surface two risks and hard questions of fairness processes need to be built for resolving these questions and a record of the decisions should be maintained along the way for this hackathon we ask you to build tools that make these evaluations easy but also not oversimplifying the complexity of fairness thank you