Algorithmic Fairness

Key Takeaways

if we collected up all the algorithms in the world we're not just that what if somehow we enumerated every possible algorithm there was could we go through them all and label those that are interpretable and those that aren't well that's sort of a matter of debate understanding a model doesn't just require machine learning knowledge it might also require some specialized knowledge about the data center the domain not always but sometimes and although we'll cover many different definitions of interpretability during this season and explore various angles on what makes something interpretable or uninterpretable one of my major takeaways is that it's a property that's personalized something might be interpreted for me and not for you or vice versa but what about ethical could we numerate all the algorithms again and go through one by one labeling those that are ethical and those that are not binary search seems pretty ethical to me although what if binary search is employed is one step in some sort of virus well again that's a matter of a hammer can pound a nail or pound a finger the ethics transcend the tool but those algorithms that make critical decisions that affect people certainly we'd like them to have the property of being ethical interpretability does seem to be linked a little bit with algorithmic fairness in that if an algorithm is truly interpretable one can inspect it and perhaps find biases or the absence of biases towards saying that something's ethical or not but it did occur to me recently that to be ethical and to be fair doesn't necessarily require interpretability what if there were some impenetrable black box which through mechanisms we couldn't completely understand it happened to make quite a cycle and socially aware choices there are some other ways we can look at this perhaps you know we have no knowledge of an algorithm or what goes on inside of it but we get certain guarantees about its fairness as you can imagine there's many techniques of anonymization we'll be covering those the season of course but there's also the opportunity for some more fundamental or theoretical guarantees kind of the way encryption relies on certain guarantees that computer science makes one of the many topics I'll cover today is differential privacy perhaps using the right algorithms and mathematical techniques and things like that there are ways we can in sure fairness and ensure privacy at least as long as they're big directly into our design welcome to data skeptic interpretability podcast about algorithmic fairness privacy and of course machine learning this week on the show we've got an interview with Aaron Roth Aaron is a professor at the University of Pennsylvania in the department of computer and information science he's also a co-author of the book the ethical algorithm he and I discussed socially responsible algorithms differential privacy and what promises tools like differential privacy can give us all that more right after the break my name is Rita Fallon I am an analyst with governor I specialize in data and analytics I caught up with Rita recently to discuss the upcoming conference in Grapevine Texas the Gartner data and analytics summit taking place March 23 through 26 2020 we expect over 4,500 of the world leading data and analytics leaders can you tell us about some of the programs and unique events at the conference one big part of the program is the peer networking experience and so we have a lot of facilitated sessions where we encourage leaders to engage with each other and share best practices we also have a unique experience that helped companies who are looking for a hands-on experience to assess different vendor software you know you get sort of insight from Gartner which is vendor neutral of course we have the analytics and bi bake-off as well as the data science and machine learning bake-off to find out more about the Gartner data and analytics summit visit gardenerd.com slash us slash data more details on the full conference schedule and how to register that's Gartner comm GA RT and ER dot-com slash us slash data my name is Aaron Roth and I'm a professor of computer science at the University of Pennsylvania my background is in theoretical computer science which means that I deal in theorems and proofs and definitions more than I deal in code and data but my particular interests recently have been in the ways in which machine learning interacts with social norms so things like privacy and fairness and the way in which machine learning interacts with incentives how the algorithms you deploy can feed back into the ways in which people interact with algorithms in some ways it seems that algorithmic fairness is in a very disparate place than these more fundamental theoretical ideas or do you see them as perhaps more connected I think they're much more connected so of course theoretical computer science is quite broad but what theoretical computer science takes seriously our definitions and their implications we've seen how useful this can be in the development of cryptography and the development of the foundations of machine learning and especially when the definitions are so important to get right and this is the case when we're talking about important things like privacy and fairness I think the theoretical approach in which you first think very hard about what you actually mean when you say that you want something to be private or you want something to be fair and then carefully think through what the implications of imposing these constraints are you know it's here where the theoretical approach is especially valuable privacy is one of those things that I feel like everyone sort of intuitively thinks they understand and obviously are for it but coming down to a rigorous definition that's not always the easiest thing are there ways which you can define what privacy is yeah that's great and maybe it's useful if I first go through some ways in which you might incorrectly think about how to define privacy before finally getting to what I think has become a relatively widely accepted definition which is differential privacy so for a long time people didn't think hard about what privacy should mean and they thought about privacy basically through the lens of anonymization or D identification so for example if I wanted to release a medical data set to the world what I might have done 20 years ago is I might have scrubbed names from the data set if I was careful maybe I also scrubbed other unique identifiers like Social Security numbers and then I would have declared the data set to be anonymized and just published it to the world and this approach is fundamentally broken and we've known this for a long time basically because data sets can be cross-referenced with other information that's out there and a surprisingly small number of idiosyncratic facts about yourself tend to be enough to uniquely identify you so maybe the first time this was convincingly demonstrated was in 1997 when the state of Massachusetts released supposedly anonymized a set of medical records for every state employee publicly to the world and this was done with the best of intentions to aid in medical research but Latanya Sweeney who was a PhD student at MIT at the time and is now a professor at Harvard figured out that although there were no names in this data set and the the information that was in this data set like zip code and age and gender alone weren't enough to uniquely identify anyone in combination they were and so she cross-referenced this data set with the voter registration polls in Cambridge Massachusetts and was able to identify the records of bill weld who was the governor at the time and sent them to his desk and so anonymization basically doesn't work and it was things like this that forced people to try to think carefully about what they actually might mean by privacy and so maybe here's one attempt that is too strong but will sort of lead us towards the right definition so imagine that I wanted to define privacy as the following maybe some analysis of a data set is privacy-preserving if after the analysis is done you know nothing new about me compared to before the analysis was done and that sounds like a pretty good strong definition of privacy if we could accomplish this then surely this would be privacy preserving but it's too strong in the following way so imagine that you are running some medical study maybe including my data and you're trying to figure out whether smoking and lung cancer have any correlation to one another and of course if you analyze the data you find that smoking and lung cancer are correlated and I'm some smoker out there everyone knows I'm a smoker I don't try to hide it and after this study is complete people now know something new about me that they didn't know before in particular that I am at higher risk for lung cancer and this might have concrete harms for me for example it might cause my health insurance rates to go up but if we wanted to call this a violation of privacy there would be two things that really went wrong here so first we wouldn't be able to conduct any scientific studies at all because I could tell a similar story with any possible correlation that you might discover in the data and so asking for this as a definition of privacy would just rule out all data analysis but maybe more disturbingly the story would play out in exactly the same way even if my data had not been included in the data used for the medical study because the thing that caused you to learn something new about me was not because you learned something idiosyncratic to my data it was because you learned some fact about the world that smoking and lung cancer were correlated and that just wasn't my secret to keep and so differential privacy is a very similar definition but with a little tiny twist that makes it realizable so differential privacy asks you to imagine a world in which the same study is carried out but without my data used anyway and it asks you to presume that if my data wasn't used at all then this study shouldn't be said to be a violation of my privacy that seems rather obvious yes it seems rather obvious so let's call this the ideal world now in the real world of course my data is used but what differential privacy asks for is that there should be no way for anyone to tell substantially better than random guessing whether the study was conducted in the ideal world or whether it was conducted in the real world and if there's no way for people to tell substantially better than random guessing whether we were in the ideal world in which we agreed that I have no privacy violation at all and whether we're in the real world then maybe we shouldn't think of the study that's conducted in the real world as substantially violating my privacy and that's differential privacy in a nutshell so I imagine it would be obvious that these techniques are very appealing to me and I love the idea of linking my privacy with problems that are like provably complex in the way encryption is pretty provably hard to solve and I don't want to put words in your mouth but I imagine you know as fellow computer scientists we can agree that hard problems do exist so it's a nice idea let's hitch our wagon to one of those hard problems and say in order to violate my privacy you have to solve a hard problem cool perhaps I have a little nagging fear that an especially clever person will come around and find some really novel technique to exploit some bit of information and reverse my privacy in a way no one expected how concerned should I be yeah well that's exactly the power of theoretical computer science so the old-fashioned way of thinking about privacy this approach of anonymization that was entirely heuristic and there your your concern is exactly right that I might think I'm being really clever and there's no way to reify anybody but tomorrow someone even more clever might come around and prove me wrong and this is actually how data privacy played out for four decades and it was a losing game for the people trying to provide privacy but differential privacy starts with this definition of what privacy means and once you have a definition you can design algorithms for which you can mathematically prove that they satisfy differential privacy so it's this very strong guarantee you no longer have to worry about how clever this hacker is gonna be who's gonna come around tomorrow because you've already proven that no matter how clever he is there's just no way for him to tell whether your data was used or not I see some interesting parallels here with encryption the world does seem to have accepted encryption by and large and SSL even if they don't know what SSL or RSA means encryption is linked to a computational property we're very sure that breaking encryption would be a hard problem and the world seems to accept that you know footnote for some things about quantum computing but if we take for granted that new encryption techniques will emerge will we be able to achieve a similar level of trust in interpretability methods amongst the general public that they say yeah you've got it that we understand the black box yeah that's a great question so I think part of it will just be familiarity with the techniques because as you say encryption is also a mathematical topic but I think we've gotten to the point where people trust encryption even though they don't necessarily understand the nuts and bolts of what works in part because encryption has become a widely deployed technology that you use every day when you for example buy things on the internet and we're not there yet with differential privacy but we're getting there so for many years differential privacy was a purely theoretical topic people like me would write mathematical papers for the small community of other people like me and these papers would basically just prove theorems and think about trade-offs and algorithms but in the last I'd say four or five years differential privacy has become an actual real technology that started to be widely deployed so for example if you have an iPhone in your pocket right now it's reporting usage that is to expect to the mothership in Cupertino using the protections of differential privacy Google Chrome does similar things and the real moonshot for differential privacy is going to come just next year when the 2020 US census is conducted because the census has committed to releasing all statistical products that result from the 2020 census using the protections of differential privacy so I think that although it's a new technology as it becomes more broadly adopted it will gain consumer trust in the same way that encryption has even though as a mathematical topic it might continue to be the kind of thing that only experts really understand the details of again just like encryption is so I feel pretty confident that I could make a pitch to a friend a non-technical friend over dinner or drinks as to why they should trust encryption could you convince them that differential privacy is a trustworthy technique I think the way to think about differential privacy is as a form of plausible deniability what it means is anyone who looks at the outcome of some data analysis tasks some deployed machine learning algorithm some private data release and thinks they've concluded something about you can reasonably be told that they are wrong and that you didn't participate in the data set at all or you did and your data was different there are sort of simple examples of algorithms that achieve differential privacy that make this more transparent that we can talk about if you want so maybe this is the sort of simplest example of how you would actually achieve differential privacy which otherwise might seem you know a bit exotic so suppose I wanted to conduct some poll of the residents of Philadelphia and ask them something embarrassing maybe I want to know how many people in Philadelphia have had an affair so one way I could conduct that poll the standard way is I could call up people on the phone I'd get a representative sample and I would just ask them you know have you had an affair in your marriage and I would write down the answer maybe in an Excel spreadsheet and in the end I'd try to compute some statistics the average may be some confidence intervals and be done with it now I've never been to Philadelphia are these Philadelphians especially known for honesty exactly this is the problem so as you correctly noticed people might be reluctant to answer this question because it's compromising information and even if they personally trust me they might worry that this Excel spreadsheet that I'm compiling with this very sensitive question about them will get into the wrong hand yeah you could imagine that it's lost or stolen or that it's subpoenaed and a divorce proceedings or something and so they might have really good reasons to not want to answer my question honestly but here's another way I could conduct the poll I could call people up again and I could say have you ever had an affair but then I'd say wait wait wait don't tell me the answer just yet what I want you to do is I want you to flip a coin okay don't tell me how it comes up if the coin comes up heads I want you to answer my question truthfully but if the coin comes up tails I want you to just give me a random answer flip the coin again and tell me yes if it's heads and tails if it's no so I tell people to answer in this randomized way and I can still write down their answers now people have a very strong form of plausible deniability I suppose in a divorce proceedings that my records are subpoenaed and the lawyer says well says right here in the Excel spreadsheet that you answered the question have you ever had an affair yes well you can now quite plausibly say that indeed you answered the question that way but it wasn't because you'd had an affair it was because that's what the coin flip told you to do that might well have been the case so if I get the data in this way then every single person in the data set has a very strong form of plausible deniability and I as the researcher don't have any strong signal about the answer that any particular person really should have provided but that's okay because my interest in this data set in any case wasn't in the behaviors of any particular person in my sample my interest was in the aggregate what was the rate at which affairs are conducted in Philadelphia and it turns out that even though each individual response is very noisy if I only want to know this aggregate then I can still calculate it extremely accurately because I know the noise distribution and when I average over a large number of people I can subtract it out this is sort of a version of the law of large numbers and so this lets me get at statistical facts that I wanted to learn without learning very much about individual people and this is one way to get differential privacy in a way that I think makes this guarantee of plausible deniability sort of visceral and easily understandable so the coin toss idea is really interesting and kind of especially dramatic for me in the sense that there's a 50/50 chance here so the privacy enthusiast in me really applauds that it seems like that gives us a good benchmark of protection but the statistician in me is a little bit more reserved here and worried about kind of like a noise threshold could there be very rare conditions or data points you know like some medical condition that we would really like to sample it with a lot of precision because we don't want to under serve that person but they're gonna get washed away in this fifty-fifty noise threshold ultimately my question is what do you suppose an information theorist might have to say about all this yeah for sure and one point that we make over and over again in our book is that none of these things like privacy or fairness that we might want are gonna come for free so differential privacy by design makes it impossible to determine things about a data set that are really facts about just one person and by extension it makes it harder to tease out facts that depend on just a small number of people and so it is true that just about any statistical data analysis task and when I say statistical I mean any kind of data analysis task whose right answers are a function of the distribution not of particular individuals any such tasks can be conducted with the protections of differential privacy but it comes at a cost and the cost is usually that you need more data to carry out the same analysis to the same degree of accuracy with privacy protections than you would need without privacy protections so you'd mentioned that the 2020 census is going to employ some of these differential privacy techniques could you elaborate a bit on what's gonna be changing there that's right so 2020 will be the first year in which this is done and it's exciting we'll see how it works out but the census collects lots of information about every single American citizen and some non-citizens that's actually one of the sensitive points and it releases literally billions of numbers tables of statistics that contain billions of summary statistics about the American population and it's required by law to protect the privacy of American citizens but the law doesn't say exactly what that means so in previous decades census used basically you know heuristic ad hoc techniques to try to offer some form of privacy without spelling out exactly what privacy means they tried to basically randomly swap people between neighborhoods before they computed these statistics but it turns out maybe unsurprisingly that these heuristic techniques don't actually work very well and John a bud who's the chief scientist at the census right now led a study that showed that actually using database reconstruction attacks that had appeared in the differential privacy literature on these heuristic anonymized tables of census statistics you could actually reconstruct a large fraction of the raw data that the census collected that these methods were intended to hide and so he decided I think quite reasonably so that these heuristic techniques weren't actually compatible with the censuses mandate to protect privacy and it's in the midst of deciding exactly how to compute these statistics with the guarantees of differential privacy which will involve perturbation with noise when I say it's in midst of deciding exactly how to do it I don't mean that they're still trying to come up with the algorithms the algorithms you know we know how to do these things algorithmically but there's a policy decision to be made because differential privacy is a definition that comes with a parameter okay remember what it promises is that nobody can tell the difference between the ideal world in which your data is not included in the data set and the real world substantially better than random guessing so what is this word substantially mean well you can quantify that and you can attach a number to it and this gives you a knob that lets you in a quantifiable way trade-off privacy you can ask for more privacy or less privacy with accuracy of the statistics in general if you ask for more privacy you're gonna get less accuracy and vice-versa so I love the framework of a knob that's you know something that someone can control I'm wondering who should control it is this something that requires an understanding a machine learning and some sort of data scientist should be tuning it like a hyper parameter or maybe somebody who has a history at the census who has some context for it or I know maybe it can be a more general setting but it feels like a hyper parameter to me how do you envision it being tuned well I think it's less of a hyper parameter and more of a policy decision because there is the algorithmic optimization question once you fix a particular privacy parameter the question is how do you actually design algorithms that satisfy that level of privacy while at the same time being as accurate as possible so that question is definitely an algorithmic question that mathematicians and algorithm designers and statisticians should be working on but once that question is solved there remains the policy question of how we should weigh privacy protections with the accuracy of our statistical products both privacy and accuracy of statistical products are different kinds of Goods and different stakeholders care about them and at some level they are sort of fundamentally at odds with one another right you cannot ask for more accurate statistics about more quantities without eventually giving up on privacy so that's not a mathematical question to figure out how we want as a society to trade-off these different things that's a policy question that different stakeholders will have different positions on so somebody needs to make the decision I don't know exactly who should be making the decision but it's a policy question not a mathematical question yeah that's very interesting I'm reminded of a thought experiment I've had off and on here what if I found out there was a cache of medical records released to the public but then what if in the next sentence I learned that it was from ancient Roman citizens I somehow I don't have the same sympathy for them I'm not exactly sure where we have the half-life of privacy here but do you have any thoughts on where we draw those lines you know I don't know whether my thoughts are particularly better than anyone else's thoughts on that question but I do think it's reasonable that we should have fewer privacy demands for the data about ancient Roman citizens and I think the census also takes this position so there is exact data that's been made available from I believe the 1940 census of course there's people from 1940 census who might still be alive even but you know at some point I think it's reasonable to decide that data is old enough and maybe the people who for whom that data was sensitive have now passed away that it can be made available and data of that sort is quite valuable because it lets you design these algorithms when you're in the process of designing and tuning an algorithm it's very helpful to have access to the data of the same sort that you're going to ultimately need to run the algorithm on and that can actually be quite difficult if your only source of data requires privacy protection it's because it would sort of mean that the actual algorithm design process itself could only access the data in a privacy preserving way so I think it's reasonable and valuable to think of privacy as something that might be a right to that degrades with time maybe you know the older the data is the less expectation we have for privacy especially when this degradation happens not at the scale of weeks or years but of lifetimes so computer science people don't swear any sort of Hippocratic oath there's no Turing's oath and I think that's right people choose to go work on you know the next version of pac-man there's really no ethical concerns there whatsoever I imagine if I looked into it someone working on like really deep Rd a Pharma stuff they might actually end up swearing the hypocrite Goeth but in general for machine learning people there is no such oath in the absence of one do you have any best practices or things you think that a practitioner needs to bring to the table to what degree is that technician need to be an advocate for these sorts of techniques in their organization yeah so that's a good question and as you say there's a wide range of roles but I think we are at the point where ordinary software engineers are making what are in effect important policy decisions without even knowing it right if you are someone who's in charge of helping to design for example the Facebook newsfeed then the minor changes that you make in principle can have large-scale effects on for example the nature of our political discourse so I do think that it's important that people who are working on product that that affect many many people be cognizant of the unintended side effects that their choices might make and I think one starting point is just awareness that there is an emerging science and set of technologies designed to both think about what those unintended side effects might be and provide concrete mitigations for them so we've been talking about differential privacy people should know that differential privacy is out there another thing that's been getting lots of press lately has been algorithmic unfairness I think it would be good for engineers to be both aware of the potential for out-of-the-box standard machine learning techniques to result in outcomes that seem unfair in various ways and that there's a large community of people and a growing body of science that leads to concrete interventions that can be used to prevent some of these harms when thinking about what it means for an algorithm to be unfair at some level I start to wonder well you know we can express those algorithms in terms of Turing machines or assembly code or some higher-level source code where does the unfairness actually come in yeah that's a great question and I would hasten to add that I think unlike in privacy where there's this one definition that many people agree upon people don't really agree on what unfairness is but maybe I can address first point just to say an algorithm is some computational process it's a human artifact right does it make sense to talk about algorithmic unfairness for example another human artifact that's a useful tool is a hammer and I could use a hammer to do unethical things I could go around whacking people on the head with hammers but nobody would make the mistake of ascribing the moral failing to the hammer right we wouldn't talk about ethical hammer design and it would clearly be my moral failing if I as a user of a hammer went around whacking people on the head with hammers but algorithms that result from the machine learning pipeline are different because many of the bad effect that we see from machine learning algorithms and maybe just to be concrete we can talk about what has become maybe the most well known example which was unfairness in recidivism prediction that that ProPublica uncovered a number of years ago so just for background in many states actually including in Pennsylvania when judges make bail and parole decisions they are actually given as one of the pieces of information the output of a predictive model trained using standard machine learning techniques whose goal is to try to predict whether if the inmate is released they will go on to commit another crime within 18 months so there's a classification algorithm that it's trying to predict whether people are going to go on to commit crimes if released or not and this is helping to inform whether they spend time in jail or not just to be a bit pedantic one would assume that if the prediction is that the person is going to return to a life of crime that that's a undesirable outcome and you would deny them the parole and parole would only be offered to those for whom the algorithm predicts they will be successful in reintegrating is that correct that's right the real world is messy and there's also some other considerations but maybe as sort of a first-order sketch you can imagine that what we want to do is send the people to jail who are going to commit more violent crimes and release the other people so machine learning algorithms of course are never perfect they always make mistakes and in this application there's a kind of mistake that is particularly harmful to the people on whom we're using this algorithm in particular the harmful direction is the following you know if you are someone who is actually not going to go on and commit a crime all right so you're someone we should release but my algorithm makes a mistake and incorrectly predicts that you will go on to commit a crime this is harmful to you because it means that you're gonna spend time in jail when you shouldn't have mistakes in the other direction or may be harmful to society but but not to you specifically so the mistakes that are in the harmful direction these are the false positives the people that we predict are going to commit crimes but actually wouldn't have okay and so the false positive rate is somehow the rate at which our algorithm is making mistakes in the harmful direction so here it would be harmful towards the inmate exactly harmful towards the people to whom the algorithm is being applied what Pro Publica discovered was that in Broward County Florida where particulars of this tool was being used the false positive rate was substantially higher amongst african-americans than it was amongst Caucasians which is not to say just that it was sending more African Americans to jail compared to Caucasians what it was doing was it was making mistakes in the harmful direction at a much higher rate amongst the African American population compared to the Caucasian population the sort of burden of the mistakes made by this algorithm was being disproportionately borne by one population compared to another and that's one of the things that people sometimes mean when they talk about unfairness and machine learning okay so back to the original question yeah why does it make sense to talk about for example ethical algorithm design when it doesn't make sense to talk about ethical hammer design we've now seen many many instances in which we have unfairness of this type where the mistakes made by the algorithm are disproportionately borne by one say minority population compared to the world at large but in almost all of these instances it was not the case that there was some malevolent software engineer behind the scenes who was intentionally coding bias into their algorithm right it wasn't akin to someone intentional whacking someone on the head with a hammer right if that was the case we would sort of know how to deal with this using existing regulatory approaches the thing that makes this difficult and thorny problem is that this kind of algorithmic misbehavior is almost always the unintentional side effect of applying standard machine learning methodology and so it's not enough to just say okay don't hit people with hammers anymore don't deploy algorithms that disproportionately harm one population over another because people weren't trying to do that to begin with it was an unintentional side effect and so what needs to be done and what is happening now is the design of new algorithmic methodologies that do not result in these harmful unintended side effects it's no surprise to me that there will be people who are close to these topics and aspiring to produce better algorithms through really good research and investigation and things like that but certainly there will be some people who say this is just grounds to take algorithms completely out of the courtroom and for me personally that would be a loss I like the idea of the algorithm there in the sense that you can audit it you know maybe there's even a fun science fiction story in here in which a lawyer tries to get a few lines of source code regarded as unconstitutional to change the outcome of a case and that resolves in some pull requests at the end of the story just in general what are your thoughts about the future of algorithms and courtrooms and things like that okay yeah so I've got a couple of thoughts about that and maybe I should preface this by saying of course I'm a computer scientist so I'm Pro algorithm and I definitely think that algorithms used correctly have the potential to make society in general better and more efficient and algorithm is not a bad word maybe the second thing I would say is that many of these issues of unfairness these quantitative measures of unfairness like the difference in false positive rates come to the fore when we are using algorithmic decision-making because there's no way to avoid quantitative thinking when we're dealing with algorithms but they're not issues that are specific or inherent to algorithms and they apply just as well to human decision-making one of the things that came out of the Pro Publica controversy and the ensuing academic literature was that there were multiple reasonable things that you might mean by fairness that are actually mathematically in conflict with one another you can't satisfy all of them and this impossibility result although it was sort of brought to the fore because people were thinking about algorithms has nothing to do with algorithms it applies equally well to human decision makers maybe I would also say that for folks who want to remove algorithms from our daily lives and go back to happy agrarian society where are we we didn't have any algorithmic decision making tools and it seems unrealistic to me the cat sort of out of the bag already yep conceivably you could remove algorithmic tools from criminal justice decisions in fact the evidence is not very strong to suggest that these tools are actually helpful in this domain but there's all sorts of other important domains where algorithms are firmly entrenched for example in lending you know if you apply for a credit line increase on your credit card typically no human being will ever look at that request it'll be automatically approved or denied by trained classifier and when you look at applications like this I think they're both completely ubiquitous and extremely valuable you know they're making all sorts of things better and more efficient these are places where we wouldn't want to remove algorithmic decision making because of how useful it is if there were a better way it is possible to design these algorithms to avoid the kinds of discriminatory harms we've seen and finally you know I would point out that obviously discrimination and racism and sexism aren't new to the computer age these things have existed for a long time and as you say although people often think about algorithms as being these inscrutable black boxes you know people can be as well people are very good at coming up with convincing post hoc explanations for why they did what they did but I think more often than not these explanations are come up with after the fact rather than being an honest picture of the decision-making process that the human being made algorithms of course can be highly inscrutable but at least you have some hope of examining them and understanding what they're doing well Aaron I really enjoyed your book with former guests to the show Michael Karin's the ethical algorithm the science of socially aware algorithm design I'm hoping a lot of listeners pick it up who's the audience for this book it's a general audience book we hope that you know for example all of your listeners will enjoy reading it we worked very hard to make sure there were no equations in the book so you know it should be easy to read but at the same time it will hopefully have interesting content even for more technically minded people because although we worked hard to remove equations we did want to write a book that was really about ideas on science and we tried hard and I hope succeeded in not watering that down so the book is generally optimistic I think unlike some other excellent books in this area which have pointed out the problems that have emerged around fairness and privacy in each of the chapters of our book although we start typically pointing out why you might want to worry about things like fairness and privacy and gaming and other things like that we quickly dive into the science that has emerged in the last 10 years usually aiming at solving the problems identified in the chapter so in the privacy chapter we introduced differential privacy talk about it at some length both about what differential privacy promises and about what it doesn't promise in the fairness chapter we talk about different approaches to thinking about fairness we acknowledge that people don't yet agree and maybe will never agree on what single thing unfairness is but we can talk about different kinds of unfairness and again reason about trade-offs in the third chapter we talk about why it's important to think about the incentives engendered by algorithms and how when you change an algorithm in a particular way it will change the way people interact with it because people actually care about the outcomes and so you have to think about what a game theorist would call the equilibrium that you're leading your population to when you deploy a new algorithm we talked about the statistical crisis in science which is exacerbated in the machine learning age when it's so easy to share datasets and reuse them and emerging tools to actually make repeated use of the same data set without giving up on rigorous statistical and in the last chapter we talk about things that we think of as extremely important for example interpretability and machine learning about which there isn't yet much to say of the sorts that we talk about in the book as I said at the beginning of the podcast we're both theoretical computer scientists and the position that we take is that the way to make progress in a field is to first start with a definition what is the definition of the behavior you really want your algorithm to have and only then explore how you would go about designing algorithms that have this property and what the trade-offs are when people talk about interpretability and machine learning this is obviously a very important topic but what's lacking is a clear definition of what people mean when they say that a learning algorithm should be interpreted will or explainable and I think that progress in this area will be limited until good definitions are out there anyways it's a book that covers lots of I think interesting ideas almost all of the major research topics that I've studied over the course of my career as a computer scientist but one that tries to explain these topics in an accessible way to a general audience you don't need a PhD in computer science to read this book yeah I've had the opportunity to recommend it to a number of colleagues and yeah it's not a reference manual for a desk it's really an armchair book you can sit down and enjoy to wind up where can people follow you online I have a website I think I'm still the first Google result if you search for Aaron Roth I have a Twitter account aaro th and you know if you're into the math then I occasionally write papers that you might enjoy outstanding I'll have all those links in the show notes thanks again for coming on thanks for listening to episode 2 of data skeptic interpretability our guest today was Aaron raw his book the ethical algorithm is available in all the places you can buy books our theme song is number 5 by the third wave goat Big D in the kids table Claudia Armbruster is our assistant producer Vanessa Burciaga is our guest coordinator I've been your host Kyle polish thanks for listening [Music]