Stanford Seminar - Ethics Governance-in-the-Making: Bridging Ethics Work & Governance Menlo Report

and we're really pleased to be here and pleased to be giving a talk together because as we're saying neither of us is given an in-person talk in two and a half three years so 20 minutes each seems like a reasonable amount of time to speak loudly so we'll see how it goes and we'll see how our voices hold um so I'm Katie Shelton and I'm Megan Finn hi I'll um and today we're going to talk about a case study that we've been working on within a broader project focused on what we're calling ethics governance in the making which is a mouthful and we will explain what we mean by that um and we've had an opportunity to present this work before um two uh scholars in history of technology and science and technology studies which are the disciplines that we sort of come out of but we haven't had a chance to do it for this interdisciplinarian audience so we're really excited uh to have the chance to do that today um but with that said please ask if there are things you want clarified during the talk we're sort of working on making it a talk that works in more environments so uh please feel free to ask a little background about us and how we arrived at this project um we both teach policy and ethics courses to students in information studies Library science HCI and um in the course of teaching those classes we both discovered this thing called the Menlo report and the Menlo report um ended up being for us something we thought was a really interesting example of Ethics self-governance a field coming together to say these are going to be ethics rules for our field I mean then we asked our like computer security colleagues about this and they didn't know what it was and we thought that's really interesting what's happening here and as social scientists we wanted to investigate this more and the reason that we were really interested in investigating this more is because um you know we have this perception that uh you know ethnic self-governance is is a Hot Topic in computer science right now um that self-governance the field of computing is going through a real moment of wanting to self-gov govern ethics um and um and you know Menlo was 10 years ago but since then the tenor of the ethics conversation around computer science has really changed and has ramped up in many ways um a range of public examples plus some really important academic work have highlighted challenges in data and research Ethics in uh the development of pervasive tools of surveillance and in discrimination in decision making systems and machine learning systems so the ethics governance in the making process of the Menlo report was of real interest to Megan and I um as this within this moment now when Computing is is confronting crises of Ethics um and we wanted to know what concurrent efforts for self-governance learn from this process 10 years ago okay so this concept that we're interested in ethics governance um is and the idea that these are there are networks of people in this case sort of whole academic fields that are attempting to resolve uncertainty about um what is the right or wrong thing to do through new rules new procedures or new consequences and sometimes all three um and those kinds of efforts are happening like right on this campus as well as in research communities that both Megan and I are part of so for instance Stanford's ethics and Society review board engages researchers um in uh early phases of AI research through a procedures that consider the societal risk of that research and they attach funding consequences to those decisions that's the consequences part Megan was the author recently of the national National academies report that offered best practices for funding agencies for academic organizations and for individual researchers concerned with responsible computing I was the former chair of the Sig Kai research Ethics Committee which is a volunteer group that offers advice when papers in Sid Kai conferences are flagged by reviewers for having ethics issues so these are all current examples of uh ethics self-governance projects but there are numerous historical examples as well uh perhaps among the most famous is the asilamar conference on recombinant DNA which is a 1975 meeting of to set voluntary guidelines for the use of recombinant DNA in research and then many of you if you've done human subjects training at your University may be familiar with the Belmont report which was in 1976 effort to identify ethical principles for human subjects research and then of course there's Menlo which would become the subject of our case study Menlo took place um almost a decade as I said before the wave of recent controversies and uh starting in 2009 a group of computer scientists primarily as well as lawyers in government security uh and network measurement research folks culminated got together and put together guidelines for computer security and network measurement research and that culminated in this 2012 report and so the recent history the fact that all those folks are still active researchers they're still around like we could talk to them gave us the opportunity to explore how is this work done uh what are the ingredients in self-governance efforts and then how do those ingredients matter to the outcomes and the impacts of the project so Megan and I as I mentioned before both have roots in science and technology studies and we began framing our interests in ethics governance and ethics self-governance in that literature so we draw on the idea of constitutional moments which are times when a field or a group of people are renegotiating the terms of unethical and ethical practice so that's what's happening is there's a constitutional moment and then we were really interested in an idea called ethics work or the idea of Ethics work which is Malta zewitz's term for when people have to do work in their like labs and workplaces to cope with ethical ambiguity so when you don't know what to do you maybe talk about it with people in your workplace you're doing work right that's ethics work and we're interested in kind of on the other end of a spectrum from ethics work ethics governance drawing on Rebecca slayton's definition of governance as networks trying to steer people in a particular direction by either accepting members into the network or cutting members that don't go in the direction they are wanting to go so the contrast between ethics work on the one hand sort of local ambiguity resolving and ethics governance cutting people from networks if necessary on the other led us to a set of research questions that are focused on how localized ethics work becomes ethics government governance on a bigger scale when we call this bridge ethics governance in the making it's a mouthful term but it was the best we could come up with we have better terms yeah so our research questions became what sparks a group of people to do the really resource intensive work and you'll see and it was really resource intensive of imagining new governance for a field um what challenges does governance in the making present what's hard about it and how does the work of envisioning new governance influence what becomes seen as ethical action and guidance for the field so the memo report gave us the chance to answer these questions for a single case study localized ethics work has been occurring in the computer security and network measurement fields for decades the computer security research Community has long debated ethical questions like the acceptability of monitoring Network traffic the political consequences of e-voting and principles for handling vulnerability disclosures there's lots of localized ethics work happening in computer security and so our project began with the question of how and why that those localized debates became this governance effort and so producing the mental report was really an effort to move from Reliance upon tacit Norms within labs to an agreed-upon standard and we analyzed the mental report documents which included uh both uh the Menlo report itself as well as a companion piece that had case studies in it we also analyzed several articles about the report that were written by report authors at the time um and then we talked to authors so in the final list of authors six of the authors were active computer science researchers in either Academia or technical research institutes four were working as lawyers or law Scholars two were employed by the Department of Homeland Security who funded the effort and we're going to talk more about that in a minute and two were working for in research protection within a research organization that means they were sort of IRB ish if you know about irbs um so we conducted interviews with 11 of these 15 authors everybody who would talk to us um and discuss the work of with them of creating the report how participants became involved how the writing and collaboration process was organized and funded uh and the Legacy and influence of the report and we also analyzed separately ethics statements in papers published within network security and network measurement conferences in the years after the report so the report itself interestingly Builds on core principles that were articulated in the 1979 Belmont report and that's the report that shaped human subjects research standards and the social sciences um in the US and so Menlo adopted the Three core principles of Belmont respect beneficence and Justice they laid those out as equally important to Computing research um so and in that move made a move where they said Menlo said that decl that they basically declared that Computing data was human subjects data and would be subject to these principles um and in addition Menlo added a fourth principle uh respect for Law and public interest and we're going to trace how that fourth principle came out of major debates that were happening in the field at the time so through our analysis what we found was that even in this case study of technologists who were pretty self-consciously crafting new ethics principles that's what they came together to do um they was it was really a process of what we call an STS pre-collage which means bringing together what's available at hand bringing together available tools at hand this was not this was way more prominent than sort of bringing together of existing tools than say moral reasoning from first principles um we kind of thought of wait as as logical reflection and then coming up with principles that you know come from from for first um first standards the first principles but it turns out that crafting ethical principles is work two like any other work like work in a lab like work in a workplace um and that bringing together and adapting existing materials the stuff that you know about turns out to be really important so as research communities in AI ethics and other controversies in our spaces now begin their work of governance in the making I think it's going to be it's going to be really necessary to decide who participates to decide um what uh what things we have at hand right what principles are we going to bring to these projects and a framework for how to study how that happens ethics in the making governance in the making can help us understand uh these efforts better and maybe be more reflective about how we set them up in the first place so we suggest based on Menlo a framework that incorporates five elements of work um and we're going to walk through some of them today the first is marshaling the resources that are available all of this is time and money so thinking about the resources available really matters to the outcomes um background uh or the second is the attempts to repair what is a group trying to repair what wrongs have gone on that have brought a group together to try and fix those wrongs third is attempts to anticipate the future so we'll talk about how Menlo participants were trying to deal with a future that they couldn't see coming um and particularly changes in Technologies because that changes quickly yeah uh next was work to close or settle existing controversies in the field so to try and make some decisions about yes this is okay no this is not okay um and finally to resolve ethical uncertainties as participants tried to understand what practices are acceptable so because ethics in the making is this free collage process um we see this list of five things as not total you know it's not a necessary but not sufficient right it's not totally generalizable things will be different in different places but we do think it's helpful to understand the process um involved in what gets put into ethics governance So today we're going to focus on one two and four but we can also talk about three and five if you are interested um in uh um you know more details in the Q a okay so first we looked at the resources that were marshaled for the Menlo project and first among these resources were the people involved it was the primary resource involved with people whose background and expertise impacted the resulting guidance for Menlo the actors involved and the resources involved influence the content of the report and they also as I'll talk about influenced how successfully the report was able to govern Menlo was bootstrapped it was bootstrapped through a U.S Department of Homeland Security existing program called predict predict standard for or stood for protected repository for the defense of infrastructure against cyber threats kind of a background but predict is what they were trying to do right predicting threats essentially um and one of the many things predict was a big program a big funding program for academics but one of the big things I sought to do was to enable data sharing between academics about online networks so that you would have a place you could go to get say ISP data if you needed ISP data so they wanted to do data sharing um and as part of their interest in data sharing they started to run up against ethical issues like privacy and things like that where if you're sharing data you might you might need to worry about so predict the existence of predict enabled Menlo almost entirely um and then that enabling happened in a variety of ways writing workshops we're connected to predict Pi meetings so that meant they generally occurred a day before or a day after a pi meeting Pi meetings are times when researchers are required by their funders to show up and be there and Report out and so what the spent is there were a bunch of people funded on predict who also became involved with memo because it was pretty accessible for them to be involved with memo um and uh it made it yeah it made it easier for all these network security folks to attend um and so the co-location strongly shaped participation within the project uh the people who participated in the report were primarily academic computer scientists and were largely from the network measurement and network security subfields of computer science so it was not really computer science but large it was a pretty specific subset um and then these researchers were joined by a handful of people who were recruited by predict organizers uh primarily lawyers with expertise in cyber security law and so we think it's also important to consider who wasn't uh at Menlo meetings so traditionally um ethicists and social scientists have been part of big self-governance efforts in various places um for instance acilimar Belmont um and they were absent from Menlo proceedings um and an author you know described us we reached out to a number of people in the ethics Community who go way back well-known names I won't say the names we tried to evolve some of them there weren't a whole lot of takers partially because it was a volunteer effort I think but also partially because it was leaving the area they have expertise in that's my gut feeling so without Ephesus on the team the team lacked at first he's to question the coherency or appropriateness of those Bell not mod principles to proceed from perhaps concrete and concrete ethical Frameworks that break a large work we think is a result of who was in the room right they were not philosophers to start from first principles um and you know it was hard to invent new forms of ethical reflection that might have been a better fit for computer security without folks trained to do that um and the authors acknowledge this limitation right like they were really Frank about this in the introduction to the report they wrote the report deliberately does not explore alternate ethical paradigms to Belmont and while not discounting that there may be novel implementations of the Belmont report principles and applications that should be considered it makes no definitive recommendations in that regard right this was a fight should we do something Beyond Belmont they said Okay We're Not Gonna We're not gonna go there um and there's another further limitation a few further limitations to this group that I think are worth talking about um without social scientists the team couldn't incorporate expertise on sociality on inequality work practices institutional Dynamics none of that really showed up in the discussions around the report um that might have supported the implementation of governance networks right you can make recommendations social scientists might have been helpful in figuring out how to govern um or how to get the governance Frameworks working um so that's not there and finally the authorship group uh reflected the lack of racial and ethnic diversity within computer science at the time um and and now limiting the standpoints that were incorporated into the final recommendations so after the Menlo report was written they still needed resources to circulate the report to engage the envisioned governance networks such as program committees or institutional review boards the groups that they thought would help implement this and menlo's transition from in the making to actualized governance was really challenged by both Logistics and resources as a first step the team took a path that had been modeled by Belmont but was pretty unusual for computer science codes of Ethics they published the Menlo report in the Federal Register which is the official channel for sharing and requesting public feedback on U.S government agency rules um so the public change for the public chance for comment had some advantages um it differentiated Memo from many Science and Technology self-governance projects which have been critiqued for a lack of public participation um this was a channel for public participation um they didn't you know public Representatives weren't part of writing but they did have a chance to comment but even government employees among the report authors acknowledged the limitations of the Federal Register as a dissemination approach as an author told us even still it's only people who follow Washington that actually know what the Federal Register really is um the Menlo report was first published online in December 2011 and it received 16 public comments so not a not a ton um so though the process of publishing Menlo in the Federal Register was the start of a process for federal agency rulemaking ultimately the Department of Homeland Security did not complete the rulemaking process uh Menlo is not part of any regulations at the government level now um as a researcher who later became an employee at the Department of Homeland Security told us someone should have led the process with rulemaking which we would we should have tried to do that list but it would have been a really heavy lift so it didn't happen so sharing the report in the Federal Register signaled that Menlo authors intended to recruit powerful existing regulation networks for governance but menlo's recommendations were never formally adopted by government agencies so instead menlo's primary influence on the field was through academic networks of peer review conference committees became the ones to effectively steer the ethics of security and network measurement research by cutting or refusing to publish research that didn't inherit adhere to Menlo recommendations as an author accountant it's about changing Norms you don't see it right away and now I can point to top security conferences that require ethical statements I consider this a clear victory so though Menlo as we studied it was governance in the making we do think it helped move the field of network measurement and computer security into a mode of Ethics governance after the report was published one of the highest status conferences in the computer security field usenix included the following requirement and their call for papers in 2013. papers that describe experiments on human subjects or that analyze non-public data derived from Human subjects even anonymized data should disclose whether an Ethics review for example IRB approval was conducted and discussed steps taken to ensure that participants were treated ethically other major network uh security computer security and network measurement conferences soon followed and we tracked how many papers we're talking about ethics and IRB in the years after the publication of the memo report and the introduction of the requirement to discuss research ethics in some cases the addition of such a requirement doubled the number of accepted papers discussing ethics and or review board approval and in 2019 the internet measurement conference which is a major conference in network measurement explicitly included links to the Menlo report in the call for papers so we don't want to argue that Menlo caused this effect of conferences deciding we don't have the evidence that that was the case but it coincided memo coincided with governance changes in the field that were associated with most associated with the author groups and the authors of Menlo went out to be on these program committees in in some cases so this was a change that was happening in the field at the time so I'm gonna head off to Megan now to talk about the next process that we discovered all right let me pause are there questions you guys are so quiet um which is great Katie and I both obviously love to talk um but please please interrupt us or interrupt me if you have any questions go inside with a shift in the topic topic published in the conference like where people doing more research that might have human subjects yeah or or was it already happening and now they were just actually referencing so we've collected this data at like a fairly shallow level and are in the process of analyzing it across this and we actually collected data from a couple of other conferences um so I can't answer that yeah it's a great great question and um which we will one that we definitely will think about any other yeah is there anything has there been any personal statements from people involved in these conferences that they're aware of this report and it changed maybe it was in the discussions when they changed the policy or anything like that yeah that's a great um so we also haven't done those interviews but that sounds like great data to collect I'll say that um our participants were also um in program committees they were also program chairs um and we've collected some data about how the MMO report authorship group um became or didn't become um sort of visible members of these different conference communities the data that's also much more interesting is right like who's doing reviewing and um to the extent that people are doing reviewing and they're sending stuff back and saying that it's not ethical that it has been data I know that Katie's actually formally tried to collect and it's near impossible to get a hold of we all hear about it anecdotally through interviews that we do people will say like oh yeah we and some of that shows up in this data some of it shows up in um another interview set that we have but people will say yeah you know I read this paper it made me really uncomfortable I didn't know if it was ethical I sent it back to the PC but they don't necessarily even know as reviewers right than what the PC does with it um great question others okay oh yeah do you know why the Mucinex conference seemed to adopt report or a kind of requirement of a full disclosure earlier and then secondly looking at the CCS conference looked like it was all there was a jump in and disclosures yeah for the adopters yeah yeah yeah so that's a great question so the second question if I had to make a guess um my understanding is that publication practices in computer science are often such that if your paper doesn't get into one conference you might reuse it in another conference with like a later I know maybe with an Ethics statement on it right maybe that's some of the reason right is somebody wrote a paper for a usenix conference it didn't get in they revised it but they kept it in some of that ethics disclosure language in a difference conference so that would be that's a total hypothesis have not looked into it yet again probably not data that we can actually even get although although I mean Michael's question that I think is a really good one is you know how much of this is menloe reverberations and how much is this topic change all of a sudden are doing human subjects research because the other thing that's changing at this time and we talk about this from our Menlo authors is that um network data went from being something that a few people had to something that everybody's got right like you could you can get your whole universities Network and and track it maybe you shouldn't or you know and so all memo authors are struggling with this and the field is struggling with this so it's possible that some especially these early pre-menlow um sort of bumps are um are about you know people having new controversial methods and and responding to it with ethics work right within Labs saying okay and yeah we thought about this right and then the question around why use NYX first um so actually like internet measurement conference and the first um conference that we know of in this sort of broader field of network measurement and security to adopt this requirement is actually a soups which I don't know what that acronym stands for but my understanding is that that's the HCI meets security conference Symposium privacy yes Symposium unusable privacy and security nice job thank you Michael the on is important here they adopt uh requirements in 2009 and I think that that's a lot because they're involved with the HCI community and the security community and so and I I've actually had some conversations with people who were sort of in the room when they adopted that requirement and they said it was basically this issue where people in HCI were like of course if there's human subjects you're talking about it in your paper but that wasn't actually clear to the security researchers so part of putting that requirement in was actually just articulating assumptions that one research Community held close and you know sort of assumed everybody understood but another research Community really didn't um and so another guess about why use NYX first is it might also be that usenix had more of a presence of HCI researchers and sort of like anecdotally Katie and I might guess that but I have no that's that's totally again us on our part really great questions you guys thank you we have a good history to trace okay I'm going to jump back into it then um Okay so moving back to our discussion about what it means to do ethics governance in the making um our second sort of pillar here is we found that ethics governance attempts to repair the wrongs of the past in anticipation for guiding the future so one of the things that really motivated researchers when we went and asked them why did you volunteer your efforts all of this precious time that you have to write this report was that they were really concerned that there were papers in their field that had been published and that these Publications were communicating to other people in this the field particularly their students that there were research methods that were acceptable that they did not think were acceptable um so we use this term repair to talk about this effort to sort of go back look at the past and fix it um and we're borrowing this repair language from from our field Science and Technology studies I just want to know we don't mean it in repair in the sense of like giving restitution right um it means that they're very much looking the memo authors are looking at specific Publications they're attempting to identify their weaknesses and they're producing guidance that they think attends to these weaknesses um okay so repair is also really strongly tied with this idea of anticipation as you've guessed from probably what I've said so far um it means that the report in this sense is both simultaneously looking to the Past like what we did wrong and looking to the Future what can we do better um and you know noting the gender pronouns that our interview select interviewee selected here um this quote kind of perfectly encapsulates that forward and backward look the idea that it's trying to anticipate the future and fix these problems of the past others were looking at ethically questionable work um and he was publishing papers and they were saying wow if he did it we can do this right this was the this was the problematic publication record that they had to attend to to move forward into the future foreign so our interviews were interested in preparing repairing the past research record but they also explained to us that they weren't at this same level of public urgency in which say like the Belmont report was being written um and there's this concept in public policy and political science that they talk about quote unquote focusing events and these are events that are catalysts for like major policy changes um they're like the revelations around the Tuskegee experiments that led to the Belmont report are like a perfect example of a quote-unquote focusing event and importantly Menlo is really not one of these situations um our author said we're not infecting people with syphilis we're not doing a Milgram experiment that's going to necessarily scar someone emotionally there's a removal a distance between the human beings and the researcher it was not felt that it was quite as visceral of a thing and so the Belmont report really as you can tell loomed very large as this paradigmatic example of Ethics regulation but our participants felt that there wasn't a lot of public attention to Computing ethics at this time right this is sort of in the mid 2000s um they're very aware of these sort of utopian imaginaries this sort of Savior ideals discourses that are circulating in the early 2000s when this project originated right the so-called techlash is really a decade away okay but not having focusing events didn't mean that there weren't problems in the field there wasn't a Tuskegee experiment but there were a lot of controversial papers that were troubling to many researchers and many funders and part of the main report companion document which was published about a year after the Menlo report included a list of papers in one of the appendices that were presented as case studies but could really be read as sort of like a naming names um that these were you know these were sort of the notable papers that when we asked uh our interviewees why did you work on this project these were the papers that they were pointing to they they read one of these papers and they thought uh oh like this isn't good for our field um it wasn't the Tuskegee experiments right but it was really really worrying to our participants so I'm going to talk about one of these papers which was published in 2007 as a way of discussing one example of what the memo authors were reacting to um when they started the project and sort of illustrate this idea of what we call like an atmosphere of concern and measurement and security research at that time and this example also helps explain a bit about why this sort of Ethics self-governance project took the specific shape that it did right as opposed to the sort of many other different approaches to dealing with these kinds of um to do dealing with these kinds of problems so broadly many of the concerning papers in this list um in the memo companion document were really fundamentally about information flows like who gets to see what how and in what circumstances and as Katie said earlier information sharing was really core to this project of predict program program researchers are trying to understand the characteristics of the internet and its weaknesses and working on the challenging the challenges that they saw in preparing network data for sharing but getting network data was not easy it required networks of Administrators that they would share data and trusted researchers that they were sharing the data with right compromised data could affect the security of the network and this paper that I'm going to talk about is called playing The Devil's Advocate it's specifically addresses this problem okay so what is this problematic paper um The Devil's Advocate about it was published in 2007 um the paper is a play on the title of a 2006 paper called the devil uh and packet trace anonymization and the authors of this original paper the devil and packet Trace anonymization paper presented a method for anonymizing network data network data that the researchers had received from Lawrence Berkeley National Labs and the anonymized data set was shared with the authors of playing The Devil's Advocate who both de-anonymized the data and then shared the details about the de-anonymized data set and as the Menlo authors explain the authors applied um third genome anonymization techniques to a publicly accessible data set prominently used in the security Community to prove the correctness of their result they published key information about the public data set in their paper thus revealing the internals about debt researchers Network de anonymizing network data and revealing networked in network information was experienced as a severe violation of the research Community Norms by the authors of the original paper and others in the community and the authors of the devil um and packet Trace anonymization almond and Paxton um had not expected that their colleagues would de-anonymize the shared data and share these details right so they published a rebuke that is very politely called titled issues and etiquette concerning use of shared measurement data and the authors emphasize that no matter how careful a network administrator was being that they were always sharing more information than they thought and they reasoned that technology could not solve many of the issues around data sharing ethics and they said ultimately the choice about what to release how to obscure the data and whom to release the data are policy decisions emphasizing that better policies and Norms needed to be established around sharing research data they put forth their suggestions for what kind of policies would really be best for mediating the sharing data sharing Relationships by spelling out acceptable uses of the data and problematic papers such as playing The Devil's Advocate really in a sense contributed to this atmosphere of concern that then opened up this idea of policies of guidelines of rules and even ethics as one of the possible modes of addressing unethical scholarship and at the very same session in which almond and Paxton were talking about this issues in Etiquette paper at this major conference legal scholar Paul ohm is presenting a paper that he co-authored with two University of Colorado computer scientists um Doug stickler and Dirk Grunwald which examined how U.S legislation applied to network measurement researchers and it explained legal issues went beyond data sharing Norms to the legality of the data collected and they're talking about the federal wiretap act the pen register track and Trace act and they suggested that computer scientists were possibly engaged in illegal activities and the author is offered a number of suggestions which move from repair to anticipation to improve the likelihood of following the law and they conclude that at the very least we should proceed informally by beginning to have a conversation about what constitutes acceptable network monitoring a codified understanding that reflects even rough consensus would be a useful tool to bring to Congress or to show courts it is important that these norms and rules are agreed upon from within our community rather than dictated To Us by some outside court or agency in order to repair the problematic research records Scholars were arguing for the needs of something like the Menlo report to be created right and importantly they argued that it should be created by the research Community itself and so for many of the authors the memo report was exactly what om and his colleagues were advocating for it's this clear set of guidelines that was in the Federal Register and could be hauled before Congress okay stepping back to our framework um so our third our third bucket is this idea of anticipating future change which I've also already briefly touched on um memo authors anticipated future research methods and attempted to create guidelines to address these future research methods they also anticipated who might read the document possible resistances to it and what the imagined audience of the report might do and they crafted the report for this imagined audience um as Katie said we're not going to talk about this one in quite as much detail so we can move on I'm going to pause again and ask for questions any questions yes people or not you know it was not quite for that yeah oh one of the examples you were using most these kind of fundamental reports were out of a crisis moment is there any research around kind of I don't for example I must be familiar with previous research before the report on human subjects yeah is there any Research into like is it necessary to have that crisis moment like if a report is made before that crisis moment does that just reduce the efficacy because people don't understand the need for it yet I'm just curious like if there's anything out there about that um so I let's see so I also work a lot in the field of disaster studies and so focusing moments are like a huge deal because oftentimes there'll be like some kind of Regulation or um policy change that particularly like scientific experts might have been advocating for or Engineers have been advocating for for years behind closed doors and sometimes it takes one of these like major disasters and then suddenly people are like I guess we do want anti-seismic building codes right um and so you know you have these sort of people who are standing in the wings saying like and here it is right um so in the in the sense of disasters yes I think Katie and I kind of are like we're in this interesting I think constitutional moment in Computing ethics right now and I don't know if you all saw there was um a sort of call for uh AI ethics ideas by the office ostp um and the the heads the head to the office for um office of Science and Technology policy um so I think it sort of looks like maybe we've we've been sort of in some kind of prices but it's um yeah but I don't know and I think Katie and I are hoping to tackle this actually in our next research project um about whether they're like these sort of particular inflection moments where these kinds of um policy and or sort of Ethics responses are particularly effective and are taken up um on Mass it's a very good question it's literally like sort of the work of our next couple of years basically how big a crisis is a crisis right and it's pretty clear in disaster studies and it's a much less clear and Computing right like what constitute are some racist algorithms a crisis maybe right but like uh but like you know it's not an earthquake right so then this like yes I think it's a really good question but what was clear from Menlo is that a few controversial papers were not a crisis but they were worried there might be one right um and so yes is that the effective moment or do we need to wait for the disaster yeah it's a great empirical question yeah and I think it's a little bit more complicated because the racist algorithms have existed for several decades without being identified as a crisis and in the last say like five to ten years a number of excellent Scholars and activists have said this is a crisis but the public attention turning towards um hey let's do something about that has not been the kind of whiplash of an earthquake um nonetheless I think you know everyone would argue yes this is a crisis or hopefully in this class you'd argue that this is a crisis um but it doesn't have that sort of like immediacy um that maybe right like an earthquake might have um yeah a piece that I didn't quite get from yeah you're talking about the Devil's Advocate paper was the was the ethical concern with the topics that they were exploring or was it with the methods that they were publishing like they should have redacted certain things in the in their research um my understanding was that it was the the latter um that it was the methods um and that there were details shared like through sharing these de-anonymization techniques of the network data set they share details about that Network that were inappropriate so I think it was like sharing sharing the entire method if that makes sense [Music] I think that also occurred around the same time as I don't know if you all remember uh I'm gonna guess no uh Netflix at one point released this really big data set of Netflix data and I think it's kind of it's like a kind of paradigmatic example in a lot of Ethics classes of like nothing Netflix released this data said help us improve our algorithm and a bunch of like clever computer scientists said we can sort of figure out who's who in this data set right um yeah it was and the other piece of it was that the data sharing agreement the data had been shared in de-identified form the researchers felt that they didn't understand that this group was going to re-identify the data and so that was it was a like a uh like a disagreement amongst colleagues almost that blew up into like is this ever okay to do yeah um I'm gonna step quickly through this number four um so as Katie noted one of the major accomplishments of the Menlo report was this explicit identification of computer research as involving human subjects research and the authors of the Menlo report believe that their colleagues for the most part um were trustworthy stewards of data but they were concerned that sometimes their colleagues didn't understand the fact that they were working with data about people um so they really wanted to sort of Shore that up and make that very clear in the report um and but by bringing in so they did this by bringing in the uh the Belmont principles which they were very attracted to because they're simple um they had this huge impact on on policy and they've been very durable right there aren't a lot of Ethics principles that have been really useful for decades and decades and decades I shouldn't say that there aren't a lot of Ethics principles that have been turned into policies that have been useful for decades upon decades um but in bringing in the Belmont report and the Paradigm of human subjects it was like this really uncomfortable fit um as this quote from the memo report illustrates they sort of say traditional biomedical and behavioral research requires this protection of natural persons um but within ICT research right um it may be an information system um uh Associated data and it complicates the assessment of potential harm of users to that system and that data it's a lot more ambiguous it's not as clear of a fit and so with the application of the human subjects data associating network data with human subjects it enabled a really clear way forward for the authorship group um but it also codified a set of governance mechanisms which is IRB review which was not as easy of a way forward for this group and so this caused controversy this wasn't without controversy within the group so on one hand you had people who said you know when we began at Menlo there was a diversity opinions about the applicability of Belmont to the network security research and there was always someone who would stand up and say well we're not working with humans we're just working with bites and I suppose one can say well I'm not working on humans I'm just working with blood samples I think the argument is about as strong in both cases and then on the other hand there were there were participants who said you know this we we don't want everything to be reviewed by irbs this is going to create this huge bureaucratic structure for our organization and this led to some really new uncertainties um that the memo report introduced so they said in some cases irbs didn't know the law and researchers would go and do things that were kind of in a gray area or might have actually broken the law and the IRB approved it because the irbs did not necessarily have the expertise in some kinds of network security network and security Computing research furthermore and again this is kind of getting back to some of the issues I was pointing out earlier the potential harm of to subjects in computer security research is much more abstract as are the benefits you don't really know how an individual person might benefit to consenting to research or what impacts there might be in waiving consent for monitoring Network traffic or just relying on consent that's in the terms of service that you click through so it introduced all of these new uncertainties to the research as well okay fifth we found that ethics governance attempted to close or settle controversies so as I noted earlier there were some really big controversies around not only whether scholarship was unethical but also whether it was illegal and so to resolve those controversies around legal practices the memo report went a step further than Belmont and they added this fourth principle of respect for research um sorry respect for the law and public interest but as we all know the law and public interest are not always the same thing right these sometimes compete with each other and so this this sort of edition of this principle it really was this um compromise amongst these different groups of researchers some of whom felt the law as it was was sufficient enough um to ensure ethical research practice and others who who definitely did not feel that way so despite appearing in the Federal Register the Menlo report ultimately governed by this partial Community consensus rather than by law and controversies of both ethics and law and how they apply they still plagued this broader research scientist community okay so we have these five overlapping processes that we argue are constitutive of Ethics governance in the making and as our discussion today hopefully makes clear we also wanted to add the sixth one which is that the memo report now becomes the static document this research this resource that's out in the world that other people can draw on and use but that in the process of sort of laying all of this work down in the document they also created new uncertainties they created um new controversies and they created you know this new resource for everybody to make use of okay so as we've been talking about computer science is definitely in this area of really interesting experimentation with ethics governance in the making across a number of different fields now um researchers and activists are undertaking really interesting projects to revise ethics codes rate new ethics guidance form new committees within computer science conferences and professional associations and to dictate ethics requirements for funding and propose new ethics requirements for publication and so we're starting a new project we would love ideas if you guys have um sort of examples of field sites that we should look at we're looking both contemporarily and historically and our ongoing project we hope is going to document the efficacy of historical current and emergence ethics governance in computer science um you guys thank you for your time and thank you for all of the great questions we have time for I'm really sorry we meant to have time for 15 minutes of questions but you have so many good ones in the middle we have time for three questions three minutes of questions so maybe can I just get all the hands and then we'll try to take them all and answer them quickly yeah good point talking about law it seems like there's Centric like participants in having Homeland Security involved but also trying to set standards for a discipline of transcends the United States I was wondering if the participants talked about that from the continues at all great question any other questions you guys want us to take um so again we have we have interviews from sort of like a broader set of folks and that is definitely a concern outside of the US um is like hey we don't have irbs in our country and you're asking us to you know um report as though in our papers to write as though we have the option of Consulting an IRB about human subjects and we don't um and so one of the things actually in the National academies report that we suggested was that um professional associations maybe develop some of these um institutional resources for folks who maybe don't have access to that at their home institution and you know we don't specifically talk about IRB resources p