Building Responsible AI at PyTorch

Key Takeaways

[Music] hi everyone welcome to pytorch 2021 hackathon my name is jyoti nukula and i'm a product manager on pytorch team for responsible ai i'm going to talk about responsible ai category for the hackathon and give you some tips and ideas for building your own projects and i can't wait to see what you'll build this year so here's what i'll cover today we'll look at what do we mean by responsible ai and why does responsible ai matter and building responsible ai projects at the hackathon some ideas and and some questions that you can ponder about we'll discuss last year's winners and we'll talk about the ideas for responsibility projects so what is responsible ai responsible ai is a broad area that covers a number of different topics we think about rai across the following categories fairness and inclusion privacy and security transparency and control robustness and safety and governance and accountability responsible ai is all about providing equitable outcomes for subgroups it's about giving users the ability to control their experiences helping them understand why a model behaves the way it does and keeping information safe and implementing appropriate checkpoints to meet regulatory standards these are some characteristics of responsibility but why does it matter in the last decade ai is increasingly used to make decisions that affect people's lives in recent years we have seen several examples where decisions that have been traditionally made by humans are now being made by algorithms from helping determine who is hired or fired or who is granted a loan or how long an individual spends in prison these decisions are as you know non-trivial and have a consequential impact on the world and the lives of the people since ai has started to play a critical role legislation is dictating that ai no longer be a black box new legislation such as gdpr and california consumer privacy act mandates products and experiences to incorporate privacy preservation and explainability to be legally compliant and these are no longer considered as nice to haves in a product or experience and if you think about it it's changing how the world consumes and builds ai and this makes a critical direction for us to think about in python as well so here are some examples of responsible ai in the community there are many other examples in the community out there but here are a few that i'm going to talk about today coherent api provides access to models that read billions of web pages and learn to understand the meaning sentiment and intent of the words used it anticipates and accounts for risks during the development process by running adversarial attacks filtering data for harmful texts and measuring models against safety research benchmarks another company fiddler enables users to access deep model level actionable insights to understand explain monitor and analyze their ai models in production rai tools like counterfeit interpret ml and fair learn in microsoft azure help developers implement these tools in their workflow as they go about their processes finally responsible ai toolkit in tensorflow works across every step of the workflow right from problem definition data preparation training evaluation and deployment so what are some questions that you could think about for these tools to answer one question could be is my model biasing on a specific group of people based on their race income sexuality nationality or limited body abilities other question could be do i know if my data sets for training and validation are inclusive or is my model development process secure so it doesn't put personal information at risk do i understand the social impact of my model inferences and lastly is there a way to visualize my model to stop any buyer's inferences and these are just a few questions there may be many other questions that you could ponder about let's look at last year's winners cossing won the third place last year as a multivariate graphical analysis tool to help interpret the causal effects of a given equation system fluence a pie touch-based deep learning library for low resource nlp research and robustness won the second place and finally fair torch provides tools to mitigate inequalities in deep learning and won the first place last year a unique feature of this tool is that you can add a fairness constraint to your model by simply adding a few lines of code as you can see the projects last year ranged across diverse topics and a wide range of problem areas now we'll speak to narenik hoplikian an expert within the responsible ai community to give you more ideas for projects to build narine thanks for joining us today it's great to have you can you introduce yourself to the participants hi jody thank you very much for inviting me to this hackathon station i'm happy to help i'm a research scientist at facebook i have been working on various different responsible ai projects last projects last two three years more specifically i've been working on model interpretability robustness and model debugging great so let's dive in then um i have some questions i'll ask you that we likely to get from hackathon participants and if you are a hackathon participant and you have any other questions that i didn't cover today please feel free to ask them in the comments or reach out to your hackathon organizer i'll also add a few links to more resources in the comments as well so i'll ask you a few questions to give our participants insights and ideas on projects that they can build for the responsible ai track first let me start with a broad one responsible ai means many things and it also means many things to many different people so what does responsible ai mean to you that's a great question jody as you mentioned responsible ai is a broad notion which consists of several areas sub-areas addressing ethical legal security privacy and transparency aspects of ai i'll go into details into detail for some of those sub areas to make sure that we understand them so the first area is a transparency and explainability this area ensures that our models are self-explanatory or their decisions can be explained pause talk by explainability algorithms the second one is fairness fairness ensures that our models are inclusive and make fair decisions regardless of our race ethnicity gender and other sensitive attributes privacy preserving ml ensures privacy of user data during training and inference and more specifically it includes for example differential privacy federated learning or learning on encrypted data and the last section that i'd like to mention is robustness and safety robustness and safety ensures that our models are safe to be used in our products and aren't prone to any adversarial attacks and perturbations thanks that it's very helpful for breaking that down into these different themes um considering the area itself is so broad it's helpful to look at it from these themed perspectives um i also noticed that each uh each of these themes don't stand always on its own that there are some interconnectedness between these themes um like for example you need to understand why your model is behaving the way it is behaving um or why it's giving the prediction that it is giving to probably address some fairness concerns uh i'd love to hear your take on the interconnectedness of these uh themes and how participants can leverage yeah absolutely so these areas are interconnected and as you mentioned one area can help help to solve problems in another area for example interpretability and explainability algorithms can help us to better understand the fairness and the bias issues and also robustness is related to interpretability as well for example the quality of our robustness of our explanations depend depends how robust our models are yeah so that's good to learn that your view of responsible ai matches how we think about it here as well and so maybe the next question i have is what are some good examples of responsible ai projects that you're seeing out there right now yeah so there are many responsible ai projects out there and many open source projects for example there for model explainability there is interpret ml there is uh captain for robustness there are various different open source libraries that help us to conduct adversarial attacks and also defense there is for example opecus project that helps us to perform privacy preserving ml for our pie torch models there is fairness flow and fair learn for the fairness projects and i think some of those exist currently in our python ecosystem as well um like captain and alpacas and fair learn um exist in the pytorch ecosystem and so for participants this is a great way to start with something that already exists and build on top of that rather than starting from scratch now coming to the next question what are the qualities of a good responsible ai project according to you yeah that's that's an excellent question so um so some good qualities of responsible ai project include in innovation so we want something innovative that is not too complex but helps us to solve specific outstanding problem in the area of responsible ai it could be any sub-area such as explainability fairness privacy robustness or safety so good qualities also include a clear defined problem statement um simple and elegant solutions the solution doesn't have to be complex or convoluted or implement a very complex mathematical equation it could be something relatively simple from cod quality perspectives um it's good to have something that is easy to use it's well documented tested and and provide some examples tutorials also future directions how this specific project can be used in any ai application to solve responsible ai problems that's great to know for our participants on how to approach something as broad as responsible ai what in your opinion would be a good recommendation for hackathon participants to approach responsibility especially considering we have many student participants this year yeah so um so as i mentioned it's very important to have clearly defined problem in mind i would recommend to choose a specific area of responsible ai because responsibly is so broad we would need to start with something um specific that we have in mind for a specific model and data sets that we are they are the participants are passionate about um also in advanced literature review um and exploring existing pi torch model can help the participants to understand the gaps um in existing tools and outstanding problems and also i would encourage the participants to use existing libraries such as captain orbicus and do not reinvent something that already exists but build on top of those those are some great books those are some great tips thank you uh we may also have participants who want to try a project in this category but they don't know where to start and so what are some major problems or problem areas or challenges that you're seeing right now in the world that they could tackle in as part of this hackathon yeah so responsible ai has many problems that could be tackled and we see many papers in conferences many of open source projects but still there are many open outstanding problems that the participants can solve and i will mention couple of them in specific areas of responsible ai and that could be interesting but i would encourage the participants also to come up with their own ideas um and follow their the direction that they are more passionate about so in terms of explainability and interpretability it would be great to see self um explainable model architectures and use state-of-the-art approaches for explaining black box models and think of how those explanations could be used in a creative way to better understand our models and our model decision boundaries um also thinking of a tools and techniques that could help us to better debug and understand model predictions um specifically the misclassified examples um it would be also interesting to look into robustness and different robustness metrics and see how these different robustness metrics could work in combination with different adversarial attacks in terms of fairness participants could uh pick a problem where there is in the data set we have protected attributes present such as skin color gender race and try to come up with the detection and mitigation techniques for those data sets and models that could be for example toxicity model and in terms of privacy preserving ml the participants can think of coming up with end-to-end privacy-preserving training and inference pipelines that leverage existing libraries such as opaquers and propose innovative solutions on top of it those are some great problem areas for our participants to tackle and also looking back at all the things that we have discussed being specific being creative and being simple in the solution would be a guiding factor in how they could approach the responsible ai projects thank you so much for uh being with us today and sharing all your knowledge with us um and participating in this conversation it's my pleasure thank you very much for inviting me jody and i wish good luck to participants thank you thank you bye bye thank you so much for joining today i'm really looking forward to seeing your projects in this category good luck on the hackathon you