CrypTen - Laurens van der Maaten

Key Takeaways

hi everyone my name is Liana Madsen I'm a research scientist at Facebook AI research and I'm going to tell you a little bit more about secure multi-party competition following up on on Andrew stock secure multi-party computation is a truly exciting technique because it brings the potential to create all the values that machine learning can creates while maintaining the privacy of data and while making sure that the data cannot be compromised as Andrew just explains this may allow is to solve important problems using machine learning that we cannot solve today these techniques are part of a larger collection of secure computation techniques that also include homomorphic encryption and trusted execution environments there have been a lot of developments in these techniques over the past couple of years which led us to ask ourselves the question what would is secure and privacy-preserving Pytor to look like this is a difficult question what does it even mean for a software platform like pi towards to be secure or privacy-preserving could we have some kind of global flag that set torch that private is true and then magically all your fighters computations are performed in in a secure way we're not there yet but over the last several months a team of engineers and researchers at Facebook has worked very hard on delivering some first answers to this question in a project that we call Krypton I'm very excited to share some of our first results with you we're open sourcing Krypton today so that everybody can build on the work that our team has done so what is Krypton Krypton is a platform for researching machine learning using secure computation techniques it aims to enable machine learning researchers who are not cryptography experts to experiment with machine learning models using secure computing techniques and to get a realistic view of what is possible what is difficult how efficient these techniques are etc Krypton leverages and integrates with PI torch and closely follows the PI torch design principles and API we hope that this lowers the barrier to entry for machine learning researchers and the who were already familiar with pi torch in Krypton we've adopted the following three design principles Krypton is machine learning first we design it to expose complex cryptographic techniques such as secure NPC in an API that is very familiar to machine learning researchers that use PI torch because it's essentially the same API Krypton uses eager execution we closely followed by torches design philosophy with an imperative programming style this makes debugging and learning about the underlying techniques easier and Krypton is realistic each party in the multi-party computation runs in its own process and if you want on its own machine all communication is real and it's performed using a byte or distributed backends there are no shortcuts the core primitive in Krypton is the Krypton sir it is an object that looks just like a PI torque sensor but it's encrypted it's provably impossible for a single party to inspect the content of the tensor unless all parties unanimously agree that the tensor can be revealed publicly which is done in this example by calling the get plaintext function yet this is not stopping the crypt answer from performing computations for example we can add to crypt answers which will result in another correct answer that contains the encrypted result of the addition note that in this process none of the encrypted contents was ever revealed to any of the parties we can also add regular PI torched answers to crypt answers so you can mix and match encrypted and unencrypted data the sum of a crypt answer and a float answer is a crypt answer and again the encrypted content is never revealed to any of the parties under the hoods the crypt answer implements a lot of complex cryptographic machinery in particular it provides full implementations of arithmetic and XOR secret sharing and tools that convert between these two types of secret sharing together this allows Krypton to implement a large number of at operations I will get to that in a sec communication between the parties involved in secret sharing is real and efficient as its implemented via torts of distributors with the glue beckons the Crypt answer Repsol this cryptography in a tenser object that looks just like a regular byte or sensor which makes it easy for machine learn machine learning researchers to use without having to understand the complexities about Galois field stealing with reference or understanding when to synchronize after communications krypton supports a large number of operations we support all linear operations including convolutions we are assuming a curious but honest security model in which a trusted third party provides random beaver triples to facilitate the computation krypton implements various efficient ways of computing integer powers it can also compute square roots and non integer powers in the logger ripping domain it can do this because it implements operations such as logarithms Exponential's and reciprocals via efficient approximations Exponential's are performed using a limit approximation logarithms using householder iterations and reciprocals using Newton represents reps and iterations this allows Krypton to implement virtually every operation you may be using in pi torch as well including sigmoids soft maxes binary and multi-class logistic loss functions as well as their gradients we also support operations such as computing minimum and maximum values are Max's signs values and other operations that require us to compare to encrypted values this may sound easy but it's actually very difficult in an encrypted worlds because the bits that indicates which of the two values is larger than the other cannot become known by any of the parties we achieve this through an implantation that converts between arithmetic and XOR secret sharing as needed not only does Krypton support a very large collection of operations it also supports these for an arbitrary number of parties because Krypton's support for operations is almost the same as that of pi torch itself you can actually take complex by George models and these models and run them on encrypted inputs without much effort in this example we set up the image net data sets take a pre trained ResNet 18 from torch vision import that model into Krypton to encrypt it and run it it on the encrypted image net images if we open up the encrypted output of the model we find that the output is just what you would expect Krypton is not only support encrypted inference but also encrypt the training to make training easy to do Krypton includes an autocrat implementation that works just like you've come to expect from PI torch this example shows how it works the current version of Krypton still has a separate autocrat crypt answer which is sort of like the variable object that used to exist in early versions of of by torch in the next release of Krypton we will merge the two so that every crypt answer is autograph alright a little box apart from that the autocrat in Krypton works just as you would expect you call the backward function on a crypt sensor and it performs the back propagation gradients get stored in the grad field of the tensor just like in PI torch the only difference is that these gradients are encrypted - together with the encrypted models I showed you on the previous slides this provides you all the tools you need to train encrypted models on encrypted data to make it easy for everyone to get started with Krypton we provide a bunch of examples and tutorials these examples and tutorials include encrypted inference of rest nets and encrypted training of Linette models linear SVM's and contextual bandits they also go in-depth on some of the limitations that results from doing encrypted computation we want to emphasize that Krypton is not a production ready platform right now it provides realistic estimates of the amount of computation and communication that machine learning via secure NPC requires but it's still a proof-of-concept implementation our work on Krypton isn't only benefitting future users of Krypton but it's benefiting all of you as part of the Krypton project we've teamed teamed up with the core PI church team to prove the parts of my torch that are most importance when you're working with advanced cryptographic algorithms namely gala fields or as we call them in pi torch lands long tensors let's face it long tensors haven't received a lot of love so in part 1.3 we've added a lot of support for more long tensor operations yes you can now convolve two long tensors we've also made common long tensor operations a lot faster than they were before the current release of krypton focuses on secure multi-party computation Google will not stop there we already have a working prototype of a crypt answer that is backed by homomorphic encryption it's not ready for release yet but we hope to be able to share with you in the coming months over time we also plan to add a crypt sensor that is backed by a trusted execution environments via implementations based on platforms like Intel HDX to be clear we're not done yet you cannot yet take Krypton and train a bird model from scratch but we hope that Krypton is the start of a journey that at the end of the road will make secure and encrypted machine learning just as easy as by torch has made regular machine learning and we want to invite all of you to join us on this journey by giving us feedback by contributing to the Krypton codebase or by developing awesome new applications on top of Krypton we're excited to see what you come up with thank you [Music] okay last session of the day so we're gonna switch