The AWS Open Source Strategy

Key Takeaways

[Music] welcome to this special edition of the new stack makers on the road we are here at the open source Summit in Dublin Ireland discussions from the show floor with technologists giving you their expertise and insights to help you with your everyday work Amazon web services is the world's most comprehensive and broadly adopted Cloud platform offering over 175 fully featured services from data centers globally millions of customers trust AWS to power their infrastructure become more agile and lower costs hey everyone here in Dublin Ireland at the open source Summit I have the opportunity to have a conversation today with David Nalley and David is head of Open Source strategy and marketing at AWS David how are you I'm doing great thanks for having me today so I had a few conversations with you today about open source overall we talked about what Paul Vixie discussed in our conversation earlier and Paul is the vice president um of security at AWS and also distinguished engineer and someone who is well known uh for his work over this past several decades in making the internet what it is today but I wanted to get to this idea of what is the core open source strategy for Amazon web services what are those pillars well you know I think it starts with sustainability and it this really goes back to the the origin of Amazon web services AWS would not be what it is today without open source we like everyone else build and innovate a top open source and that means that there's uh you know a long history of US benefiting from open source and investing in open source but ultimately we're really here for the Long Haul we're going to continue making Investments we're going to increase our investments in open source because it's it's important to us but not not just to us it's also important to our customers so that's number one yeah that's number one I kind of hinted at number two there right which is that it's not just important to us but it's important to our customers and we feel like we have to uh to make investments on behalf of our customers but the reality is our customers are choosing open source to run their workloads on and whether that's open source databases or open source AIML platforms that's what they tend to to be focusing on they're choosing that repeatedly and so because that's important to our customers we also have to to go take a uh an investment there make sure that that runs incredibly well for our customers uh one of the things we saw news earlier about the pi torch Foundation a couple of days ago launching and Pie torch moving from a single vendor to a vendor neutral governance one of the Investments we made over the past few years is to invest in making sure that Pi torch runs really well on the arm architecture and and so we're making Investments like that because those workloads are important to our customers and I think that's that's part two part three is uh you know we've got we've got a space where we have to advocate for not just the the what's allowing us to innovate but what's going to be the future for uh for for the rest of the world in terms of Open Source and uh and open source gives us tremendous opportunity to get involved a tremendous opportunity to leverage some of the strengths of AWS and bring some of those resources to bear and deliver on that to improve the entire environment that we're operating in so I heard you talk about sustainability and how that's part of your open source heritage and how we seen an investment in open source buy Amazon web services since it's very beginning I heard about another theme I I've heard from AWS over the years and that's uh support for the customer uh and what does the customer want and the third part I heard about is advocacy and how important that is there's also a pillar I'm curious about and that's how internally important it is to your own developer teams yeah you know again AWS just wouldn't exist today without open source we're building a top open source all the time whether that's directly or we're using uh you know foundational pieces of the open source infrastructure we're looking at open source as being able to stand on the shoulders of giants yeah to deliver so much faster but what about the Upstream participation so we're doing a lot of interesting things in in a number of upstreams and again a lot of our Upstream investment is dictated by what is being seen as important to our customers but tell you about a couple of things that really have me excited today we have built Atop The Linux kernel and built a very small environment container runtime for uh that's really stripped down and and has a lot of memory safety controls in terms of uh in terms of rust tooling and what's that that that's called bottle rocket yeah okay and and you know we built that originally because we needed it yeah uh turns out it was also very useful to our customers we released that as open source and people are doing uh lots of interesting things and it turns out doesn't even require AWS lots of people are doing interesting things with it because it is it minimizes the attack surface really well but that's us releasing open source and and we love doing that we've got a number of things that uh that we've released over the years but we're also making investments in in upstream and so we've got uh you know the traditional methods of support where we're supporting uh Upstream foundations and Upstream projects with money and Cloud credits but we're also making significant contributions so I mentioned a few moments ago about improving arm support in pi torch we've got dedicated Upstream teams now for the rust programming language that are working on improving the language itself and the ecosystem around it we've got folks who are working on Jupiter notebooks and you know their sole Focus every single day is to wake up and improve only the open source uh project uh we we've got a team focused on Kafka now and are doing a lot of things in the Apache Kafka community to improve that code base what's more exciting to me is it's not just those things but we're I'm hearing about a lot of teams that are starting to hire and are building up dedicated focused open source responsibilities so that they can spend all of their working day contributing Upstream that's amazing people love that don't they well certainly the folks who are hired into it love that they can go contribute back to something bigger than themselves they love that we can have actual impact for our customers and even for folks who are not yet our customers uh you know we can deliver improvements to those open source projects and we're also doing you know doing well by AWS because we're we are delivering Innovation uh even though we're delivering it to the Upstream project it gives you the opportunity to work with peers too I mean you probably have friends who've like moved around to all kinds of companies over the years and you get to and open source allows you to work with them in in a way that almost transcends uh the organization who you're with and and of course like you know you you work at AWS you're very dedicated AWS but you also get to work with people from Facebook or or Google for that matter or any of the other companies out there that are contributing yeah you know so one of the things that really excited me uh I learned about this yesterday um one of our teams over in redshift saw that one of our customers had built an open source tool to ingest data much more rapidly from from specific data sources into redshift and they were excited about it they went out and talked to that customer they're now collaborating with that customer to improve that open source tool and it's uh it's a tool for ingesting and shuttling data back and forth between Apache spark and and redshift and so not only are we getting to work with peers but we're getting to work with customers where they are building tooling and and it doesn't have to come from AWS Sometimes the best idea comes from one of our customers and and we get to go help them with that and help them out in the public where not just that one customer benefits but everybody in that environment can you know I had a nice chat with Paul vixi earlier and uh this and one of the things you said in you know we were chatting earlier makes me think of kind of what the work that you're doing now and Paul did work in the 90s you know and over the you know and since then that really defined a lot of the protocols that we depend upon today DNS for example and one of the things that you you were talking about before was that allows you to work on that next two to three percent those next layers up what are those next layers up for you in many ways I think we have traversed so incredibly far and yet we're we're still seeing lots of opportunity for innovation for me database and analytics uh is a is a huge burgeoning sector we're seeing a lot of uh innovation in how folks handle store ingest and make useful the data that they have uh the next step in that Evolution I think is is AIML turning that data into something useful and and doing so at scale and so you know we talked earlier about the pi torch foundation and I think there's lots of interesting things going there but I think you're right you know one of the one of the beautiful things about open source is that it stops me from having to do the boring stuff and get to focus on the most valuable things because no one wants to rewrite a web server so Maybe in the last few minutes we can talk about the role of security then too because data insecurity are in are are bound together forever right so open ssf is uh had a big uh um event here earlier in the week and six store for example is a popular project um that we're starting to see a merge uh you know your commitment has been to make a big investment in openness ASAP uh and uh so I'm curious on like you know considering Paul's background considering your investment in open ssf what priority does security have especially when you're thinking about the open summer supply chain at AWS Securities job zero it is the first place thing uh if we don't have security uh you know that is table stakes and we have to have that in place so when we think about supply chain security it's obviously very important we've been trying to Signal the level of importance that we think we we made earlier uh this year a 10 million dollar commitment to open ssf which is which is one of the things that we're doing in that particular vein but supply chain security affects not just us but it affects our customers and the workloads that they may be running as well so you know we've talked about a number of different Avenues openssf is certainly one of them we've got a number of teams inside AWS security I'm sure Paul talked about that are focused on securing the the open source supply chain from a software perspective and figuring out how we can move faster there to drive more value not just for ourselves but also our customers and the rest of the the internet with Paul aboard that takes on a different realm doesn't it you know to start I'm really kind of uh in awe of Paul I mean he he built a lot of what we know as the the DNS system today uh and our domain name system and I kind of feel like uh he is this rock star and I'm super excited that he is working on security one of the things that really amazes me is that he's one of those folks who very early on back in back in the 90s uh he had software that was ubiquitous it was on every device it was consumed everywhere and we're starting to realize that some other projects have that but very few have the kind of ubiquitous uh deployment that some of the DNS tooling and KRON tooling that he was developing has had over the years and so he has experienced these problems firsthand he understands the the supply chain um concerns and constraints and I'm super excited to see folks like him working to solve that problem well thank you so much for your time David really fun to talk with you about these pillars that uh AWS really believes in around open source the interview with Paul I think in your Reflections on the work that Paul has done and the ability to build on that the next two to three percent is really we're going to look forward to learning more about uh from AWS in the next year or two thanks so much for having me it's been a pleasure to chat thanks David if you like this video please give us a thumbs up and if you'd like to see more videos like this you can always subscribe to our YouTube channel we're on all the major social media platforms you can always find us at the newestack.io we hope to see you soon [Music]