Copilots Everywhere: Thomas Dohmke and Eugene Yan

Key Takeaways

[Music] I'm delighted and honored to welcome a very special guest Thomas domy Thomas has been fascinated by software development since his childhood in Germany and he's built a career building tools death love and accelerating Innovations at that are changing software development currently Thomas is CEO at GitHub where he has overseen the launch of the first at scale AI developer tool GitHub co-pilot so please join me in welcoming to the stage Thomas [Music] domy thank you Thomas it's out here yeah well thank you everyone thank you Thomas um let's start with co-pilot many people have shared their own takes on the co-pilot origin story so but what was your personal experience seeing it in greub I don't know you have a sneak preview take us back to the start in 2020 so imagine it's 2020 um it's lockdown here in San Francisco and Seattle everywhere where GitHub Engineers are sitting so like all of you probably were on a zoom call um one of us had early access to a new model that um open AI had just um released in preview um a version of GPD 3 called codex and uh you know one had the um uger I think had the keyboard the leader of GitHub NEX at the time and we were dictating prompts and ask the model to uh write some code and I think the first aha moment that I had is that you could ask it to write Java Script code and put the curly braces in the right places and whatnot and you could write it ask it to write python code and the model in a way you know doesn't work like a compiler it doesn't have a syntax tree it doesn't know these things or you could also argue it knows them exactly like we know it so that was probably the first moment um we kept building uh uh we kept exploring the model and then decided we build this autoc completion co-pilot that you know was the first co-pilot and we build it all you know while being remote while being on lockdown so if uh event if your investors tell you today you need to be in in a room and on the front of the Whiteboard um you can innovate if you want to while being while being uh in your home offices around the world um I think the next moment was um that we shipped um a preview to our internal engineers and we call it a staff ship at GitHub and uh The NPS um survey with those Engineers was through the roof I think 72 73 something like that and typical our early stage products especially you know with a large language model and you know all the hallucinations and the UI wasn't really figured out yet uh is is much lower so that that was kind of like a holy moment uh that we had and um as the product then shipped in mid 2021 and you know Co was still going uh uh we we started looking at Telemetry and the team came and says it writes about 25% of the code in in those files where it was enabled and I remember saying don't believe this like your Telemetry is wrong please go back and validate that and turned out you know that was actually right and uh by now it's about half you know the code that's written some language like Java even has have a higher acceptance rate and more lines written and so I think those kind of this journey that we went through over the first two years really was like one one moment after another where we saw uh the future of AI long before um chat gbt actually opened everybody else's mind amazing and now it's available to everyone here as well so I think co-pilot started as an autocom ID and now it's over GitHub I know I have PR boards Etc what do you do to make co-pilot um and integrated across all of GitHub like what are some experiments what work what didn't work I think the first thing is to think about you know what do I do as a leader as the CEO of a company and it's really about constantly reconfiguring our approach um so much of you know the AI world is changing almost daily um there's you know some news uh on the information elsewhere uh every morning and so there is no more a I have a long-term strategy uh I have my features all laid out and as work through the backlog it's really like operating as as agile as possible even as we are you know 3,000 person company as part of you know one of the largest company uh on the planet the second is that we really try to meet you know the developer where they are um we say you know we're not trying to build an AI engineer we're trying to build AI for engineers a human Centric approach you know that's where the name what the name copilot ultimately uh visualizes um but also you know we're trying to make the developers lives better um because we are developers ourselves and um every productivity Improvement we can find ultimately helps us at GitHub to build you know our AI product so that really is the approach like looking at what what can we do next uh to make you know our our work um a little bit easier of building more features for co-pilot you mentioned a great point you trying to meet the developer where they are so for now we've been bringing the AI to the IDE yeah can you are we going to try to bring the developer the ID closer to the AI how are you thinking about that you know the idea of bringing AI into the IDE really into ghost text you know autoc completions was a way of getting around hallucinations um it was a way of saying okay the model is not always going to be perfect but neither are autoc completions right like whether you have Auto completions in your Google Docs or in your email or in your editor in in the old intelligence way as you typing it cannot know what you wanted to type and so you're used to adjusting your typing and then you find this moment when you press the Tab Key and um even without Auto completions so we think about what developers do in the editor while they write code and the best developers write a lot of code before they get stuck and the newbies and and those that rarely write code like I you know get stuck more often and then you you know control tab or command tab into um into your browser and you open um Google or stack Warlow GitHub right and what you do there is you find code and and you argue with other developers and then you copy and paste that code into your editor and you modify that as well so it's kind of like in a way stack Overflow has as many hallucinations as as as the model I might have and not because the answers are bad but because the world is changing so much you know I code a little bit on iPhone projects and Swift and there's always a new Swift version after dddc or new excode version so things have changed of how you use the apis and so it keeps the developer in the flow that really the crucial thing here was we didn't you know in a world you know 10 years ago we probably wouldn't even call this AI we would just call it you know more smarter Auto completion and um the AI piece is not the core piece the core the core feature of copilot that helps developers to stay in the flow to get the job done and not be in this constant distraction between the editor and and the browser that's a great point and I think a few months ago you wrote this post about workspace what was the journey to creating workspace and maybe for folks who don't unfamiliar with it what is workspace yeah so you know you already mentioned autoc completion that's what how we started um in um November 2022 chat GPT happened so early 2023 we added chat and and GPT 4 uh to co-pilot in the IDE as a as a separate um um sidebar window so we have that available and it has Rag and and all the information the contexts available in the IDE but ever since we have been thinking how can we make the developer flow even easier and workspace does exactly that it takes a GitHub issue or just the task and idea that you write down on github.com and it helps you then as part of your code basic repositories to figure out how to implement that change it Bridges from the issue you know from the task description into the PO request into the code and the the magic behind this is that a the human is still in the center so every step of that way you know writing a specification analyzing the current Reaper the current behavior and then using your description to figure out how do you modify this and then writing the plan which sh shows you how to change all the files to the implementation which is the diff view if you will the human can interact can change those bullet points can change the code and um what that really does it it gives you a an a pair programmer that helps you to explore the codebase right because the challenge we all have as Engineers is that as as soon as you get moved onto a new project or you want to you know modify an open source project um or you're just you know coming back from vacation you're trying to remember what in what is implemented where in your thousand plus files that is navigating the code base is the first challenge you have figuring out what's the current behavor havior and what's the new Behavior so you're having an AI native um a co-pilot native developer environment that helps you along that Journey that you naturally also doing in your IDE and that really is the key here it's not about you know building an autonomous agent I'm sure you have heard a lot about that in the last three days it's about building agents that helps us as humans to solve a task and learn along the way as we figure out oh you know there's this test file that I also have to modify if I want to implement this feature I love the point that you mentioned which is not building autonomous agents and also helping the developers so how should non-developers use workspace they can and in fact you know once we announced this um last year at GitHub Universe in November I think the first email we got the feedback was from a program manager or product manager saying this is awesome because now I can uh not only write you know a user story or um a work item I can also see what that would mean to implement in the code base in many ways you know the biggest challenge we have today is can we be as specific as possible when we write down a task you know as product managers or as Engineers ourselves you know often everything is obvious until it is not um and then um you know you you kind of need to size the task right like how long will it take and uh the mythical men month um I think the pragmatic engineer had that a couple of weeks ago is still true most uh most estimates are half as uh half the time that the the drop actually takes and so we really bad at estimating how much time it takes to get uh uh work done whether it's encoding or whether it's building houses or or holes or infrastructure and so um workspace helps you with that as it helps you to figure out what I just described is it actually specific enough to write the code for that or to even figure out what the plan would look like can you share a bit about your vision on how you think we will build and code in natural language and how it help us collaborate better devs and PMs coders and non-coders across languages and across the world for me you know the very first thing which you say natural languages and I have it on my t-shirt here co-pilot speaks your language is because chat these large language models that we're using today in gith up co-pilot and many other AI applications are the same models that are also helping us in chat agents they speak almost any language or any major human language and so whether you um you know want to explore coding in English and you don't understand the concepts of uh you know true false Boolean logic yet or whether you want to learn that in German in Hindi in you know Brazilian Portuguese and and Spanish and Chinese you can do that now and if I you know look at kids today in in school most of them are growing up with mobile phones um you know when you go into a restaurant here in San Francisco on Seattle or elsewhere in the world at night you probably see a family with little kids where the kids have their phone because the parents want to enjoy five minutes on their own and then as then kids grow up you know they see Super Mario or or Minecraft and they get into gaming and that naturally that means how can I create my own game how can I create my own web page copilot enables that and it enables that in the language that the kids grow up with which you know for the majority of the humans of the on the on this planet is not English um so that's number one it democratizes access to technology it also democratizes access for those that don't have parents at home that have a technical background or that don't have parents at home that have infinite patience but most parents do not I have two kids I speak from my own experience at some point you're just done you know with explaining the world to your kids and you just want to you know switch on the TV and uh and watch your watch this Netflix show and and and but that keeps going if you look into the professional context one of the biggest challenge we have is you know if you would join my company or I join your company uh tomorrow the biggest challenge we have is what's all the institutional knowledge how are things being done you know and what we don't like as humans is ask a thousand questions um especially if you're a new employee and in a big company you're like having this anxiety in your head that everybody else thinks you're you're dumb you why did you get hired in the first place so a co-pilot also democratizes access to all the information and companies and I think that is going to be changing how we work and not only for developers in the workforce but for really every human thank you thank you Thomas for sharing your vision I guess the next thing I want to ask is maybe a little bit more unhinged speaking of agents in your opinion what makes an agent or a co-pilot what's the definition what what's your definition of an agent I think an agent you know is like an AI dishwasher um you fill it um with you know the dishes and you let it uh let it do its thing and then at the end you to take the output and you put it back into to the shelves right and today um we have you know we called it used to call it Bots um um you know or cicd in many ways that's an autonomous agent right you push your pull request and you run your cicd GI up actions or or or a similar product and many compute Primitives that we have today are agents as they get a job done on their own and my monitoring you know to figure out if GitHub up or down is somewhat autonomous um hopefully it Pages somebody without us hearing from you that you cannot access repository so you know I think in many ways um uh what we're building is still tools that help us to get the job done and there's many jobs that developers have to get done many jobs that now ai Engineers need to get done you saw on the slide earlier all the things that are also still true you know even though you can automate things with large language models um and a lot of work in software engineering is is bogging us down um a lot of boilerplate a lot of security compliance you know that Friday evening uh when you when you want to you know enjoy the barbecue because the sun is out and instead you have to update all your lock for Jade dependencies right like security Tooling in fact you now is creating more work it's not a dishwasher it's actually a tool that shows you that the tells you that the dishes are dirty and then you have to do the dishes yourself today and so um that's security tooling right it just adds stuff to our backlog while we actually want to work on the creative side and we want to we want to build new features we want to build Innovative product that creative things um I think many software developers do not understand themselves as a production worker understand themselves as artists as creators yes and but you know our companies our governments you know the world is requiring us to do a lot of other work and we need AI tools autofix you know things that that scans uh not only for security issues but then fixes those security issues we need those pieces um supported by AI so we have more time for the things we don't want we do want to do and AI takes over the things we don't want to do and that's that's where the agents will go fantastic what's an agent you want to have and how far are we from it I mean I want to have these agents that burns down all my security backlog um it's um as in any company the the challenge is that I have way too many of these items um and there isn't really a book you can buy um that tells you as an engineering manager of how to balance those two things um you cannot do all the your work into security compliance um accessibility and whatnot you cannot put all your work in into Innovation because your customers will lose all your trust the moment you have a security issue that uh threatens their that data and as such you have to balance those two things or you find AI agent that Springs the work down and I think as any you know leader of a software development company I always want to go faster I always want to get that feature done faster and um I'm sure it's you know the same for you folks at Amazon when uh when I have an idea and I ask my folks how long will it take to implement that the estimate I'm getting is like I'm scratching my head I'm thinking I could does that can do that myself faster than than waiting for for my team to do it but of course that that's not the truth the truth is that there's so many other things in the process these days that um we need to find new abstraction layers um that help us to to get control over our development life cycle again that's a great point so last question do you have any advice for devs both new and experience on how they should they should navigate this new world of tools this new of new world of abstractions um in what some say is the biggest Technology Innovation since the internet I think you know the most exciting thing about this new technology is and you saw it hopefully over the last three days at this conference is that we are moving into a new world of software development and there have been multiple step functions you know over my uh life um I was born right before the PC was invented uh I remember getting my commodor 64 on a PC in the '90s I remember the open source in the internet and you know internet Open Source before the internet was buying CDs and DVDs um in bookstores the internet came you know um Source Forge and then GitHub came all of a sudden developers started collaborating the mobile wave came and every time we had those step functions software development got more exciting and I think you know we are again at that at that step function it means we can embrace our nerditude we can build new new and I think you know the it's really like like you know for me as the CEO of GitHub I don't get to touch code often and so when I get to touch code on a Sunday afternoon I don't want to spend all my time of updating all my okay that's all we had thank you Thomas pleasee join me thank you so much around us that we want to bring the F and the I BR F development I want know how you all back home