I Quit My GitHub Job Because AI Breaks Software

Key Takeaways

Today, I resigned from GitHub after spending almost 4 years there and growing to senior engineer. I left to help improve AI safety because there is a dangerous trend in how AI coding agents are being deployed right now, and this is going to affect you personally. I'm going to break down what I observed, why it made me walk away, and what I'm doing about this problem. So, first of all, I didn't leave GitHub because I hated my job. It was genuinely the best job I've had so far. I built the first version of newer, respectful AI support systems with my team, helped other engineers across the org figure out how to build mature LM solutions, and those experiences shaped who I am as an engineer today. I've also experienced the birth of agentic coding from more traditional AI auto-completes to newer agentic systems like Claude Code, GitHub Copilot, and others. Now, we as programmers and non-programmers experience the benefits of agentic coding daily, and it has been a massive productivity boost. But over the past year, I started noticing an extremely dangerous trend across the wider industry that I just couldn't ignore anymore. Companies are not just using AI for code generation, which I'm a huge fan of, just like many of you. They are now replacing entire parts of the software development life cycle with AI agents. Review, testing, deployment decisions, and even architectural choices are being handed to agents with less and less human oversight. And I'm not only talking about small startups here that can afford this risk. The industry pressure has become so high that companies with millions of users are shipping code that a human barely touches before it goes into production. Now, before you misinterpret me, I actually just want to say that at GitHub, I found there to still be a lot of respect for good, high-quality software every single day. It's just that the industry trend overall is causing software quality to drop quickly. There already projects and platforms trying to test the limits of how far we can push coding agents to automate everything. And the direction here is clear. Some organizations are starting to point at pull requests as an annoying bottleneck that slows down their AI-generated output. Now, think about that for just a second. The one remaining checkpoints where a human looks at the code before it reaches your users, and the industry is calling that a bottleneck. The ultimate step here seems to be to displace human review altogether. So, why does this problem make me quit my great job at GitHub? Well, it has been one of the best jobs I've had, and I will easily admit this. AI that I've used at my job very often writes better code than me, and it outputs code faster than me. One of the ways this has excited executives in the tech industry is that it promises that every developer can be a 10x or even a 100x developer because 10x is not enough anymore. Now, before we get too enthusiastic, think about what that actually means for a moment. If one developer can generate a hundred times the code, they can no longer meaningfully review what was produced. And I'm noticing this every single day as I work as an engineer. Instead of the industry seeing this as a problem, the collective response has been that agents should just take care of reviewing code as well. The critical issue with this is that AI writes better code often, as I just said. But, it does not always write better code. It will continue to hallucinate. It will continue to make mistakes, and when tasks are long and complex enough, it will create incomplete solutions. But it does always write more code than me, way, way more code. In a normal work day in the past, I might have had days where I wrote a few hundred lines of code. Now it's easy to write thousands of lines in a single session. So, what happens when you have an imperfect system generating enormous amounts of almost perfect code? You get a statistical guarantee that there will be bugs in the system that you as a human do not have the capacity to check for anymore. This is not a gut feeling. It is a mathematical fact. And the industry's current response to this problem is to just throw more agents at it. Oh, let's just evaluate agent output with more agents with just slightly different system problems. Let's write automated tests with agents. Let's let agents test software end-to-end. Stack more AI calls with slightly different prompts on top of each other and just hope for the best. Now, in my brutal opinion, this is the software industry's equivalent of just playing bull doll house and pretending that there is a solution to this problem. Because we are currently trusting the same type of technology that's creating the mistakes to catch its own mistakes. Imperfect systems stacked on imperfect systems do not converge to perfection. The industry right now is actively choosing to prioritize raw code over everything else. The amount of code generated can no longer be stopped by humans. And this is not because of some fictional rogue AI like you might read in some fiction books. No, this is just because the industry is letting this happen on purpose as part of a strategic decision. Now, unlike many statements made in the AI space, which are often based on hype or ulterior motives, I'm confident in stating the following. The future deployment of the fully autonomous AI software development life cycle will lead to an absolute increase in broken software. Now, I want to be clear about something because you might be thinking, "Zen, do you think that we should be stopping AI coding altogether?" No, I'm not saying that we need to stop the point coding agents because this is highly unrealistic and I'm quite progressive. And honestly, I never want to go back to manually writing all my code myself anymore. And to be honest, some drop in quality is acceptable because some bugs just don't matter that much. Even in professional settings, not everything needs to be perfect. Nobody is going to get hurt if, say, a support portal button breaks in an edge case condition. In general, being able to ship away more and break a few small things along the way is a net win for the majority of software out there. And I also understand that five coding is awesome. It's very cool to one-shot your favorite childhood game. And that as a non-programmer, you can get into software easier nowadays. It's very nice. The problem is that the push to use these agents is so pervasive across every industry that bugs will inevitably appear in software where we cannot afford it. In healthcare systems, on financial platforms, and infrastructure that millions of people depend on every day. And without fundamentally new forms of monitoring, AI coding agents will cause more harm than good in these kinds of critical systems. So, it's a big problem, but what am I doing about it? Because it's easy to complain from the sidelines, right? Well, that is what made me resign from GitHub. It is very easy to just complain about problems that AI is causing. But if an opportunity presents itself to actually contribute to a possible solution, it is only right to try it, even if it can be difficult. I happen to find an opportunity at a research lab to work on exactly this problem. A new role building monitoring systems specifically designed for AI coding agents. Now next week, I'll share the full details on this new role on my LinkedIn. So, if you want to be the first to know, connect with me using the link in the description down below. But I'm also sharing the story because 4 years ago, I was turning out to be junior and now I get to work on problems that I think genuinely matter for the wider industry. And this shows that it is still possible to get a high-paid career while making a positive change or at least trying to do so and also genuinely loving your job. And this is also why I push and help others towards their own ideal AI career. As a way to give back, I'll be happy to answer any questions you have about the path that got me here. Make sure to leave a comment down below and connect with me on LinkedIn.