Vibe Coding: LLMs Role in Modern Programming

Key Takeaways

in this video I wanted to take a look at the most recent Lex Freedman podcast with the primagen who uh is a very uh popular YouTuber and he does a lot of videos regarding programming and uh this podcast in particular was over five hours in duration uh which is extremely long there was one particular sort of part where I was curious uh which is a very sort of Hot Topic and one that has you know impacts all of us as coders machine learning Engineers Etc which is using uh AI for programming so in this part he they discussed for like 50 minutes um we'll just try to react to this and basically listen to what arguments they have for and against and then let's have like a Nuance discussion on you know is this something we should use um when are the cases that it should be and can be used what are the sort of drawbacks um and just have sort of a a discussion around this so let's just start uh the podcast and let's have a listen to what they say speaking of which let's talk about AI all right you've been both sort of positive and negative on on the role of AI in in the whole programming software engineering experience as a stance today what do you think uh what's your general view about AI uh what is it effective at what is it not so good at okay so my general view is it it comes down to something that's pretty simple which is that if you're doing something in which is very predictable AI is really nice when you're doing something that is just not predictable AI is not very nice to use if you're using anything that's more Cutting Edge AI will not be using it or AI won't be very good at doing stuff with it like it's it's not at Zig because Zig is just like say less documented it's really great at typescript uh I think there's a lot of there's a lot of interesting things that are going to come down through AI that I think a lot of people aren't really prepared for or thinking through uh TJ is kind of the Genesis of this idea but the idea that U I think there's going to be a lot of kind of Market manipulation if you will through AI meaning like hey you want to research say best woodworking tools well someone's going to be buying an ad someone's going to be buying premium tra training data right they're the ones that get the uh the big boost in the llms but llms don't really have to Market as an advertisement because it's not really directly an advertisement they just had a more premium spot per se in the training data a little bit extra learning to it you know it's like there's a lot of things about AI that I I fear upcoming yeah uh a lot of it just comes down to people not uh learning or making the trade-off where productivity is the only thing that matters and I don't think productivity is the only thing that matter matters if you want to build something complex and difficult productivity is not the only thing you actually are going to have to do deep learning and kind of pursue it beyond the basics and so I see AI as kind of like this really cool thing it it feels like a magic trick I remember the first time I used it I got Early Access to GitHub co-pilot Nat in fact Nat fredman saw my twitch clip of me asking GitHub for it and he sent me early access himself it was awesome and when I used it it predicted an if statement correct and that my mind was just absolutely blown because I had nothing before then and now it's just like first time ever and I just remember thinking man this going to change programming so much and then the more I used it the more I just for me personally I kept introducing bugs and I couldn't figure out why and what I realized is that I kind of developed I wasn't co-piloting well I was autop piloting much better and my ability to read code versus my ability to critically think and write code they're definitely different sets of skill levels all right so there's a like a few things that he's brought up here and I think he has some really valid points as well I think the first one is that um AI is really good at doing things that are very predictable I think I would slightly disagree with this sort of statement because um I I personally find that it can do things that are things that I would have never thought of and so it can be used as an inspiration source and I feel like it can sometimes have quite High creativity in how it approaches problems some sometimes I just rerun it multiple times um and you can change things like the temperature Etc and then you can get multiple ideas on you know how it would approach it and then you can sort of um review the options that it gave you so I do think that it is good at things that are not predictable um the second sort of point was that it can sort of lack if it lacks training data on that particular documentation Etc um if it's state-of-the-art that makes complete sense right because usually they have a knowledge cut off of let's say 2023 or something like this and so the most recent libraries uh the most recent updates on Packages Etc uh especially if they're new sort of packages or libraries then they won't have good access right so you have to provide the documentation and do that additional work um but yeah that that is completely valid and I think also another thing is it does feel like magic I am cautious of this tradeoff where he talks about productivity and sort of the learning aspect so if you only care about productivity and you can do things much faster using sort of AI assisted coding but then you reduce your learning so this kind of goes into this idea of of vibe coding which uh is a recent Trend where people you know kind of not use AI as an assistant coding but rather they let the AI code and they're assisting it basically it's like human assisted coding or something that way of coding with Vibe coding I think has high prone ability to introducing bugs and I do think it significantly uh reduces our ability to learn with on the other hand though I think we've kind of done uh Vibe coding for a while now like even before I had uh LMS to assist me with coding I was Googling and I was copy pasting from stack Overflow and I was trying out things I found from GitHub uh it was just a slower version of it right so it's not like this has completely changed the landscape I would say I think Vibe coding if we see it on sort of a graph of popular I think we've increased the frequency a lot but I think as we introduce stack Overflow as we intro introduce uh GitHub Etc I think the level of things we play around with and just copy paste Etc that has always been the case uh or as increased uh since these sort of uh websites Etc has been available and now with LMS I don't consider as well when I just read code as opposed to when I write code and so I I struggled there I do think that's a skill set yeah skill issue for sure skill issue for people who are not aware that's like a hashtag thing sometimes use mockingly in this case there's like a several layers mockingly but also seriously yeah meaning like the criticism is grounded in the fact that you lack the skill versus of some kind of fundamental truth yes I think that uh that's the reason I use actually copilot cursor a lot is for developing the skill of editing AI so I can just learn how to do that better and better because I think as I do that better and better I start to utilize AI better at this time it is a bit of a boiler play code thing mhm uh but you can do out of the box kind of Novel design decisions or tricky design decisions from scratch but fill out stuff uh using uh Ai and then just so one thing I personally use it a lot for as well is the LM has sort of a very uh large breath of knowledge which you don't have so you might be specialized in using very specific things and so it it might not you know it might be um prone to introducing bugs Etc but on the general sense it has a pretty good idea so if we're looking at um you know something that we're not that familiar with of creating some design decisions and these sorts of things then at least it can steer Us in the right direction so what tools can I use for this what are the general principles uh of this of this area right and these are things that you can very easily get a sense of just by you know talking to the LM and getting a sense in areas that you're not the most familiar um and similarly you know even in like let's say python there's so many different libraries and packages you can uh you know talk to the llm for a sense of what exists in this area what are the most popular um packages and then you can use that as a much more educated guess of where you should do your research um and so that's also one case where I think they are you know incredibly useful it's more fun to program with AI even when I delete a lot of the code it's more fun it's uh less lonely it's more it's uh what I imagine like PA programming to be and I've never done it but the it just feels like that uh friction that you get when you're like staring at an empty thing is not there like empty function empty uh empty class it's just more fun less lonely and I do think that a lot of the easier type of coding it really helps with like interacting with apis um basic things that I would usually have to look up the stock overflow for uh it's just really fast at that like yeah as example just interacting with the YouTube API uh the Youtube API documentation is not very good and you can just load it all in there and ask it to generate a uh a set of functions that access the API do all kinds of read and write operations and it figures it all out and then you could just well you do have to read you have to read and check everything and you start to develop the skill of understanding where it misinterpreted the task so you're what is that skill I don't even know you have to kind of be empathic about what the AI is what its limitations are a lot of the times that has to do with um uh prompt engineering you have to like at the same time uh understand what the AI is aware of like what did you actually give it as data to be able to generate the code a lot of times we don't realize that we're not giving it enough information yeah so you have to like actually okay okay all right you have to like be empathic be like okay these are the code so Lex is talking about some different sort of ideas here but I think first of all it's Lex always has these interesting perspectives on you know like it's true that like when coding now going back if I remove the AI assisted part it just feels completely flat and I guess lonely is is a good description of it uh and then um basically that you need to learn as well of you know when is it prone to making mistakes so if you don't give it all information so you know you become wary of you know maybe I haven't given it all information or perhaps it doesn't know this too well right then your trust level of what it's going to generate needs to decrease a lot and you have to be more um you need to review more carefully right um and so there are certain points where you need to learn you know in these situations I need to spend more time reviewing in these situations I can trust it more um so if you're doing something very sort of standard that's been done a million times then you probably don't have to review it right um and I think this is a very valid point you know in addition I think this sort of uh test driven development I think is going to increase um as llms become more and more popular uh and I think I think basically we need to separate so we create the test and then even though we will manually review the code of course to see that it's doing the right thing we need to validate that it works by running the test that we developed so I think U this is something that will become much more popular um you know it is already becoming more popular but it will increase uh a lot more because just because of llms is my sort of uh guess the files it's aware of this is the specifics of the question you asked it like you have to like imagine you're an intern that doesn't know anything else like often times we want to AI just like figure out the things that's left un unspoken but you you can't know those things you have to like specify those things and so you have to actually be much more deliberate and rigorous in the things you specify is to spell it out and so I just have this like C of prompts that I have saved up and I'm building these like library of different templates for prompts and it's a mess and and I'm sure there's a lot of developers that have this similar kind of mess so a lot of it has to do longterm with the tooling that's going to improve that one the systems are going to get much more intelligent where you don't need the nuance and two there's going to be the tooling that allows you to specify those things and load it in correctly and give all the context that the system needs inorder to make the good decisions and maybe the system asks you followup questions wa here's things you didn't make clear all that kind of stuff a lot of that has to do with the interface with the actual design of the tools like we said with cursor it's going to keep getting better and better and better so my sense is like uh developers in general should be learning this to see uh to not be left behind to see what how that can be used uh to Super as a superpower to to boost their productivity their effectiveness their Joy of programming versus like uh be seen as a competitor to them or something like that so but I you know I for me already uh it's been it's it's it's been a big boost to productivity like actual like if you measure the actual how quickly you're able to get a thing done mhm it's been a big and uh measured not across minutes and hours but days also like sometimes there's things I have to do that are not that important that I'll just like out of procrastination will push off yeah and AI helps me actually get it done like actually because like that thing the empty page like I mentioned before it helps me write the thing get it done get it tested like ship the thing maybe it's just because it's just less lonely to work with an AI I don't know I don't know if any of that made sense but it all made perfect sense I really do like that phrase it makes it less lonely I think there's something to that that's kind of interesting having just some level of interaction that's not just like an LSP autocomplete yeah like having something that's actually a little bit more than just that where it actually is kind of thinking through and you can see a different thought and you're like oh wow that's like that's a way different approach than I would have taken hey that's kind of cool I like these kind of things and the thing is I'm not like a AI negative person I I can see why people really really like it um I just haven't like I just every time I I used co-pilot for from when Nat gave me the uh access all the way up until about six months ago like that's how I used it for quite some time and I really I really did enjoy the things I used out of it it just never it kind of did the opposite for me I felt like I was more reviewing than writing and I felt like I was more kind of just letting things slide or I just didn't really think too heavily about stuff and it just I wasn't as engaged and so I'm like okay so something's kind of wrong here and that's just like a me personal thing so I I recognize that is not how someone should approach the things that's not a good reason for why you should or should not use AI like I just don't think that that's right because I could probably correct that and figure out a better way to do it I've been meaning to have another AI round and so I've been thinking about like maybe I just need to spend like two weeks in cursor and just like fully Embrace what does it mean so so I think like these are also really valid points like your skill set of what you used to do versus what you now do more of when you use AI assistant coding it's very different right so like you become much more of a reviewer of code so perhaps what you spent let's say 10 15% doing previously um has now been swapped so the 80% work you did previously is now that 15% right so the skill set becomes more how good are you at quickly reviewing code um and accept reject you know uh and so that's very different it's a very different skill set from actually writing the code uh and I do also resonate with you know that you're not as deeply engaged perhaps um you know if you're writing the code it's a much more challenging it's kind of like this NP complete uh kind of problem where it's like a lot more challenging difficult to generate a solution but then it's quite easy to verify it so kind of the most difficult part of the job becomes lost and so you're not that which required that really deep um in inside sort of um very sort of deep work uh becomes less right because the the job becomes a bit easier I think we just need to sort of change that though so I think our job becomes a bit different where we need to instead become kind of like U all of us becomes kind of like a manager I would say like we need to manage this LM of what what the solution should be right so we should perhaps use that time we get from that to spend more time on you know what are the most efficient Solutions on a general sort of level right what are the what are the ways we can solve this problem and then we stare the direction towards that solution so I think um basically that might be the way we solve how you know we don't do the most sort of challenging um part of generating the code which used to be what we did the most um but yeah I think you know this is a really long they discussed this for 30 minutes longer I think now we're reaching about 20 minutes of this sort of reaction as well I think this is a good time to stop if you want to listen to the full uh I'll link it in the description I think it's an interesting listen um and um yeah what are your thoughts on AI is coding using llms you know tools like cursor um and uh you know there's so many different tools now as well being used um and so what do you use uh what have you found most helpful and one question do you feel that your skill level as a coder uh has deteriorated in any sense since you started using uh LMS to assist you in coding all right uh that's it for this video thanks for listening to this one and I hope to see you in the next one