Harvard CS50 prof David J. Malan on why you should learn programming slowly [Podcast #197]

Key Takeaways

Welcome back to the Free Code Camp podcast. I'm Quincy Larson, teacher and founder of free codecamp.org. And today we're talking with Harvard computer science professor and CS50 instructor David Men. He's taught millions of people programming and this is going to be quite an interview. Let's first take a look at some quick community news. Uh, Free Code Camp just published a one-hour course that will teach you how cryptography works and how developers use it to secure both data and communications. Free Code Camp published this course to teach you Python functions for symmetric and asymmetric encryption. You'll learn about Shaw 256, AES, RSA, and public private keys as well. You'll even code your own command line cryptography tool. It's a 1-hour course on the Free Code Camp YouTube channel. As with everything we do, totally free. Check it out after you finish listening to this podcast. Free Coke Camp also published a course on building your own 3D games that run in a browser using 3JS and Blender. I am so hyped about 3JS. I have loved this technology ever since I first saw it. You'll learn how to model characters, design levels, detect collisions, and make the camera follow your playable character. You'll even deploy your own game to the cloud so your friends can play it right in their browsers. It's a six-hour YouTube course. We also published a course that will teach you advanced JavaScript architecture. This is a fulllength handbook that you can bookmark for your own reference that you can read. It's got tons of code examples. You're going to learn about event loops, task cues, call stacks, back pressure, websockets, pub sub models, and more. Take your full stack development skills to the next level. And be sure to share this with your developer friends. This is going to be a great reference for you going forward. Free Coke Camp also published our first ever guitar course. That's right. We're doing music courses. You'll learn beginner music theory concepts like chords and scales. Then you'll map them to the guitar fretboard. You'll also learn guitar specific techniques like bar chords. I learned guitar during the pandemic and I am having an absolute blast playing it and I hope you will too. Then I wanted to share with you this year's uh JS13K winner. So JS 13K is a competition where you try to code a game that fits in 13 kilobytes of JavaScript. That's not a lot of space. You got to get the sound, the assets, everything. And so this game is called uh I I actually can't remember the name and I didn't write it down, but it is a cat stealth game in the browser. I've got a link to it in the description. and you're a cat sneaking around trying to get revenge for your master who has been executed by one of these hoodlams. And uh you sneak around, you can like meow to distract them. It's very Metal Gear Solidish. And it's just really fun. It's a quick game. I played it for like 20 minutes and got to level nine. I couldn't get past level 9. If you get past level 9, comment if you beat the entire game, tell us your high score in terms of speed and number of deaths. Uh this is one of those games where you can die a bunch of times and uh just keep coming back and it's just about getting better. But it's a great game and it fits in 13 kilobytes of JavaScript. Insane. Other thing I want to do is share this awesome song with you. The song of the week is 1988's Kiss Like Judas by UK band It Bites. I love the interplay of the FM synthesizers, the guitar slashes. I also love the steady off-kilter guitar arpeggios throughout the verses and even the vocal harmonies. Listen to this one with headphones. Again, I've got a link to it in the description. Check it. Then, I wanted to pitch the Free Code Camp shirt. It is made right here in the States by Raphael uh Hernandez, who hosts the Free Code Camp podcast, and he also has his own screen printing business. And we will ship it to you anywhere in the States. If you're outside the States and you want to get one of these highquality tribal shirts, uh tough. We We don't do international shipping anymore. It's just too unreliable and too costly. Uh but you can just take the logo from uh one of our assets directories and you can print your own shirts if you'd like for non-commercial use just for your own personal use. But these shirts are 20 bucks. Super reasonable. I wear one almost every day. I've washed it hundreds of times and it just never wears out. Uh it will eventually wear out I guess if you wash it like tens of hundreds of times which would mean thousands of times. So let's jump to our interview. Very hyped to have Dr. David J. Mail on the podcast again. I had him on in 2019, so six years ago. Uh over the past decade, millions of people have taken his CS50 course over at Harvard, both in person and online. He joins us to talk about why he still recommends learning the C programming language in 2026. uh how he intentionally nerves nerfs his students coding editors and LLMs to help them learn fundamentals faster, his vision for self-paced learning and how it improves on traditional university education and where he thinks the software engineering field is heading in light of recent AI tool improvements. David Mail, welcome to the Free Code Camp podcast. >> Thanks so much for having me. Yeah, I just want to dive right into AI because this is the big thing that everybody's talking about right now, grappling with in both education and in software development. >> You uh as part of CS50 encourage students to disable a lot of code editor features like auto spacing, tab completion, and essentially take away a lot of the power of the code editor. Why do you encourage students to do this? >> I would say it's stronger than that. It's not so much encourage as it is we require them to turn these things off or at least we do so for them by default in CS50's programming environment which is essentially a cloud-based version of VS Code. It's specifically to enable them and empower them to develop the muscle memory that so many of us after programming for years take for granted. But in those first few days, first few weeks, even months or years of learning how to program, I do think there's value in actually having to remember some of that syntax and get into the habit of creating it yourself. And then these tools like autocomplete and the like really are meant to be productivity boosters once you know what you're actually doing. And so we do for instance toward the end of CS50 when we get to the world of HTML have students uh allowed to uh have those tags automatically generated for them. If they open the tag it'll be closed for them because that's partic not particularly intellectually interesting to be typing really the same thing you just typed to open the tag. Um, but by the end of the semesters too, students are welcome to offboard themselves from what we call csf.dev, which is our cloud-based environment and turn on anything and everything of interest to them for their final project. >> Okay. And when you do software development yourself, to what extent do you use uh LLMs? >> Oh, LLMs increasingly. I mean, I have a conversation with CHPT about one thing or another. Probably most every day nowadays. been experimenting uh all the more with GitHub copilot which is one of the in uh IDE uh AIS that you can use nowadays uh experimented the other day with claude code which is another popular one. So I think my own usage is only going to go up and up and up just as it is for us in the course. Yeah, and of course you just emphasized that people need to learn how to do things properly manually before they start. What I often say is you need to learn how to use your basic tools, your your hammers and your saws before you break out, you know, the power tools, the chainsaws, the jackhammers and stuff. And uh it sounds like you're thinking the same thing in terms of uh people who are coming fresh to programming that they should learn how they should learn like the basics. Uh one question I have for you is how important is it to learn all the data structures, algorithms, the things that are traditionally tested in job interviews? Uh you hear people talking about like completing a lot of leak code type challenges. How important do you think that is going to be right now and going forward with uh LLM being able to produce boilerplate, you know, code really quickly? >> Oh, I mean all of those fundamentals, data structures, algorithms and the like. I mean that's what computer science and really what engineering is is an understanding of those fundamentals, those building blocks that you can assemble into projects of your own. Uh it's the code and the syntax that was never all that intellectually interesting. And while it's valuable to understand what it is you are typing initially or what the AI is doing ultimately, I think it just lends itself ultimately to a better understanding of the solutions you are creating with AI's help to problems. So those fundamentals I don't think are going away at all. I think maybe memorization of or practice with very specific APIs or function signatures and the like that's always been and can remain less interesting. And I think that's what AI is really helping with right now is enabling us to do the fun part of solving problems and assembling those building blocks without having to worry about exactly what the glue looks like. >> Do you worry that developers in the future are going to be doing more code review than actual coding? And do you think that that is going to be a net negative for the field? >> That's a really interesting question. Um, and it's timely in so far as I've indeed been playing with Copilot and Claude Code and the like. And I find that for instance, most recently in some bug fix or feature that I was trying to have Copilot implement, it was sort of akin to the quality of like a junior developer where I as the human did indeed need to review code and I sort of overrode some of its design decisions and redid some things myself. It was still helpful because it got me started. I didn't have to slog through the documentation to figure out this or that. So, still was a productivity boost, but it did cost me some time. Um, I think my own interest in software engineering would grind to a halt if all I were doing was reviewing someone else's or something else's code. But I think this is probably a short-term period in which we are spending maybe a disproportionate amount of time reviewing code that the AI has generated. And I think we'll have to spend less and less of that in time as the AI gets better and better and we can rather trust like a good PM that the engineer even if it's an AI is solving the problem as we would intend. >> Yeah. And can you talk about what you think that period is? So we had the before uh LLM tab completion like really before LLMs were used outside like visibly by developers. They've been used for years uh essentially for like Google Translate and and tools like that kind of behind the scenes but before they became like a part of many developers day-to-day workflow. What do you think the next few periods look like? So right now we're in the okay we do have to do a lot of code review period. Do you have any predictions for when and how that's going to change to where we can hand over more of the decision- making or or the actual I I guess kind of like lower level decision-m to LM without having to babysit them as much. >> Yeah, I mean no predictions that you should trust because I think all of us are just sort of guessing and going with gut here. I will say I have been consistently surprised over the past few years how quickly things are in advancing and how much better each uh iteration of GPT for instance is from OpenAI uh and their counterparts. Um so I think you can probably take with a grain of salt what I am inclined to say but I would say just after a couple more years I think we'll be worrying a little bit less certainly within well- definfined domains like software engineering about hallucinations so to speak about uh improper coding decisions and I say this because all of these tools increasingly let you tell them a little something about your coding preferences your imple your preferred implementation details whether it's in the form of a big markdown file or something else that they're getting better and better at adhering to. So much like you would be training a human uh junior developer to get better and better. I dare say that's what's happening already with some of these AIs. And so I think within a small number of years will we worry a little bit less about having the same conversation about the the fine print the disclaimers that we have to put when it comes to the these AIs writing code. I think more likely certainly for non-m missionritical type applications self-driving cars and the like where there's actually safety components involved. I think it will probably suffice to just look at the behavior of the final product. Look at the web pages for instance that are being dynamically generated or QA it from a user perspective at a higher level as opposed to getting into the weeds of the actual code. So I do think within a you know medium number of years interpret as you will uh we might have to worry less about what the code looks like too. Um but even then for the most important of applications I think some smart humans should be really getting their hands dirty still. >> Okay. Agree or disagree with this statement and you can elaborate. >> Okay. Within the next 10 or 15 years, developers jobs are going to come become more resembling technical product managers. We think of product managers today running like a specific product and just technical versions of those who can if necessary dig under the hood. >> Yes, I do suspect that's likely because I think that's happening already. Uh I for one who am not a full-time software developer sort of welcome that future because the part of software engineering that I've always enjoyed is yeah sort of the problem solving process but not so much the code. I mean the fact is that at least for me who enjoys coding as a sort of hobby but also it's sort of intertwined with my my vocation. It's [snorts] just so much more work to get all of the code out of my brain through my fingertips and into the computer than it is to sort of envision how would I solve this problem. The fun part for me is chatting with a buddy at a whiteboard and sort of deciding like what are our database tables going to look like and what are the API calls going to be look like and what's the URL format or what's the UI going to look like. all of the sort of highlevel decisions that ultimately a PM would be involved with and it starts for me personally to get a lot less interesting that I now need to sort of start churning through that to-do list and implement all of those wise decisions I think I just made. >> Yeah. And with that in mind, uh, talking a little bit about the way you teach CS50 students, and for most of the people in CS50, this is just their first exposure to programming and computer science, but you do start with C. Uh, and you do teach the C programming language >> and that is very low level. you're worrying about uh you know memory management and uh you're basically accomplishing extremely simple things when you program with that versus using JavaScript or Python or uh even you know building like an app where your LLM is helping you with a lot of the boilerplate or you're pulling in like powerful libraries and things like that. What is the value [clears throat] in 2025 2026 of still learning C? It is the same as it was in October of 2022 before Chachi PT came out. I think it's to understand from the ground floor up how computers work, how systems are designed, what it is these programming languages are doing for us. And I think perhaps most importantly to really help students appreciate that it's not magic. Like at the end of the day, the only building blocks we have at our disposal are these zeros and ones. And inside of a computer, there's hardware, but that stores these zeros and ones and performs operations on them like math. And when you think about your computer's memory as really just being this canvas of like a whole bunch of zeros and ones that you can lay out however you want, I think there's something sort of liberating about that in so far as surely nothing we're doing in CS50 can be all that complicated if at the end of the day it reduces to just zeros and ones. And so when we talk be by way of C about pointers and memory management and how you can actually build an array or a linked list or a hash table or anything more at the end of the day you truly have this bottomup understanding of how it all works. And I think the dispelling of the other what you might otherwise perceive as magic is just incredibly empowering because I think for many students a takeaway from that learning process is like okay I got this and that's indeed what we mean by building a foundation. And I think it's less interesting, less important to go and build your own link list or hashts or the like down the line. But understanding what they do, I think is compelling. And I also think in CS50 where we transition in the class from C to Python, it is wonderfully eyeopening when we transition over the course of one week from our so-called week five to week six that we can take the dozens of lines of code we wrote the previous week to implement a large hash table into a single line instantiating a Python dictionary object. And that is exactly what we've meant all prior weeks when we talk about encapsulation and abstraction and and the like. >> Yeah. I mean that's got to be a remarkable moment like kind of an inflection point in the learning where uh I often liken like learning programming to the movie the karate kid where he's learning the basic movements. So wax on wax off put his jacket on the hook take it off. he doesn't necessarily understand why he's doing all this, but once he goes to the karate tournament, he understands that all the key muscle memory and all the key movements uh necessary to they all kind of come together and he's able to do it. And part of that is just the the movie being dramatic and uh not actually telling him why he's doing these, you know, very boring chores and these very boring drills. But I do think that to an extent uh what you're teaching in CS50 is those skills coming together and suddenly boom a single line you accomplish a week's worth of work two weeks worth of work in terms of uh writing a whole lot of different um >> you know very micromanagy code. So that's that's really cool. >> Agreed. >> Yeah. And I want to talk to you about theatrics. One of the things that you do as a teacher is you try to illustrate, show, not just tell, how a lot of these concepts work. For example, binary search where you're ripping the phone book apart. Uh showing how that's like an efficient uh algorithm for finding somebody's name in a phone book, for example. Uh, and I want to talk about the role of those kinds of showing versus telling and how you come up with some of those ideas for demonstrating on the stage in front of all your students. >> Yeah. Well, the latter question is easy. I keep an eye on what other teachers are doing and undoubtedly many most of those demonstrations that I've adopted for CS50 came in some form from some prior teacher. In fact, the earliest one I can remember is that on some years in CS50, we talk about algorithms and the importance of precision, step-by-step instructions to give some other human or computer or robot. And we do a peanut butter and jelly making example. And there was a dad and his child that went viral a few years back for doing exactly this on the internet. Like, hey, wait a minute. My home room teacher in fifth grade did exactly that for us. And her goal at the time was to demonstrate just how important it was for fifth graders to follow directions carefully. And she did this by way of having us give her instructions for how to make a peanut butter and jelly sandwich. And the mere fact that I'm talking about this um what 30, 40 years later, I think is testament to just the power of those memorable moments where the overarching goal is to instill in someone an understanding of what could be an a a difficult a challenging topic. But the goal is to give them some visual or at least some mental model that they can cling to when trying to master that same topic. And so with the phone book in CS50, the overarching goal is to make a fairly arcane topic like binary search and logarithmic uh running time a lot more real because students have that real world example that even as the jargon gets complicated and the ideas might start to evolve to be all the more sophisticated. They have that very simple thing that even they could do at home. And so much of what we do on the stage in CS50 is really meant to give students by way of theatricality memorable moments that stay with them far longer than some of the jargon or the acronyms or the technical terms that we might use verbally alone. >> So you spent a great deal of time thinking about these different ways of conveying and making these concepts memorable. uh and it sounds like that's paying off a lot if you're able to uh draw from your own personal experience things that you have not forgotten over the years and uh for people who are getting into teaching uh and I I want to address the teachers in the audience uh of course I'm a former ESL teacher and we have like pretty much everybody on the free co camp team and a lot of people in the free co contributor community have a teaching background as well uh what are some ways that you go about thinking and discovering these kinds of analogies that you can teach. Of course, you said you uh heavily draw from what you see other people doing that works. But uh what is one that you've come up with by yourself that you're really proud of? >> So during COVID times we had the opportunity to collaborate with another local theater, the American Reparatory Theater in Cambridge, whose team was otherwise furoughed because of everyone was at the time working from home. So there were no shows to put on. But we had this wonderfully unique opportunity in the fall of 2020 to collaborate with this team of artisans, electricians and carpenters and prop masters to bring to life a lot of the physicality that you enjoy in a physical space like a theater. And so they helped us bring to life any number of computer science topics that way uh that year in ways that I couldn't have done on my own. And one of those was to help students visualize a linked list, which for those less familiar is this data structure in memory where you have so-called nodes, chunks of memories that are somehow linked together underneath the hood by way of pointers or in higher level languages via something like called references. And I had always fancied the idea of trying to bring this to life by using actual wooden blocks and somehow interconnect them. And so with some input to the prop masters there and their their wood shop, they kindly built these wooden nodes in the form of wooden boxes. They had uh disconnected electrical outlets in one in uh a orange extension cord coming out of the other. And the whole point was that on stage while filming that year with no students present, just a stage and some cameras, we could literally interconnect these nodes by plugging the orange extension cable from one box into the other into the other into the other. And I loved that they were able to bring this to life. My regret with that particular visualization was that when they asked me how big do I want these blocks to be, I just kind of spitled like, oh, I don't know, 3 feet, 4 feet, something like that. Not at all appreciating how damn heavy a 3 or 4 foot cylindrical block of wood would actually be. And you can see me even the video like struggling to move these things around stage to sort of maloclock them one out after another. So, it was a nice idea, but poor execution on my part. Um the the carpenters did it exactly to spec. So we have since disassembled those blocks of wood, but uh the irony is that's perhaps now more of a memorable moment for me than for anyone else. But that was one that I don't think I borrowed from someone else because no one was crazy enough to build such a thing. >> Yeah. Yeah. I mean that reminds me of the documentary mockumentary uh Spinal Tap. This is Spinal Tap. If anybody has seen that, you'll know exactly what I'm talking about here. Uh great great uh documentary if you're just late at night you want to watch but yeah the carpenter followed your instructions exactly but that uh ended up being uh a lot of backachche for you it sounds like so >> yeah in the CS education community there's this recurring theme of things that seemed like a good idea at the time which was that uh but not in the end. >> Yeah. Well I want to talk a little bit about CS50 itself. So the course precedes you and and that's so cool that you ultimately took over the course and have you know elevated it to incredible levels of prominence like literally millions of students uh on edex YouTube like all these different platforms are going through the course and uh learning a tremendous amount of computer science of course free code camp we've published a lot of your videos over the years including a security video which I love and I'm just going to I don't want to spoil the course for anybody, but I did go through the the course. And I love how you teach social engineering [laughter] uh in in a very illustrative way. You're like, "All right, everybody, write down a password or write down one of your passwords on this piece of paper. Uh we're going to examine this." And and you basically trick people into writing down a password and you're like, "All right, I just social engineered you." And people are going to carry that humiliation of like, "Oh man, I got duped that." And they're never going to forget what social engineering was, especially the ones that actually wrote the the name down. So in a very low stakes way, you taught them a very important lesson about security, right? Um let's talk a little bit about your background with security and that is what you study and I I believe your your dissertation was security focused uh when you did your PhD, right? >> It was. Yeah. On the rapid detection of zero day outbreaks of internet worms, which are like viruses that can spread on their own. >> Yeah. And a lot of people think, oh, he's the more general CS guy, like he teaches computer science in general. But you do have that area of specialization. And um what inspired you to create like a CS50 course that's focused on security? >> Oh, so with CS50 cyber security, that's just long been an interest of mine from graduate school and even prior. So that certainly informed my graduate studies. Um, it's just so practical though nowadays and it's just so important for technical and non people non-technical people alike to have a foundation of understanding in what the threats are out there, what the possible defenses are to give them a healthy fear of things they could do wrong when it comes to something as mundane as like opening their email and what can go wrong. Um, I think it's very much consistent with CS50's overarching vision of providing students with the fundamentals of understanding in any number of topics. So cyber security felt to me a very natural extension of that and it just so happens to be ever more practical for a broader audience even than the lower level courses like CS50X itself. >> Yeah. And let's talk more generally about CS50. What is the history? And I want to point out that I did interview you maybe like five or six years ago and that recording is on the free code camp RSS feed. Uh you might be able to find it in like Spotify or Apple podcast if you search for it. But uh what how did you end up running CS50? Like the just like the the very quick explanation of how you ended up here and uh what you've done since you got it uh to expand. >> Yeah. Right time, right place. So I took the course myself in the fall of 1996 when I was a sophomore in university. Uh fast forward to graduate school where I returned to Harvard for 5 years to work on my PhD. And the year I was graduating, spring of 2007, the previous instructor for the course was moving on to become dean of the school. And there was an opportunity, a need for someone to fill in. Uh the school was uh intending to have a senior faculty member teach this introductory gateway course for the undergraduates, but somehow or other, I talked my way into the job for what was supposed to be one year, after which they would find someone more senior to teach it. Somehow or other, that was 19 years ago. um and I'm still at the helm and it certainly evolved as you note over time to have not only this on-ampus component but a very large um outward-f facing online component not only through Harvard's extension school and summer school but through platforms like edex and YouTube that have really challenged us to think all the more about how you teach effectively at scale how to provide resources for students not only on campus but off and generally treat the entire community of CS50 as as one class taking it as the shared experience Yeah. And CS50 has expanded and there are like different variants of CS50. For example, I believe you have like a CS50 for MBA students like Master of Business Administration. You have like a CS50 for lawyers. Can you talk about these variants and how they differ and how you kind of tailor computer science as a subject for people in other fields that are unlikely to I guess go directly into software engineering but will most likely be working alongside software engineers? Of course, uh there's a popular book called uh physics for presidents which the the essence of which is all about well what are the minimal what is the minimal physics knowledge that a president should have to make good decisions and I think that's true of every field not just about physics but about computer science as well and so in crafting this course at Harvard Business School about 10 years ago now aka CS50 for MBAs the goal was still to empower students with a bottomup understanding of how computers and technology work but the overarching goal was not necessarily to transform them into engineers, but rather better decision makers. And those aren't certainly mutually exclusive. But the reality is at the MBA level for graduate students probably don't need to spend as much time on how to stitch together a linked list or data structure like a hash table in memory using C and pointers. Rather, let's teach them what those data structures are. Let's use abstractions. Let's use pictures so that they have a general understanding of what those building blocks are. even if they're never going to build them themselves, give them a taste of programming, whether it's with Python, we use Scratch in the class for a day as well, so that they have a general sense of what it is software engineers in the real world actually do, but accepting that this is not their goal professionally or personally. And their goal is to just better understand the world around them when they go off into the business world or the legal profession or beyond. And so with both of those courses, we take at the ultimately more of a top-down approach whereby we introduce students to some salient concepts, whether it's cloud computing or cyber security or the like and then give them a familiarity with the underlying building blocks that built up those higher level abstractions. And the business course is focused there for more broadly on how you can leverage technology at scale, uh, store large amounts of data, crunch lots of numbers, leverage artificial intelligence. Nowadays, the law class had a bit more of a focus on intellectual property and cyber security that might intersect with the world of law. But at the end of the day, it was all about empowering a broad citizenry to make better decisions and better understand the world of technology. >> Yeah. And on that note, I really want to kind of get any advice you have for teachers who are potentially trying to communicate the fundamentals of computer science to people, especially if they are giving just like a talk uh where they have only maybe an hour to impart some important concepts. when you're dealing with, you know, people that are focused on business or focused on law, like how you figure out the amount of detail, like the resolution if you're going to present concepts. >> I I think in my courses, we certainly don't shy away from level of detail. And I do think that most of our courses probably combine more information, more topics, more concepts than is typical in a course. Even CS50 for instance is what we have long called an amalgam of what other universities call CS1 and CS2 combined. So there's a lot we move quickly but the goal really is to enable students to understand a broad a broad spectrum of concepts within the limited amount of time that we actually have with them. As for how I decide what goes in there I mean I just don't shy away from really getting into the weeds. And for instance in CS50 for MBAs and for lawyers we talk about pointers. We don't show them the syntax of dreferencing and following pointers using say C syntax, but we talk about them pictorially in terms of like here's an arrow and here's a grid of memory and here's how you can stitch something together therein. And I think that that is valuable for the same reasons it's valuable for the undergraduates taking the the flagship core CS50X in that it just helps them realize that there's really no magic here. At the end of the day, it's just zeros and ones, and I can use it however I see fit. And this is how people who have come before me have decided to use it to build up these building blocks in computing. And so, I think the challenge though is getting students to that that level of understanding when you're starting with literally just the zeros and ones. And so, I generally approach it as telling a story. And where does that story begin? In CS50, for instance, and in all of those variants, we start with zeros and ones. And if that's all you had, all right, well, how do you count to the number two, let alone three and four and five? Well, we talk about how you can permute the bits and actually represent numbers in binary notation. All right. Well, once we can represent our decimal world of zeros through nines using just binary, well, what other problem should we solve? Well, how about being able to send a text message using English or some other human language? So, we talk about ASI and Unicode. From there, it's a natural progression to like colors of the rainbow and therefore images and thereafter video, if not music. And so by the end of just like an hour like we've already talked about this massive stack of abstractions that has been developed over the years and I hope in retrospect at student to students it doesn't feel like it was actually that scary a ride because we started from zero we got up to 60 but there was an acceleration that was continuous and not just discreet jumps week to week. Yeah, that's really uh beautiful the way that you've figured out a way to so quickly kind of remove a lot of people's apprehension from learning more about math and programming and computer science through your course because I I think people are daunted and they do think it's magic and and it sounds like you figured out a way to demystify it very quickly in in a very pain like low pain low uh apprehension way. So, I want to talk a little bit about questions and answers and the way that you've traditionally dealt with people wanting to get their questions answered, uh, for example, through office hours, uh, through having TAs that can check in with learners. You have something called the CS50 duck. And for those who are listening who are unfamiliar with duck debugging, maybe you can explain that and then explain what the CS50 duck does and just your more general thoughts on AI assistance. >> Sure. So for those unfamiliar, it's a thing in programming to you leverage what's called rubber duck debugging whereby you literally keep an inanimate object like a yellow rubber duck on your desk and the in the absence of having a colleague or a mentor or a family member or friend who knows more about you than uh about code than you do, you should literally just talk to the rubber duck and if you're having some confusion or some bug or mistake in your code, just tell the rubber duck about it. And invariably that process of communicating your confusion step by step often yields to like that proverbial light bulb going off over your head because you realize I've been an idiot because you hear in your own words the mistake or the confusion that you've had. Doesn't always work but it can certainly work. And so the exercise of talking to a rubber duck is just to help you express what your confusion is in hopes that you better understand the problem and therefore can find your own way to a solution. Now, in the absence of being able to find your own way, nowadays we have um co-opted that idea of a physical rubber duck and virtualized it thanks to some of our my colleagues and some of our former students back home in the form of a virtual rubber duck. It's c50.ai that anyone on the internet with a free GitHub account can access. And it is attenuated specifically to CS50, but it answers your questions in English or really in any human language with which you engage. And it is designed to have some pedagogical guardrails in place so to speak whereby we essentially have aspired through code to make the default version of chat GPT less helpful because chat GPT as many have discovered is all too willing to just solve all of your problems for you if it can. Um not to mention and do all of your homework for you which is not I think most people would agree educationally valuable. And so we've tried to make it less helpful our duck in the sense that it tries to respond akin to a good teacher or tutor whereby we try to have the duck lead students to solutions as opposed to just handing it to them outright by giving them the code in question. So more often than not, the duct is designed to respond in English, not an actual C code or Python code or anything else, and to really respond with some frequency with questions of the students so as to help them even more so than the physical rubber duck can get to the uh point of clarity. >> Okay. So it's kind of more of a Socratic method type approach. I mean, you you you talk about uh pedagogical guardrails, essentially nerfing the LLM, and this is a recurring theme, nerfing the code editor, nerfing the LLM, making the tools less powerful so you can kind of force people to uh kind of learn how to fly for themselves. They're you're pushing them out of the nest. And uh maybe that's not the the like an exact analogy for what you're doing here, but um can you talk about the evolution of the duck and kind of there's also a system I believe called ED that you've used with uh CS50 and how that has impacted learners and office hours. >> Yeah, for our onampus students and also our on uh our on-ampus students and also our online students. We've long used a cool tool called ED or edsteem.org or which is a wonderful tool for Q&A, sort of a modern version of the Q&A form that you can use for a specific class. And early on, it was used only for human interactions among students and teachers and including myself. Uh but very quickly did we insert some of our own AI features into ED by having a virtual rubber duck respond to students questions back in 2023 uh when OpenAI and others first started releasing APIs or application programming interfaces via which you could leverage their services in your own applications. Since then we've evolved into having our own CS50.AI tool which is essentially our own version of chatbt. We've integrated this virtual duck into cs50.dev dev which is our own adaptation of GitHub code spaces which is essentially VS code in the cloud so that students do have at their fingertips um these various AI based tools that they can interact with either by clicking buttons or interrogating them with uh written questions. >> Yeah. And what are some questions that uh a learner might traditionally ask a TA or ask you uh that they're now asking uh either Ed or uh Duck AI uh >> almost Oh, sorry. Go ahead. >> Yeah. Yeah. Yeah. Yeah. Go ahead. >> Almost everything. Um, we have seen a marketked decrease in the number of questions that our human students are asking our human TAs. It is a good thing in the sense that there was not that many years ago that we couldn't keep up with the volume of questions. And in my early days of teaching CS50, I still remember to this day the night or two before a homework assignment was due, sending out an email to the teaching staff pleading pleading with people to uh pluck off as many questions as they could because it was simply overwhelming for any one of us to answer all of those questions. Now it's it's dare say crickets uh on the discussion fora because students are turning much more frequently synchronously to a tool like the duck uh which is a good thing I do think in terms of immediia for help. It is there is it is a two-edged sword and that some students we know statistically are probably asking too many questions. I I don't claim to know exactly what the right number of questions is to ask on a particular problem or homework assignment, but it's probably not 200 is one of the outliers we've seen statistically in the data. uh feels to me like the right model is thinking back to you know maybe high school class where you'd raise your hand or go to the front of the room and ask the teacher a question and at some point even the best of teachers is going to say David you need to go back to your seat and start to wrestle with this on your own think about the problem for a bit more and that's what we've tried to do with the duck so indeed it actually has a built-in throttling mechanism implemented in the form of uh hearts or HP kind of like a Zelda game uh whereby those hearts uh diminish uh decrease over the course of your usage and they automatically do regenerate, but at a rate that we think is is reasonable. Um, we've also seen a market downturn in attendance at physical office hours, which are in-person opportunities for one-on-one or small group help on campus. That too has been a good thing in the sense that those two we could not keep up with during the heights of popularity of office hours, even though our ratio of students to TAS has really been invariant over the years. Um nowadays we have sometimes more TAs than students at office hours because they too were turning to the duck. Um that said I'm not sure it was ever the best use of students time to have to for instance slog across campus get in a seat raise their hand wait for help. So I do think we've provided better pedagogical support. That said a side effect that is I think we do a worse job at training our own TAs or our own future teachers because there's just fewer students for them to interact with. fewer students with whom to practice the art of teaching, fewer questions that they get asked to think through and then answer in an effective way. So that's little different from what I think is happening even in the software industry where junior developers are uh potentially being replaced by AI and so it calls into question well how do you train the senior developers if you have this gap? So that too is a very valid concern but one that I think can be addressed by just all the more education, all the more training so that the humans are progressing from u 0 to 60 um and perhaps even passing that that junior step in some sense. Well, uh, it's cool that you bring up this new problem, I guess, that nobody necess not very many people would have anticipated that, oh, like people are these these LMS are too helpful, uh, to the extent that, uh, we're just not getting the teacher training that we used to get. Uh, being a a teaching assistant, you just don't have the same volume of interactions with learners because they can go and, uh, ask NLM and don't have to be embarrassed by their their own ignorance. And that affects me even at my level. I sometimes I'm like, is this a dumb question? You know, [laughter] and they don't have to worry about that because the LLM won't judge them. Uh, and the LLM, they can just delete the conversation and never pretend that they didn't know how to properly pronounce pedagogy, for example. That was one of the question before. I read this word all the time. Is it pedagogy or pedigogy? You say it one more time. >> I say pedagogy. >> Okay, cool. Yeah. And uh I don't have a background in Latin, I guess, or Greek. Maybe Greek is probably where that word comes from. Uh but uh I guess public service announcement, don't be afraid to ask dumb questions. Uh but you know, asking an LLM a dumb question, nobody ever knows that you didn't know that, right? [laughter] >> Indeed. And hopefully it's not making up the answer. So >> yeah. Yeah. Hopefully it's not making up the answer. Uh that's that's never good. Uh since you mentioned like the the rush at the uh right before the homework assignment is due, the rush right before the exam where people are asking questions, I wanted to talk about academic honesty because you did publish a paper about this uh and uh even at a you know high level like Harvard we think basically one of the best universities in the world. It's very hard to get into Harvard. There is academic dishonesty even at that level. There there are people that will given like a whole lot of pressure uh they're like oh man if I go just turn in what I have I'm going to bomb and I'm potentially not going to pass CS50 and that's going to be reflected forever on my university transcript. So people do have an incentive to essentially commit acts of academic dishonesty and I wanted to talk about how uh you all have approached this over the years. you've experimented quite a bit and uh what some of the problems are and what are some of the solutions you've tried. >> Yeah. And it's a good question and something that we and so many other teachers wrestle with. Um and it was painful in my early years of teaching to see students crossing these lines by submitting work that wasn't their own either by having someone else do it for them or submitting work that had already been submitted or nowadays of course leveraging AI uh in ways they that they should not according to the course's own policy. Um I do think it happens for the very reasons you allude to. Students uh at all levels are under pressure whether it's academic or personal or professional. They meet uh they have some deadline to meet. They might be taking a course that they don't necessarily want to take or don't particularly care about. And so in that sense it doesn't feel all that important to worry about those lines. Um, at the end of the day though, one of the things we've tried to do in CS50 as best we can though is to instill in students an appreciation of just how important it is for your work to reflect yourself. At the end of the day, you're putting your name on this and it should reflect your work, ideally good quality work, but if not, uh, better that it be poor work as opposed to that of another. And so we do, like many peer institutions and introductory courses, cross-co compare all submissions that students submit, looking for statistically unlikely similarities across submissions. And if we humans then upon seeing the results of that analysis by a computer decide, yeah, this just didn't seem to happen by chance, we then refer the matter in Harvard's case to the administrative board or the honor council where a different group of people um then adjudicate the matter formally. Um typically we administratively discipline therefore 5 to 10% of CS50 student body on campus. This is unfortunately in line with peer institutions. Um, a silver lining I can say in all of this is that even in the advent of AI, we CS50 on campus at least have not seen an uptick in misbehavior. Um, I don't doubt that there are some instances of academic dishonesty that we're not catching and that we've never caught, although 5 to 10% is already a big chunk. Um, and I'd like to think that the fact that we haven't seen a market uptick in inappropriate behavior in CS50 is because we've spent the past 20 plus years messaging to students just how important it is to be mindful of these issues and also providing them to the extent we can with mechanisms for dealing with that stress and that sense of being overwhelmed or up against a deadline. So some years back for instance in CS50 we introduced what we call regret clause which is some language in the syllabus that essentially prescribes what students should do if they realize that they've made a poor decision. Because I think all too often in CS50 and in academia more generally you might very well make a poor decision under stress under no sleep at like 2 a.m. in the morning and it's only after you finally get some sleep and give it some thought that you actually regret what you did and in hindsight perhaps would not have done it. But typically in a class there's just no mechanism to then solve that problem after the fact. And so what we proposed in CS50 syllabus a few years ago that if students come forward to us uh within 72 hours of having made a poor decision, the 72 is somewha