What we're learning from online education - Daphne Koller

[Music] [Music] [Applause] like many of you I'm one of the lucky people I was born to a family where education was pervasive I'm a third generation PhD a daughter of two academics in my childhood I played around in my father's University lab so it was taken for granted that I attend some of the best universities which in turn open the door to a world of opportunity unfortunately most of the people in the world are not so lucky in some parts of the world for example South Africa education is just not readily accessible in South Africa the educational system was constructed in the days of apartheid for the white minority and as a consequence today there's just not enough spots for the many more people who want and deserve a high quality education that scarcity led to a crisis in January of this year at the University of Johannesburg there were a handful of positions left open from the standard admissions process and the night before they were supposed to open that for registration thousands of people lined up outside the gate in a line a mile long hoping to be first in line to get one of those positions when the gates opened there was a stampede and 20 people were injured and one woman died she was a mother who gave her life life trying to get her son a chance at a better life but even in parts of the world like the United States where education is available it might not be within reach there's been much discussed in the last few years about the rising cost of healthcare what might not be quite as obvious to people is that during that same period the costs of higher education tuition have been increasing at almost twice the rate for a total of 559 since 1985 this makes education unaffordable for many people finally even for those who do manage to get a higher education the doors of opportunity might not open only a little over half of recent college graduates in the United States who get a higher education actually are working in jobs that require that education this of course is not true for the students who graduate from the top institutions but for many others they do not get the value for their time and their effort Tom fredman in his recent New York Times article captured in a way that no one else could the spirit behind our effort he said the big breakthroughs are what happens when what is suddenly possible meets what is desperately necessary I've talked about what's desperately necessary let's talk about what's suddenly possible what's suddenly possible was demonstrated by three big Stanford classes Each of which had an enrollment of 100,000 people or more so to understand this let's look look at one of those classes the machine learning class offered by my colleague and co-founder Andrew in Andrew teaches one of the bigger Stanford classes it's a machine learning class and it has 400 people enrolled every time it's offered when Andrew taught the machine learning class to the general public it had 100,000 people registered so to put that number in perspective for Andrew to reach that same siiz Audience by teaching a Stanford class he would have to do that for 250 years of course he' get really bored so having seen the impact of this Andrew and I decided that we needed to really try and scale this up to bring the best quality education to as many people as we could so we formed corsera whose goal is to take the best courses from the best instructors at the best universities and provide it to everyone around the world for free we currently have 43 courses on the platform from four universities across a range of disciplines and let me show you a little bit of an overview of what that looks like welcome to calculus 50 million people are uninsured models help us design more effective institutions and policies we get unbelievable segregation so Bush imagined that in the future you'd wear a camera right in the center of your head Mills wants the student of Sociology to develop the quality of mind hanging cable takes on the form of a hyperbolic cosine for each pixel in the image set the red to zero vaccine allowed us to to eliminate polio virus does LOF tanza serve breakfast and San Jose well that sounds funny so this is which coin you pick and this is the twoes so in large scale machine learning we'd like to come up with computational it turns out maybe not surprisingly that students like getting the best content from the best universities for free since we opened the website in February we now have 640,000 students from 190 countries we have 1.5 million enrollments 6 million quizzes in the 15 classes that have launched so far have been submitted and 14 million videos have been viewed but it's not just about the numbers it's also about the people whether it's a Kos who comes from a small town in India and would never have access in this case to a Stanford quality course and would never be able to afford it or Jenny who is a single mother of two and wants to hone her skills so that she can go back and complete her master's degree or Ryan who can't go to school because his immune deficient daughter can't be risked to have germs come into the house so we couldn't leave the house I'm really glad to say recently we've been in correspondance with Ryan that this story had a happy ending baby Shannon you can see her on the left is doing much better now and Ryan got a job by taking some of our courses so what made these courses so different after all online course content has been available for a while what made it different is that this was a real course experience it started on a given day and then the students would watch videos on a weekly basis and do homework assignments and these would be real homework assignments for real grade with a real deadline you can see the deadlines in the usage graph these are the spikes showing that procrastination is a global phenomenon at the end of the course the students got a certificate they could present that certificate to a prospective employer and get a better job and we know many students who did some students took their certificate and presented this to an educational institution at which they were enrolled for actual college credit so these students were really getting something meaningful for their investment of time and effort let's talk a little bit about some of the components that go into these courses the first component is that when you move away from the constraints of a physical classroom and design content explicitly for an online format you can break away from for example the monolithic 1hour lecture you can break up the material for example into these short modular units of 8 to 12 minutes Each of which represents a coherent concept students can Traverse this material in different ways depending on their background their skills or their interests so for example some students might benefit from a little bit of Preparatory material that other students might already have other students might be interested in a particular enrichment topic that they want to pursue individually so this format allows us to break away from the one-sized fits-all model of education and allows students to follow a much more personalized curriculum of course we all know as Educators that students don't learn by sitting and passively watching videos perhaps one of the biggest components of this effort is that we need to have students practice with the material and really in order to really understand it there's been a range of studies that demonstrate the importance of this this one that appeared in science last year for example demonstrates that even simple retrieval practice where students are just supposed to repeat what they already learned gives considerably improved results on various achievement tests down the line than many other educational interventions we've tried to build in retrieval practice into the platform as well as other forms of practice in many ways for example even our videos are not just videos every few minutes the video pauses and the students get asked a question prospect theory hyperbolic discounting status quo bias base rate bias they're all well documented so they're all well documented deviations from rational Behavior so here are the video pauses and the student types in the answer into the box and submits obviously they weren't paying attention so they get to try again and this time they got it right there's an optional explanation if they want and now the video moves on to the next part of the lecture this is a kind of simple question that I as an instructor might ask in class but when I asked that kind of a question in class 80% of the students are still scribbling the last thing I said 15% are zoned out on Facebook and then there's a smarty pants in the front row who blurts out the answer before anyone else has had a chance to think about it and I is the instructor I'm terribly gratified that somebody actually knew the answer and so the lecture moves on before really most of the students have even noticed that a question had been asked here every single student has to engage with the material now of course these simple retrieval questions are not the end of the story one needs to build in much more meaningful practice questions and one also needs to provide the students with feedback on those questions now how do you grade the work of 100,000 students if you do not have 10,000 Tas the answer is you need to use technology to do it for you now for fortunately technology has come a long way and we can now grade a range of interesting types of homework in addition to multiple choice and the kinds of short answer questions that you saw in the video we can also grade math mathematical Expressions as well as mathematical derivations we can grade models whether it's um Financial models in a business class or physical models in a science or engineering class and we can grade some pretty sophisticated programming assignments so let me show you one that's actually pretty simple but fairly visual this is from Stanford's computer science 101 class and the students is supposed to color correct that blurry red image they're typing their program into the browser and you can see they didn't get it quite right Lady Liberty is still seasick and so the student tries again and now they got it right and they're told that and they can move on to the next assignment this ability to interact actively with the material and be told when you're right or wrong is really essential to Student Learning now of course we cannot yet grade the range of work that one needs for all courses specifically what's lacking is the kind of critical thinking work that is so essential in such disciplines as the humanities the social sciences business and others so we tried to convince for example some of our humanities faculty that multiple choice was not such a bad strategy that didn't go over really well so we had to come up with a different solution and the solution we ended up using is peer grading it turns out that previous studies show like this one by Sadler and good that peer grading is a surprising ly effective strategy for providing reproducible grades it was tried only in small classes but there it showed for example that the students assign grade on the y-axis are actually very well correlated with the teacher assign grade on the x-axis what's even more surprising is that self- grades where the students grade their own work critically so long as you incentivize them properly so they can't give themselves a perfect score um are actually even better correlated with the teacher grades and so this is an effective strategy that can be used for grading its scale and is also a useful learning strategy for the students because they actually learn from the experience so we now have the largest peer grading pipeline ever devised where tens of thousands of students are grading each other's work and quite successfully I have to say but this is not just about students sitting alone in their living room working through problems around each one of our courses a community of students had formed a global community of people around a shared intellectual endeavor what you see here is a self-generated map from students in our Princeton sociology 101 course where um they have put themselves in a world map and you can really see the global reach of this kind of effort students collaborated in these courses in a variety of different ways first of all there was a question and answer form where students would pose questions and other students would answer those questions and the really amazing thing is because there were so many students it means that even if a student posed the question at 3:00 in in the morning somewhere around the world there would be somebody who was awake and working on the same problem and so in many of our courses the median response time for a question on the question and answer form was 22 minutes which is not a level of service that I have ever offered to my Stanford students and you can see from the student testimonials that students actually find that because of this large online community they got to interact with each other in many ways that were deeper than they did in the context of a physical classroom students also self-assembled that any kind of intervention from us into small study groups some of these were physical study groups along geographical constraints and met on a weekly basis to work through problem sets this is the San Francisco study group but there were ones all over the world others were virtual study groups sometimes along language lines or along cultural lines and at the bottom left there you see our multi- cultural Universal study group where people explicitly wanted to connect with people from other cultures there's some tremendous opportunities um to be had from this kind of framework the first is that it it has the potential of giving us a completely unprecedented look into understanding human learning because the data that we can collect here is unique you can collect every click every homework submission every Forum post from tens of thousands of students so you can turn the study of human learning from the hypothesis driven mode to the datadriven mode a transformation that for example has revolutionized biology you can use these data to understand fundamental questions like what are good learning strategies that are effective versus ones that are not and in the context of particular courses you can ask questions like what are some of the misconceptions that are more common and how do we help students fix them so here is an example of that also o from Andrew's machine learning class this is a distribution of wrong answers to one of Andrew's assignments the answers happen to be pairs of numbers so you can draw them on this two-dimensional plot each of the little crosses that you see is a different wrong answer the big cross at the top left is where 2,000 students gave the exact same wrong answer now if two students in a class of a hundred give the same wrong answer you would never notice but when 2,000 students give the same wrong answer it's kind of hard to miss so Andrew and his students went in looked at some of those assignments understood the uh the root cause of the misconception and then they produced a targeted error message that would be provided to every student whose answer fell into that bucket which means that students who made that same mistake would now get personalized feedback telling them how to fix their um misconception much more effectively so there's so this personalization is something that one can then build by having the virtue of large numbers personalization is perhaps one of the biggest opportunities here as well because it provides us with the potential of solving a 30-year-old problem educational researcher Benjamin bloom in 1984 posed What's called the two Sigma problem which you observed by studying three populations the first is the population that's studied in a lecture-based classroom the second is a population of students that studied using a standard lecture-based classroom but with a Mastery based approach so that students couldn't move on to the next topic before uh demonstrating Mastery of the previous one and finally there was a population of students that were taught in a one-on-one instruction using a tutor the Mastery based population is a full standard deviation or Sigma in achievement scores better than the standard lecture-based class and the individual tutoring gives you two Sigma Improvement in performance to understand what that means let's look at the lecture-based classroom let's pick the median performance as a threshold so in the lecture-based class half the students are above that level and half are below in the individual tutoring instruction 98% of the students are going to be above that threshold imagine if we could teach so that 98% of our students would be above average hence the two Sigma problem because we cannot afford as a society to provide every student with an individual human tutor but maybe we can afford to provide each student with a computer smartphone so the question is how can we use technology to push from the left side of the graph from the blue curve to the right side with the green curve Mastery is easy to achieve using a computer because a computer doesn't get tired of showing you the same video five times and it doesn't even get tired of grading the same work multiple times we've seen that in many of the examples that I've shown you and even personalization is something that we're starting to see the beginnings of whether it's via the personalized trajectory through the curriculum or some of the personalized feedback that we've shown you so the goal here is to try and push and see how far we can get towards the green curve so if this is um so great our universities now obsolete well Mark Twain certainly thought so he said the college is a place where a professor's lecture notes go straight to a student's lecture notes without passing through the brains of either I beg to differ with Mark Twain though I think what he was complaining about is not is not universities but rather the lecture-based format that so many universities spend so much time on so let's go from back even further to Plutarch who said that the mind is not a vessel that needs filling but wood that needs igniting and maybe we should spend less time at universities filling our students Minds with content by lecturing at them and more time igniting their creativity their imagination and their problem solving skills by actually talking with them so how do we do that we do that by doing Active Learning in the classroom so there's been many studies including this one that show that if you use Active Learning interacting with your students in the classroom performance improves on every single metric on attendance on engagement and on learning as measured by a standardized test you can see for example that the achievement score almost doubles in this particular experiment so maybe this is how we should spend our time at universities so to summarize if we could offer a top quality education to everyone around the world for free what would that do three things first it would establish education as a fundamental human right where anyone around the world with the ability and the motivation could get the skills that they need to make a better life for themselves their families and their communities second it would enable lifelong learning it's a shame that for so many people learning stops when we finish high school or when we finish college by having this amazing content be available we would be able to learn something new every time we wanted whether it's just to expand our minds or to change our lives and finally this would enable a wave of innovation because amazing talent can be found anywhere maybe the next Albert Einstein or the next Steve Jobs is living somewhere in a remote Village in Africa and if we could offer that person an education they would be able to come up with the next big idea and make the world a better place for all of us thank you very much [Applause] [Music] w