Lecture 6: Dynamics and Programming

Key Takeaways

Okay, we'll get started on time. If you look at the reading list and scroll down to where we are, which is dynamics and programming, you'll see lecture six there. There's a chapter on variant which is starred called time. That's because the application today will be featuring dynamics and what's what happens over various dates and we're we're going to do it in the context of a storage problem where people were close to starving to death as in the medieval villages. And that is chapter three of Medville. It's also starred, which uh fills out even more than I'm able to in one class the material for the lecture today. So, that's the reading list. And there's study guide. So, I went back and earmarked what we did last time, which was decision making under uncertainty linear programming. Some of these questions are worded as if you were doing it at home, but nevertheless, the question asks about important concepts. So, let me read the first one. Draw a consumption set which consists of discrete number of points. Argue that this set is not convex. And then show how lotteries, which put mass on these discrete points, transforms the consumption set into something convex. So, there's two sentences there, effectively two questions. Could I get a volunteer, please? Now, you can't actually draw it. Can you describe a consumption set? Huh. Maybe these words suffice. Try to describe in words a consumption set which contains a discrete number of points and tell me why it's not convex. Yeah, um so, um so, I guess I think this is I think we discussed this earlier, but one consumption set was consuming consuming lunch in New York in New York at noon versus consuming lunch in Washington D.C. at noon. That's true. So, that's so that would be a discrete consumption set and since you can't have a mix you can't consume lunch in a mixture of those two, so it's that then the consumption set is not convex. But, if you if you had a some probability of eating lunch in either location, um then you can have any a probability between zero and one. And what uh V How would you draw it? Um So, so for the original discrete consumption set you'd you'd you'd just be it would be two points. I think one of those points would be 1 0 uh Yeah. And one of those points would be 0 1. Good. Yeah. And imagine how I'll open a second if it's not clear. Can you imagine how you would be able to convert that set into something that is convex uh using the lotteries? Is there something you could draw to depict it? Um if you drew if you sort of drew a line between those two points like your location instead of doing how many lunches you eat at each um at D.C. and New York, you could do the probability that you eat at D.C. or New York. Exactly. So, so now we have a line running on the diagonal from zero 1 0 to 0 1. The end points are still there. Those are lotteries that are degenerate. They're putting probability one on each each of the end points. An intermediate lottery, a non-trivial lottery, say halfway between, would put probability one half on would be distance one half to 1 0 and distance one half to 0 1. And that represents an equal weighted probability and you can inch closer to one end point or the other. Those are still non-degenerate lotteries. So, so a lottery, when you put a point in there, represents the weights. But, yes, your answer was perfect. Thank you. Questions about that? Okay. What does risk aversion imply about the consumer's utility function? Again, I'll take volunteers. If you're risk averse, then you have diminishing marginal utility of income, so the utility function should be concave. Good. Strictly or weakly? Um convex. Um yeah, let me repeat. The question said it says risk What does risk aversion imply about the consumer's utility function? So, your answer concave, yes, and I'm asking back you mean strictly concave, weakly concave? I mean, I think it would be weakly concave, right? Because if you have like a straight line or actually would that still be considered risk averse if you're like utility from getting a certain value is like completely equal to the utility of like having that as your expected value. All right, that's perfect. You figured it out. So, the answer is we typically think of risk aversion to mean strictly averse to the lottery. Um and hence the uh the linear case of uh of a linear utility function or weakly concave function, but not strictly concave, we call them risk neutral and not risk averse. Uh but I never said that in class, so I'm amplifying. But, yes, risk aversion as being risk averse, strictly speaking, implies strictly concave utility. So, perfect. Thank you. Um So, one more here. Um this again is going to require you to describe something combining states and time of the world. Time and states of the world, what is the complete commodity space for a consumer? And what I'm really looking for is for you to describe to me the the tree of possibilities that we had on the slide last time. And again, I'll take volunteers. So, like this is a a tree that's like oriented upwards and so at where time increases as you go up in this tree, um as you make a specific choice in any given time, you limit your future options to some specific set and you like exclude some others that are no longer possible for you to consume. Yeah, that's right. So, the uh the tricky part of it is to remember that that the state of the world at any date is not just a set of possible realizations of the contemporary shock. It also includes the history of shocks up to and including the contemporary date. So, that's what's going on with that tree. You pick a a node in the tree is described not just by the set of all things that could have happened on the current branch, but the set of all things that could possibly have happened um up to and including that point. And actually, I made the comment in class that one has to consider whether you're already at the current date or whether you're at date zero imagining what could happen in the future. Because if you're at zero and imagining what could happen in the future, then you have to delineate all pos- possible histories and all possible states that could happen in any given date including up through and including the date in question. Great. Okay. Um So, again, I earmarked some of these. I only went I went through three or three of the four. That was very helpful. Thank you. Um okay, so the um the lecture today is So, this is on dynamic dynamics and dynamic programming. It's going to focus on this application for medieval villages on storage or call it inventory or carryover as well as the seed that they put in the ground. And I'll come back to that momentarily. And if you go back to the economic science lecture, this lecture is illustrative of something we call calibration. So, you have a model of the way the world works and you find somehow parameter values that are appropriate for the model and then you simulate the model and figure out how well it fits the data. And if it fits reasonably well, then it's a success that you have a logical explanation of in an artificial economy of how the world is actually work or in this case past tense was working. There should be another bullet point here because I like to feature and earmark that when it comes up, we go through various techniques, and the technique that's going to be come apparent is dynamic programming. So, most of the lectures up to this point, not all of them were static. We made an exception with that interest rate and income and substitution effects and a couple of the consumption set diagrams. So, time is playing a big role here, and so if we're going to solve a problem or a program, it's going to be a dynamic program. And there's some useful techniques that fall right out of the end of this lecture. Okay, so we're going to talk about actual and potential use of risk reduction arrangements, namely carrying over grain from one year to the next. This first line says alternative, begging the issue of what else have we looked at? Well, we looked at an individual putting his strips of land all over the village, and I referred to that as sort of portfolio diversification at one point. We looked at and we are about to see again the picture of the estates of the Bishop of Winchester, which were spread out over space, arguably again to diversify. And now we're going to be stuck in one village and effectively with one person in one village on his or her own looking for alternative ways of reducing exposure to risk. Now, the stylized fact, if we go back to the same Bishop of Winchester accounts, we don't see a whole lot of carryover. There are one estate at the time rare spikes where the carryover jumps up, and then the next year it's down. I should say, all this comes up later, carryover means carried over from one harvest to the next. Obviously, time passes by the day and by the month, they only produce one crop a year, they have to have something to eat during the interim. So, there is some storage going on. The issue is whether by the time they get to the next harvest, there's any storage left at all. And if there is, we call it carryover. Well, likewise, if you look at the same estate accounts at a point in time, you can see that some of them have carryover, but most of them don't. So, this model is going to be similar to a macro model. That probably doesn't mean much to you right now. A neoclassical growth stabilization model. We're going to put in two kinds of storage technologies, grain in the bin for storage, but also seed in the ground, cuz that's another way to carry over the crop by planting it. And we're going to choose the parameters to be consistent with the available data, and we're going to again look at the numerical predictions of the model and see if we can match the fact that carryover was not common. This goes more into how uncommon it was. Where if they're not carrying anything over and there's only one person in the village, like Robinson Crusoe as an abstraction, then they got they're eating all their output. Well, maybe in over a 12-month time period. So, variability in consumption is one to one with variability in output. And if output varies a lot, that's a lot of risk to bear. We have data from that same lord, treating him as the single villager, we can look at output as representative output in the population, and and see in the cross-section in 1236, only three of the 18 estates had carryover. In a good year, there was carryover. 1223 was good, for example, for one of those estates with 57% of the crop being stored till next year. And the following year, for that same estate, 1224, zero carryover. This is the review picture. These are the estates of the Bishop of Winchester, and as I said last time, it's hard to figure out what what country you're in, let alone where you are in it. But this is the English Channel down here. So, all these dots are the Bishops of Winchester estates. So, that's the context that's generating the data. Okay, two ways to store, well, actually four. We're going to adopt two of them and rule out the two others, not model the two others, I should say, in the artificial economy. First, grain can be stored after harvest, as I've been saying. Second, they could take some of that harvest and put the grain in the ground as seed planted for next year's crop. The other things we're not going to consider is taking the grain and feeding it to the animals. And eventually, you might kill the animal. So, in some respects, you're carrying over the calories from the grain, but it's not very efficient, since the animals are also surviving on the the grain. So, it's a a bad So, we don't have animals in the model. Another more humorous possibility is beer. So, they would take the grain and ferment it and make barley malt, for example, which English people still like to drink quite a bit, no doubt due to this history. But we're not going to do that either. No drinking in the model. So, let's take this storage possibility, the most obvious one is you take the grain and put it in the the bin and potentially have some left or not by next harvest. How rate being delta. Why does grain go bad? Well, in England, it's a very wet climate. If it gets wet and the grain gets wet, if it's not secured, it spoils. Mildew, even if it doesn't directly rain on the grain, is not good for the grain. And grain gets eaten by rats and other rodents. And may think that's humorous, but it is it is real. I know in Thailand, we would go and look at what they call the rice bank, which was a shed where they had stored the rice, and they were showing us the shed, and as they opened the door, all these mice ran out. So, they were eating the grain. How big is delta? We have to calibrate this model. So, drawing from McCloskey, you can see prices increasing month after month after month from harvest up through the next month. And from this fact, McCloskey gets a depreciation rate of 30%. So, here's that table that he's using. Between September and October, grain prices inflated by 3 and 1/2%. Between October and November, 2 and 1/4%. So, these are kind of grain price inflation rate through the year. And roughly, if you add them up and do a little compounding, you'll get 30%. Now, who cares? Well, let's imagine that whatever you store at one date, you get back 1 minus delta that number following say the following month. It's linear. You don't get out what you put in one to one. If the depreciation were 30%, you put in one, you get back 0.7. You put in 10, you get back seven. It's linear. The storage technology displays constant returns to scale. If you were a firm maximizing profits by the choice of how much to store over time, you'd be thinking about this constant returns to scale technology and asking how much would they be storing depending on how much the price next month is relative to the price today. So, it says exercise. So, I urge you to write that down. I did it just to make sure this morning. You know, it's like maximize P1 times Y minus I, where I is investment, plus P2 times Y2 plus I multiplied by 1 minus delta. It's kind of silly be describing this in words. Now, what do you know about constant returns to scale? You already know with constant returns to scale, when we did the production lecture four, that the solution is either infinite or indeterminate or zero. Well, it's not indeterminate, and it's not zero either, because they were storing from month to month at least. So, the premise is that in order to compensate, the economics is, in order to compensate for the loss of the crop, rather than selling it today at a certain price, they can sell it tomorrow, they're going to have 70% left, 30% loss. So, the price has to accumulate at 30% over the year. I'm going back and forth between month and year, sorry. So, that's the economic intuition, and you already know the math. So, that's one advantage for us, having spent some time on production and in particular on profit maximization. So, you know how McCloskey is calibrating parameter delta. Second possibility is putting seed in the ground. Now, it turns out that's a a pretty healthy return. We have some data on that, and the yield to seed ratio is about 2.6 for one of these manners and 1.67 for the other one. So, that's a yield to seed ratio, meaning you get, roughly, taking the average, two units of output for every units of input. So, that's pretty productive. So, if we stopped right there, you know, you'd say, well, putting seed in the ground is a lot more productive than depreciation, even setting aside the risk. Well, so there's a couple of things. One, there's a limit how much seed you can put in the ground, and two, we're going to be considering the risk. So, here's the technology. If you planted KT units of put KT seeds in the ground at T minus one, the output would be a function F of that amount of seed, the harvest. And there's some shock epsilon. In fact, let's assume this is a linear technology. So, whatever seed got put in the ground, alpha times it is the output. Alpha could be two, and epsilon is a shock that's going to move make it random and make it move around. Okay, first thing, you can't plant an unbounded amount of seed. In fact, the best way to think about this is the seed per unit land is constant. You have so much land. There's no point in putting down more seed. So, it's really a decision about how much land to plant. And we'll put an upper bound on that because there are finite land holdings. So, K bar is going to be the most land that you can plant. You plant all of your land. And if you were to try to put down more seed, it's like throwing it away. You're still going to only have the maximal amount, K bar in this case, times epsilon alpha. So, we're about ready to get to a key equation here. Let's do it this way. If you focus on equation one, you've got output F, harvest, which is a function of the seed planted last year, plus an influence of epsilon. Now, you could have had something in storage as well. Imagine they had done both things. So, they put some seed in the ground and they had some storage. You're going to get one minus delta times that amount of storage back. And that would be the end of the story for consumption, except they have to plan for next year. So, they still have these two options, storage planted now at T, that will be available at T plus one, and seed, which they can squirrel away now, which will be planted and influence the harvest at KT plus one. It's kind of weird dating. Please be alert to that. In fact, the last slide for today, we'll we'll change the dating. Different people use different conventions. What I'm complaining about or alerting you is this is consumption at date T, and these things, seed in the ground and inventory, have a date T on them, but they are predetermined from T minus one. They are not decision variables at T. The decision variables at T are labeled T plus one. The pro case for doing it this way is that we care about the state variable at a given date T, and that's determined by the seed that they planted before and the inventory that they have from before. Anyway, just to alert you. I trust you will interrupt me when you have questions. So, this is the amount that's available. This is the amount that you squirrel away for next year. And the the difference between what's available and what you store in one way or another is consumption. Now, they're risk averse. We discussed that a minute ago. So, this is a strictly concave utility function over consumption over a finite number of dates T. So, this runs from one to capital T. Hopefully, capital T is really a long time away, and I'll come back to that. Alternatively, I could have written cap T equal to infinity. They live forever. But we're not quite ready to do that. So, I'm just going to show you finite time horizon first and then argue how to extend it at the end of the class when we get to dynamic programming. Now, this is a discounted utility. Beta is a discount rate. Beta is some number less than one. So, you care about the present more than you care about the future. Present utility more than future utility. This is with beta less than one, geometric. You can see as the horizon gets further and further away, you care less and less relative to today's consumption. But it but these are strictly concave, so you don't like variability. We talked about concave utility when we introduced the concept of risk. Here, even if there were no risk, they would not want they would try to avoid variability in consumption. They'd rather have the same consumption every day rather than having it low one day and high another because the utility consequence of that would be less than having the same amount. So, that's where the concave strict concavity is coming in. So, we already calibrated the depreciation rate at 30%. The discount rate we're going to put at .95, which is kind of standard, although we could vary it. We could make it lower. Once we have the program written, we can plug in whatever we want. Ideally, though, the spirit of this should be that we're somewhat confident about these numbers we're about to choose. Constant relative risk aversion, I defined that utility function last time. That's C to the gamma divided by sometimes normalized. And we're going to set that at .5. This is kind of a low number. It's definitely a strictly concave utility, but they may have been even more risk averse than that. However, you'll see, you know, it's it's ex post justification that that all three of these numbers kind of work in terms of explaining the paradox. Namely, they're about to starve to death every 12 years, and they weren't storing anything. That's the paradox. Back to that epsilon. This is the way you put the uncertainty in. One unit of land planted today yields two tomorrow. So, that's the yield to seed ratio. But there is a little uncertainty. 10 out of 12 times you get two tomorrow, but 1/12 of the time you get one, half of it. And uh 1/12 of the time you get four, double it. So, the expected yield is a little bit above two since the weights 1/12 are same on the low and the high end, but four is two times greater than two, and the difference between two and one is just one. So, it's a the expected value is slightly higher than two. We're going to set the upper bound on storage at one. And this number 1/12, I actually just said why, and you may may be reminded of that when you go back and look at in the data the coefficient of variation that we introduced last time of yields, it was .35, which meant roughly every 12 years output fell below less than half of the mean value. So, we called that disaster last time for these scattered plots. So, this mass over here under the solid density is 1/12. Now, well, it's also true they could get lucky and have a high yield. So, you saw this picture last time. I'm really just using it to remind you of the context of where the risk is coming from. We wanted we just did calibrate the yield to seed ratio in such a way as to deliver a disaster every 12 years on average. Okay. So, let's look at the dynamic decision problem. Let's call this the total available this year and denote on the Y axis, so to speak, the amount of each of the possible savings decision, either storage or planting. Back to the key equation. The amount available is this, output plus storage, and the savings decisions are inventory or seed. So, that's what's being plotted. But we are varying the total amount available going from zero to six. And let's look at Oh. Where is this coming from? So, we actually solved this problem. We maximize two subject to one putting in all these parameters. And I didn't show you the code. You can solve it on MATLAB or something. I'm not asking you to do that. But anyway, it's like this picture could be anything. It's it's it's pinned down because it's the solution to a max problem at the calibrated parameters. So, this is our model version of what they were doing in fact. So, if the total amount available was a little less than one, then they were not planting all the land despite that high yield. However, for anything else, they hit the upper bound, and they plant everything they can at the upper bound, which was one. Uh let's look at the storage decision as a function of the total available. It's only when the total available is three or higher that positive storage kicks in. And when it does, it's it's moving linearly with the amount available this year. It's actually not moving one to one. If it one to one, higher available mean higher storage, it would be parallel with this 45 degree line, but it's not. So, so they never store one to one. They may store less than than the harvest. But they are storing some something here. All right. So, those are the savings plans. Um things to kind of keep in the back of your mind when we go through some of the other slides is the fact that mostly they put all their seed in the ground, but not when the amount available is less and the amount of storage is zero unless the amount available is above three. Now, how much is consumption? Well, back to this thing. So, consumption is the difference between the amount available and those two savings decisions. So, if you look at this diagram, consumption is the difference between the amount available and the sum of these decisions. This 45° line is now playing a role because the amount available, say two, go up to the 45° line, then this amount is two. That's the definition of the 45° line, right? It's a way to convert the units on the x-axis axis to the same amount on the y-axis. That's convenient here because we can look at, okay, here's the amount that's available that would be available next year if they didn't save anything, but they are storing in this case. So, consumption is the difference between this 45° line and the sum of these two other lines. That means that as we get over here, the 45° line is really close to the only thing they're doing, which is planting seed, and very close. So, when you have a less than 1.1 available this year, they're still eating and they're not planting all the land, but they're not eating much. It's a really hard problem. Do we starve now or starve later? I mean, that they kind of gamble on getting a a reasonable return. If they eat the the seed, they have nothing tomorrow. They're starving to death for sure. In fact, I don't know if you've heard this expression, "Don't eat the seed corn." Yeah, that came right out of this episode. Eating the seed corn is the worst thing you could do because you're kind of like guaranteeing zero for tomorrow. But, they actually do cut back. They eat some of it. Okay. So, this is a related way to think about the dynamics. Let's take total available this year, implement those two savings decisions on the previous diagram, and then ask, "How much are you going to have next year?" Sorry if I'm belaboring this. This is the amount available this year. You can save it or eat it. If you save it in one of the other of these two devices, we're now talking about the first part of this equation with the two terms, but the date would be t plus one. So, this would be the amount available next year given the amount available this year and the endogenously determined Oh, I used that word for the first time. Endogenously determined savings decisions. So, so here, you know, we can't plot savings and the total amount available unless we're in some four-dimensional three-dimensional space. So, the savings decisions are suppressed, although they're still active. Well, first of all, output is stochastic. Epsilon could be and likely to be medium, but it could be higher or it could be low. So, the amount that's available next year, even given the savings decisions, will vary with the epsilon. So, if you have three units available this year, which is right on the margin of storing, say zero, the most likely outcome next year is medium, and hence there would be two units available next year. And so, there would be no storage next year, either. So, what do those words correspond to? You're at three units available now. The most likely outcome next year is medium. The amount available next year is two. And again, from the savings decision, storage at three and at two is zero. So, no storage. So, this is like what what would happen over time, but let's just come back. Suppose we re-did this experiment and you had two units available this year, and then you got the medium output. You would have two units. So, what is that referring to? If we have two units available this year and you had the medium output, which is the most likely thing, you get two units available for the following year. So, two in and two out is the most likely outcome if you start with two. So, you could refer to that as a kind of steady state, meaning things aren't moving. Actually, they are moving. So, technically, it's not a steady state because we picked a particular shock, but it is the most likely shock. So, it's approximately true that if you have two today, you're going to have two tomorrow. But, if you had two units available this year and you got the low output, epsilon 1/12 being probability, the output available next year is about one. And at that point, you're not planting all the land. So, that's over here. You could have two today and get really unlucky and have less than two available, about one available next year. And again, if you memorized a bit those savings diagrams with one available, you don't You're not storing and you're not even you're eating some of the seed. So, let's go the other way. Suppose you had two units available, which is roughly speaking a reasonable starting point, and you got lucky and got the high output, then you would finally be storing. Suppose you had two, and instead of getting the medium yield, you got the high yield, then you'd have four units available next year. And at four units available, there's definitely storage going on. So, that's carryover. Rare, but it can happen. Does it persist? Well, suppose, just to run this experiment, that you get lucky and lucky and lucky. So, where where would you end up? You'd end up at roughly a little above four. Four replicates itself. Four in, four out. So, on the 45° line. So, that's the best case scenario, that you would converge to that and you would be storing. But, I'm sure you can see this coming. If you were at that fortunate situation with storage, you got a medium or low output, then you would climb back below three, and you wouldn't be storing at all. So, they had carryover once in a while, but not often. Questions? I have a question. Yes. If the depreciation rate goes down, then the inventory line will move right? It will move up. Yeah, it would shift up. They would Well, it would it would not only have a higher slope, but it would kick in earlier. Yes. So, but it is also implies that in the medieval England, maybe the depreciation rate was so high, so the carryover was so so low or we cannot observe the carryover. Say that again. I mean, maybe it implies that in the medieval England, the depreciation rate is so high, that is why we cannot observe the carryover. Yeah. Well, it Yeah. Yeah. I mean, we picked it to try to match the data. You could run the model with a higher or lower number. If we make the depreciation rate less severe, so it's like 20% is 30%, they would store more often. And then we would have to look at that diagram and see whether carryover is still rare. I mean, I imagine if you go from 30% to 28% or even 25%, not too much might change, but you know, it's clear if you ran the depreciation to zero, then they're going to be storing quite a lot. They would have they would have a way of carrying the crop over one to one. It would still be uncertain, but that would be a big step toward keeping their consumption steady. Uh thank you. Yeah. So, no, it's a great question because that parameter and all the other ones are key. You know, we could vary each one at a time. We could make them more risk-averse, for example. That would be Yeah. an interesting experiment. Um then they're going to be uh more worried about that low output. And that probably means they're going to be you know, trying to maintain more seed and more inventory. Which one, I'm not quite sure. Well, there's an upper bound on the amount of seed and the amount of land they can plant. So, that's probably going to push inventory up. That that's a version of what we saw last time in a sort of static context, namely that you're willing to accept something less than the mean return in order to guarantee to get rid of the risk. So, in this case, even with uh storage being somewhat costly, when they're risk-averse, and we can make them even more risk-averse, they're going to be more inclined to give up some of the consumption today, even though it's going to not survive the rats, into tomorrow because when they have it, they're the consumption today and consumption tomorrow are more likely to be the same. Thanks. Beta matters. You know, if beta goes to zero, you don't care about the future. So, obviously, these guys would be very myopic and they would be behaving very differently and so on. So, all these parameters matter. I guess I got lucky. I mean, um some of these parameters were picked, and I was complaining they seem a little bit off, like beta and risk aversion, but that, along with the depreciation rate and the very carefully calibrated technology did produce succeed in producing the um the data. So, I want to come back to this uh we cheated a bit uh because we had a finite horizon and then I kind of told you to ignore it. That diagram was produced as if the horizon was infinite and how would I know that? Because if the horizon is finite, then unlike those figures, the amount stored is going to depend on how many years are left. You know, intuitively, if you're at the terminal date, sounds kind of foreboding, no? The terminal date. Wow. You got one more year to go, that's it. Eat now. You're not going to be around tomorrow. So, there's not any savings in seed, there's not any storage. Now, you could move back from there to the next to the last date knowing that you're not going to save tomorrow, you're still facing a choice between today and tomorrow or this year versus next year. So, you could solve that two-period problem. We call this working back from the end, right? We start at the last date, we move to the next to the last date, and solve the maximization problem, substitute in the values, and then we have the maximized utility at the next to last date because we've already solved that problem. And then you move to the third to last date, but it's still going to look like a two-period problem because we've collapsed the last date and the next to the last date into one because we already solved this that last two-period problem. I'm going to show you some notation. I'm not sure you could possibly be following me, but I'll say it in English first, and then we'll we'll do it in the notation. Anyway, there's a sequence of two-period problems as you let the horizon T get larger and larger, you're kind of farther and farther away as you go back toward T equal 1. So, the larger is that distance between T equal 1 and cap T, especially when beta is close to zero, it's as if you're solving an infinite horizon problem. And that is the way numerically to solve it, literally. Start with the last date, do the two-period optimization, from the next to the last to the last, plug in the solution, move to the third to last, and so on, and then keep iterating forever. Well, keep iterating until the solution isn't changing anymore. And then amazingly, you will be guaranteed to have solved the infinite horizon problem. So, let me let me show you the math of that. And I apologize the notation changes a bit. This is from engineering or economics, it's called a control problem, the Bellman problem. So, X is the current state, which in our application was the current harvest plus the storage. And Y are the control variables, namely how much you can store and save for tomorrow, that you get to choose. Also, you choose consumption. You choose two out of three, that determines the third. And then you'll get beta times the utility for next year, which in this control problem language is this so-called value function, which has in it Y, the endogenous variable chosen today becomes the state of the world for next period. So, this is the two-period problem. Utility, contemporary utility today, and um utility next period, it's really already maximized utility from tomorrow on, but think of it as a two-period problem, that's the way I was describing it. And what is the overall solution? It has to do with J, the guess about the next to the last date and so on, but T, now here's my apology. T is not time. T is the mapping in this language of Bellman, and I realized only this morning that it might have been better to change the notation to be consistent. But T does not refer refer to the time horizon. T refers to the mapping from the utility from next period on to the utility today. This is the maximized utility, where J is is a given. You're surely going to have questions about this, so interrupt me. Now, there's something called the contraction mapping, and don't worry about the details here, but it turns out we have this mapping T here, and we have the underlying space being a metric space, uh and we have some distance metric, some way to measure distance in that metric space, then the difference between two guesses about V times beta is an upper bound for the distance between the the result. So, so again, here we should think about V1 and V2 as as guesses for the right-hand side, so this would be J1 and J2, and then we apply T to it, and this would be TJ instead of TV. So, that's probably the second thing to correct is to change the V's to J's. Anyway, the point is the distance over here on the left-hand side is not greater, typically less than the distance on the right-hand side. And so, intuitively, what that means is things are going to contract, hence this is called a contraction mapping. Or to put it more formally, if you start with an arbitrary J or an arbitrary V and keep iterating, start here, get this, put it in, do it again, get this, do it again. As as N gets large, and we keep successively applying that mapping to an initial guess value of V, we're going to converge to something called V star. And when T of V star is equal to V star, we we are done. That's the infinite horizon solution. That's like saying the J up here and the TJ is also J. So, the time horizon has disappeared. This is kind of tough stuff. I thought it would better to show you the math rather than be mysterious about it, but let me pause for questions. All right. We have everything we needed in that application to apply the contraction mapping. Our guess for V0 is just the solution to the last period problem. We get the optimized value of you as if you're at the last date, substitute in the maximized utility number, and then go to the next to the last period, and iterate again with that new guess for the second period utility, and just keep iterating, and we'll we'll get to the infinite horizon solution. So, we can think about applying this technique more generally. So, this is a related but somewhat distinct problem. It's uh in a contemporary setting, uh a household trying to maximize utility of consumption, but facing stochastic earnings, say from year to year. And they're limited in the way that they can save, but there is a savings account in the bank at some interest rate or even a riskless bond like treasuries. So, it's like putting stuff in the ground today and getting stuff out tomorrow. That's like storage. In fact, in the first lecture, I said something about these agrarian metaphors. And we just jumped like that, we went from grain in the bin as kind of a storage technology to riskless bonds as another way to store money over time with a with a hopefully with a positive yield. Actually, the real rates are negative right now, so anyway, that's a bit of a problem. So, we maximize utility, and now I I have the courage to put infinite horizon here, so we're going to maximize discounted expected utility over the infinite horizon. Beta is still less than 1, this thing would converge, so this term is finite, and the choice will be over how much to eat and how much to save. So, the way to think about this is you're you're coming in with a realization of your stochastic income, you're having the amount of the bond that you bought last period, uh you get that back plus the interest rate. That's the resources available today at T, and you can either eat that or buy a bond and carry it over to the next period. So, this looks really similar to the medieval storage problem. A couple of comments about it as I tried to alert you, the timing here is the date the dating is different. We're now at T, this is no longer a state variable, this is the decision variable. Bonds to be coming due next period. Likewise on the right-hand side, this doesn't have a T, it had a T minus 1 because the decision the savings decisions are dated at the time T that you make the decision rather than the time T that where the yield comes in. And this is also standard, you just have to keep an eye out in terms of what the authors have in mind. And you know, this starts at date zero, so you need some initial conditions, which should have been K at T equals zero minus 1 or K1. This is also a typo, this should have been K minus 1, not K0. And the Y0 is fine, that's the contemporary yield. So, I apologize about these typos, although it is kind of an interesting story. They're not mine. I just I took this slide from somebody else, and they had actually made the mistake, and it wasn't corrected in the publication. But we'll fix it. The other thing about the typos, I know you hate them, I do, too. When we did that input-output matrix for Leontief last when we did the last part of lecture four the first part of last lecture beginning of lecture five we did we went back over lecture four and I was saying how much of the service sector you need as an input produce one unit of output of services how much do you need from manufacturing and raw materials it was right in the slide the typo had been corrected but that typo exists in in Wikipedia and it's still wrong there so I'm what I'm saying is I work very hard to eliminate the typos but sometimes they still persist if you're reading carefully though you can spot them and please don't hesitate if you find other things that you find questionable that's the bad part then it becomes confusing about whether there's a typo or not so if in doubt ask us and we'll we'll try to answer questions. Here the Katie could be negative right? It in the last term constraint and say that a Katie just larger than minus K. Oh good question. So So So borrowing is allowed. So I refer to this as a riskless bond as if you could save but in fact if savings is negative you're borrowing. So if you multiplied through by the negative sign you'd have a minus KT less than or equal to K bar. So KT positive for savings minus KT would be now denoting borrowing and borrowing has an upper bound K lower bar. So this formulation of the problem allows borrowing but it's limited. So that's a great question thank you for Is it clear? Yeah. Um another question is what is the difference between the self-insurance and full insurance? In the self-insurance that means we cannot borrow or we can borrow but there were some constraints like this we can borrow at a certain range or Yeah so I actually took out slides that I used to have. Um Self-insurance is what we've been studying the whole lecture because it's like one village or one person and they're pretty much in isolation here it's still self-insurance although it there is this bond or this borrowing possibility at a fixed interest rate which I didn't tell you where that came from but this is like partial equilibrium problem in which the agent can borrow or lend with the rest of the of the economy and and and that's still referred to as self-insurance although it's a bit misleading because one person's saving is another person's borrowing but but anyway we're not doing the general equilibrium here we're taking the interest rate is given. But the other thing would be what if there were multiple people like this? And and some people had high income at a given date and other people had low income at a given date. Then we're back to thinking about whether they could make deals with each other. So we'll get there actually next time but the risk sharing problem would be um an extreme version of that is get rid of the storage all together and the seed get rid of I'm not the seed get rid of the storage get rid of the bond but imagine these guys can contract with each other write financial contracts or informal risk sharing arrangements with each other so as to pool the risk that they're facing. Like in the village you know they had scattered those strips so they were trying to stabilize their starting point but on top of it they could be borrowing and lending they could be uh making a deal if I'm high and you're low I'll give you money and likewise vice versa. And that's not self insurance that's kind of dynamic uh explicit risk sharing insurance. Thank you. Yeah no it's a great question as I said I I used to have more slides here about that and I thought it was a little too early in the class but there are clearly trade-offs about um how can I put this whether or not you want to do self-insurance as in a dynamic problem versus whether or not you want to do uh risk sharing as in but do it repeatedly over many dates. In fact the very first lecture after we did economic science we went through uh some example economies and I gave you five of them and the very first two were one medieval villages with these scattered strips that we've been studying now um last time and today implicitly uh or the Thai temple scheme remember the Thai temple scheme? They um had lowland and highland if the monsoon rains were really abundant the lowland flooded too much and they only got crops off the highland and when the monsoon rains were adequate or low then the highlands failed and what they did was contribute their crops to the monks not for eating I but that really threw me because I know monks don't uh um I mean you can feed the monks but the monks don't feed you but nevertheless they were putting their crops under the auspices of the temple with the monks and then handing it out to people that had either drought or flood. So that's an active risk sharing scheme and your question comes back to that which is comparing and contrasting these two different ways to handle risk. Okay. So um this is lecture six there aren't quite as many slides here but you can see there's some very important concepts and and we've now succeeded in having both time and uncertainty and uh dynamic programming techniques for solving that so um but we'll come back and use those techniques in in subsequent lectures next Tuesday is a lecture on Pareto optimality uh which is finally the full-blown general equilibrium although we've kind of been going back and forth and getting very close to that in the last couple of lectures but in the lecture on Tuesday we will fully embrace it and talk about efficiency. Okay that's all for today.