Markov Chains for Quant Finance

Key Takeaways

[Music] Marov chains are extremely popular probability models due in part to their versatility. They can be applied in so many different ways. They can take what seems to be a very very difficult problem in probability and turn it into something that is so easily solved. So it's no wonder I see everywhere in my Discord and my YouTube comment section requests for a video on marov chains. applied to quantitative finance and of course advanced extensions, these regime switching models, this idea of a hidden marov model, so on and so forth. But we got to start somewhere. So what we're going to do in this video is take a look at marov chains in the context of quant finance. To motivate and understand this idea of a marov chain, we're going to look at a real world example from quantitative finance. This is a model that a quant would actually build in the real world. In fact, I've actually seen it built and deployed in this capacity at a firm. So, what we're going to do is start with this idea of a random variable. Talk about then this idea of random variables over a series of steps, a stochcastic process. Then we're going to talk about this independence assumption and realize very quickly in our real world example, this independence assumption is not going to be sufficient for our model. And that is where marov chains come in. And then we'll talk about Markov chains. We'll talk about the assumptions. We'll talk about the application. And we'll wrap up with some closing thoughts in future topics. Very rarely when you're building some sort of quant is it safe to assume independence. However, when it comes to a Quant video, it is always safe to assume that you will have access to a Jupyter notebook. This Jupyter notebook will be linked in the description below. I will also post it to the Quant Guild library on GitHub where you can find all of my Jupyter Notebooks and associated YouTube videos along with all of the source code for my Quant builds. At the top of this Jupyter notebook, you'll notice some related Quan Guild videos applying statistics to finance and trading, covering ideas like retail versus institutional trading, trading versus investing, time series analysis, its place in quantitative finance, why the expectation or a nonlinear conditional expectation in a machine learning sense is the best forecasts we can produce in the face of uncertainty and randomness. All of these videos applying statistics are certainly candidates for applications of marov chains. So if you are unfamiliar with any of these ideas, especially the idea of applying statistics in the space, this idea of overfitting, this idea of forecasting, I highly recommend that you check these videos out before approaching one like this. Moreover, these videos certainly take a lot of effort to create. So, if you'd like to help support the channel so I can continue to create videos just like this one. Please like, comment, subscribe, share, check out channel membership. It helps me out tremendously. It is so greatly appreciated. And if you'd like to master your quantitative skills, check out quantankgild.com. Maybe you come from a background in business, economics, physics, and aren't too sure where to get started on your quant journey. Maybe you are a working professional and you're looking to sharpen those quantitative skills. In any case, Quant Guild is for you. We have over 90 Quant lessons in math, probability, finance, an adaptive practice engine that scales with your skill level so you can progress to more and more difficult questions with gamified rank-based progress. Interview questions with fully worked solutions, trading games coming soon, courses from A to Z in math, statistics, coding, and all that's included with Quank membership. and you can get started right now for free. So, if you'd like to help support the channel and master your quantitative skills, check out quant.com. Without further ado, let's get started with this idea of a random variable so we can build towards this idea of a stochcastic process and this real world example from quant finance. So, what are random variables and why do we use them? Well, anytime the outcome of an event is uncertain and we want to consider various possible states of the world, we generally model these events as random variables. Now, whether or not these events are truly random is subject for discussion. That's definitely a topic for another time. But in any case, when we go about modeling this uncertainty as a random variable, we typically assume some sort of distribution. And what that does is it tells us something about the likelihood of different outcomes for that particular event. So what does this actually look like? Well, here I have three different random variables. I have X, Y, and Z. And these all have different distributions. So X is a normally distributed random variable. Y is distributed via a binomial distribution. And Z a Berni. Now, it's tempting to treat these variables like deterministic quantities from algebra, pre-calc, calc, so on and so forth. But x, y, and z here are not numbers. They are distributions. Whenever you have a random variable, whether it's in an equation or you're defining it as such, you need to think of these letters literally as distributions. They represent sets of possible outcomes. It's not just one outcome. So whenever you see it in an equation, think, hey, that letter literally represents a distribution, a set of possible outcomes, all with different probabilities and likelihoods. So I actually plot these distributions for you. Here we have X, we have Y, and we have Z. All of which follow different distributions. So this letter X literally means this distribution. Y means this one and Z means this one. This one we have a normally distributed random variable, binomially distributed and a Bernoli distributed random variable. All three of these are fully characterized by their distribution functions. And those distribution functions enable us to visualize the subsequent distributions just as we see them here. So these random variables are literally these charts as we see them here. Now it's really important to note whenever we define something as a random variable, we can't say anything about what the outcome is going to be. It just tells us something about the likelihood of different outcomes. So let's look at Z for example. The distribution is fully characterized by this bar chart. there's a 70% chance of observing one from Z or a 30% chance of observing zero. Now, even though it's extremely unlikely, we could still see 10 zeros in a row from Z. So when it comes to modeling random events with random variables, all we're trying to do is get a sense of the likelihoods associated with different outcomes. That's all we're trying to do. So if I'm looking at returns in my portfolio, for example, I'm probably going to consider, hey, what is the likelihood of observing an extreme loss? maybe something like 10 20% in a day or a week that the likelihood of that event is definitely going to be of interest to me. And if it's not to you, then maybe you shouldn't be a portfolio manager. But in any case, this is how we use randomness in practice. We're trying to answer these questions about different states of the world by tying different likelihoods to these certain outcomes. So what is the elephant in the room? the question that your finance professor isn't going to want to answer. Well, how do we go about coming up with these probabilities? If we're interested in all of these states of the world, then how do we effectively come up with a random variable, the parameters for that random variable or the likelihoods of all of these outcomes that are uncertain? Do we use historic data? But I thought historic data was an indicative of future performance. all those data points already happened. It says nothing of what's going to happen. I mean, think of it this way. If we have Apple and historic data was a reasonable proxy for what was going to happen, then how far back do we look? Do we look back a week, a month, a year, 5 years, 10 years? Apple was an entirely different company 10 years ago. Not to mention that the overall macro climate, the administration, everything was entirely different 10 years ago. That is nowhere near reasonable to include that in a model for today. So what I'm getting at here is the data generating distributions for these uncertain events change over time. To understand what this actually means, I have an example here. So imagine we're looking at stock returns. This purple distribution governs the different likelihoods of returns we observe on any given day. And what's going to happen is this distribution is going to change over time. But we can't observe it. All we can observe is the outcomes of this distribution. So what you'll notice is as I play this animation, you can see that we're realizing values. what in the extremes over here and over here. If we just model it as a purple distribution, then it's going to look like, wow, this was really unlikely. It's also going to look like, wow, this was really unlikely, right? But look at the animation. The likelihood changes over time. At one point it was very likely to observe values over here and at another point it was very likely to observe values over here. This is more akin to what we see in reality. We don't get to observe this data generating distribution but certainly the probabilities and likelihoods of different values change over time. Now there are very effective ways to model this time variance that we're seeing in this animation. In fact, I did a video on the arch and GACH models last week. So, if you're interested in modeling volatility, I highly recommend you check that video out. But in any case, we saw conditional heteroscadasticity was able to better capture the dynamics observed empirically by making the volatility term vary over time based on what was just recently observed in the process. So there are effective ways of capturing these dynamics, but acknowledging that they're there is the most important step. It's the first step and the most important step. Otherwise, we're just going to go and produce naive models that omit this very important reality check. Now that we have a good understanding of random variables, their place in modeling, and some important considerations in quant finance, we can approach this idea of a stochcastic process. And it sounds crazy, but all a stochcastic process is is a random process. Well, we know randomness very well at this point. So, what is a random process? Well, it's just a series of outcomes of a random variable or a set of random variables over a series of steps or time. So, it's just a random variable indexed at different points. And you get to define what those points are. So I have a very simple example here to hit this point home. This is a stock portfolio whose value is governed by a dice roll. So a dice roll is a random variable. I'll call it D. We know D then is a distribution of possible outcomes. We know it's a dice roll, so it's got to be 1 through six. I plot the distribution here. This is a discrete uniform distribution. And every single day I am just going to roll a dice. And that's going to dictate my portfolio value for that day. And you can see here I get this sample path for my portfolio value for the next 100 days. Now, if I was to reimulate this, I would get a different sample path and so on and so forth. That is what a stochastic process is. Typically, you'll be able to simulate some sort of path. That path may have a whole bunch of properties and we we're going to consider and analyze those properties. that is what is of interest to us. So here for example on day nine my portfolio value is five. But if I was to rerun the simulation it could be anything from this dice roll. That is the general gist of a stochcastic process. So what was the point of this example with the dice rolls? Well, we're trying to approach these marov chains. We need to understand this idea of independence. No matter what information I give you on any day here, whether it's day 49 and we're looking to figure out what the portfolio value is going to be on day 50, it doesn't matter. Any information that I give you is not going to influence the likelihood of the next outcome. The next outcome is still going to be a dice roll. Every single dice roll is independent. That is the key here. Especially when we're looking at this stochcastic process. No matter what information I give you, every single day is going to be governed by this random variable outcome. When we look at marov chains, this is not the case. And we're going to see why in a moment by looking at a violation of this assumption in practice in our realworld example. Our real world example has to do with a portfolio of loans. So imagine you're trying to analyze the risk associated with this portfolio of loans. Maybe you work at some sort of credit agency and you're particularly interested in modeling these different states of the world. Well, we need to consider this process of loans over time. Let's break down this real world example to motivate why we want to use marov chains to model this portfolio. So this is a portfolio of dozens and dozens, hundreds and hundreds of loans. And each loan at any point in time can either be current, 30 to 59 days late or delinquent, 60 to 89 days late, or 90 plus days late. And we're going to just exclude default altogether for now. Okay. Well then, what does this look like for our portfolio? Well, we have a portfolio of a lot of these loans and we can look at the overall proportion of loans in that portfolio that are in any one of these states and I have a pie chart to break this down as an example here. So, this is our loan portfolio. In this particular portfolio, it's not doing too great. 50% are current, 20% are 30 to 59 days late, 60 to 89 days late. we have 20% and then 10% are 90 plus days late. That is a rough portfolio of loans. So what's the deal here? Well, this is not just going to stay the same every single month. It's going to change over time. So what we're going to do is we're going to try to model the proportion of loans in each of these states over time by simulating it as a stochcastic process. This is exactly what we did here with the dice rolls. So, what I'm going to do is I'm going to look at historic data. I am going to measure a proportion according to this estimator. Not going to dive into the weeds of that right now, but what I'm going to do is I'm going to try to simulate a couple of months of this portfolio to see what the overall proportion of loans that are late are in each category. Okay. Well, let's do this. So in month zero, I have 100% current loans in my portfolio. All right. Well, this is exactly like starting right here in a sample path. Okay, we're starting at this initial state. Then I'm going to simulate a draw, simulate a draw, and keep marching forward in time. But instead of visualizing it as a path, I'm going to visualize it as a bar chart. So we can look at the proportion of loans in each category. So here what I have is I have month zero. Everything's current. Month one. Okay. Now we're at 81% are current. Now we have 4% are 30 to 59 days late. But hold on a second. We have 11% are 60 to 89 days late and 4% are 90 plus days late. These states are not possible because this is just stepping forward in time one month. A loan in our portfolio can't jump from current to 90 plus days late in one month. Clearly there's a problem with this construction with this simulation. This is the perfect example of how naive independence does not match reality. If I assume that each step is drawing from these distributions, whether they're the 30 to 59day distributions, the current distributions, so on and so forth. If I assume every single month, we're just going to draw independently from those distributions, I'm going to end up with something like this, something that actually is not possible in reality. And we want to create a model that's as close to reality as possible in a parsimmonious way. We're not doing that at all here. We just went from all current loans to some that are 60 to 89 days late, 90 plus days late. That's literally impossible. So independence is completely out the window here. Now, if you're thinking maybe what I'm thinking, then maybe what we should do is restrict this simulation to say, hey, we can only draw values that are 30 to 59 days late in this next step. And then in the next step, we can only draw values in the next bucket up if we observed values already in the previous bucket. And if you're if you're kind of catching what I'm floating out there, really what you're doing is you're starting to impose this idea of local conditional dependence. And that is effectively captured by a marov chain. So maybe you beat me there, but that is this idea here. We want to navigate away from this naive independence because it is not capturing reality. We can't possibly observe a transition like this. But if we model it as a marov chain, we will never observe a sequence of evolutions that cannot happen. That is the power of these marov chains. this local conditional dependence, this marovian structure, and we're going to discuss all of these things right about now. Now, a marov chain can be diagrammed extremely effectively. So, I actually want to start with the diagram and then move towards the properties and applications. Let's look at this diagram. It looks kind of crazy, but it actually makes a lot of sense. So what I have here is I have what's called a state transition diagram for this marov chain. And effectively what I'm doing is I'm modeling each loan in my portfolio and I'm saying hey the loan can only exist in one of these four states. Again I'm excluding default for the sake of this example. So a loan can either be current. It can be 30 to 59 days late, 60 to 89 days late, or 90 plus days late. And each of the arrows here dictates where it can go from each state. So if it's current, it can stay current. So the arrow points back to itself, or it can jump to 30 to 59 days late, but it can't jump to 60 to 89 days. It can't jump to 90 plus days. And that makes sense because each one of these transitions, the arrows out, represents one month, 30 days. You can't possibly just jump to this orange or red circle. And you'll see similar logic at each one of these other states. 30 to 59. Well, I can go back to current or I will stay 30 to 59 days late or I will transition to 60 to 89 days late. I can't jump from 30 to 59 all the way to 90 plus. Remember, each increment is with is a 30-day step. So, I can't just transcend time and jump to 90 plus days. That would be like what we observed in this naive example here when we saw one month step had 90 plus day late loans. It doesn't make any sense. So effectively this transition diagram is capturing the dynamics of those loans that we're looking for. And that is like the perfect example of a marov chain bringing what would otherwise be a very naive model closer to reality. we're better capturing the transition of each of these states and the associated probabilities. Before we get into the probabilities, let's talk about now some properties of marov chains to actually understand how they work. When it comes to marov chains, there are a couple of key properties that I want to highlight here and we'll talk about some more properties along with the assumptions in the application below. But for now, let's talk about this idea of the memoryless property, the time homogeneity, and the finite state space. The finite state space is probably the easiest idea to understand. We can only have a finite number of states. Okay, not too much of a stretch there. But what about this idea of time homogeneity? Well, transition probabilities are constant over time. This is a tricky problem because we're assuming that these transition probabilities from each of these states. So that is maybe this current loan state to the 30 to 59 days state, current loan to current loan, so on and so forth. All of these transitions are constant over time. Now based on what I showed you up here, is that likely to be the case? probably not right. So very important to consider that. What about this idea of the memoryless property? Well, effectively this is saying, hey, it doesn't matter where I've been. It only matters where I am. So it doesn't matter if I was 90 plus days delinquent or 60 to 89 days delinquent in the past. If I'm current now, that is what is going to dictate my transition probability. That is the probability of going from current to current or current to 30 to 59 days late. Now whether or not that's reasonable, it depends on the dependency structure of your problem. Maybe if you spent a whole lot of time in 90 plus days delinquent and you get back to current, maybe you're at a higher risk of going back to that state. That's not going to be captured in this particular model. You'd have to expand your state space. it would create a whole bunch of complexity so on and so forth. So when it comes to these marov chains those are some key properties and considerations when you go about constructing this as your probability model. Now what's really cool is this diagram that we've observed can construct what's called a transition matrix. And this transition matrix contains probabilities of transitioning from and to all of these different states. And here we're dealing with months. Okay, so keep in mind in this example we're looking at transitions that occur from from a month-to-month perspective. Okay, so this transition matrix looks something like this. There are probabilities and there are zeros. Zeros represent a zero probability of transitioning to that state from another state. And this makes sense, right? So if there are no arrows, then there's going to be a zero probability of transition. So for example, I can't go from current to 90. That's not possible. So the probability of a transition from current, you read it in the rows first. Current to 90 plus is zero. And we can read this for any state transition. So what is the probability of going from 30 to 59 days to 60 to 89 days? Well, I'll go to 30 to 59 and then I'll find my column 60 to 89 and that is my probability of transition right there. It's P sub23. So that is how the transition matrix works. I find in the row my current state and I find in the column the state that I am going to transition to and then I can read out the corresponding probability. In fact, these probabilities can be filled in and I have an example here. Here are a whole bunch of probabilities that I've estimated from data. And we're going to look in a moment how to actually come up with these estimations from data in practice. But for now, I just want to show you how to fill in a transition matrix. So what I'm going to do is I'm going to track the transition of each of these states to different states and to themselves. And I'm just going to go ahead and fill in the transition matrix. So for example, the transition of a current loan to itself has a 0.99 probability. So I'll go down to my transition matrix, current to current.99. And I'm going to do this for every transition. And remember, this is on a monthly basis. And you can see here I have a completely filled out transition matrix with all of the probabilities of transitioning between different states. Now that we have this, we can answer some really, really cool questions. Now, here's an example question that we might want to ask. What is the probability of starting in the current state and then after 3 months ending up in the 90 plus days delinquent state? What is the probability of that? Well, we need to be able to somehow compute multiple state transitions. So I'm not just talking about a one-mon transition that's given by these probability values. But now we need to consider multiple steps. This is effectively what we were trying to do here. But remember we have this independence assumption. This didn't capture reality here. We're using marov chains. So we're going to be able to capture that dependency structure. How do we answer that probability question? Well, we can use the Chapman kmagurov equation and that is going to let us compute those arbitrary step transitions and associated probabilities. In other words, if I wanted to figure out what the probability of going from current to 90 plus days delinquent was, I could just use the Chapman Kagrov equation and then find the entry in the corresponding matrix and that will give me my probability. Now, who in their right mind would derive this result that we're seeing here from the Chapman Kagra equation? Well, I would. Believe it or not, I have a very, very old video and article that fully deres this result that we're seeing here. So, if you'd like to check it out, I will leave you to it. Nevertheless, the actual implementation of this result is very, very easy. Literally all we're doing to figure out these probabilities is multiplying this transition matrix that we filled out together from our transition diagram with itself. And the number of times that we multiply it together with itself will dictate the number of steps in the future. What a a tremendously elegant result. Here I have a couple examples of the really cool questions we can answer. So for example, what is the probability of going from current to delinquent in 12 months? Well, I can apply the result from the Chapman Kagrav equation. I can see here I'm taking the matrix power. So this is the transition matrix that we defined together up here from the diagram. And all I'm going to do is I'm going to multiply that matrix with itself 12 times. and then I am going to get the corresponding row and column to that transition. So remember if I'm looking for current to 90 days delinquent then what I'm going to do is I'm going to look in the zeroth row and the 0123 third column. And that's exactly what I do here. I have the 03 column. I run this and I can see that there's a 1826% chance of me going from current to 90 plus days delinquent in 12 months. I take this one step further and I show actually all of the transition probabilities for 12 months. You know, starting from these different states, you could start from current 30 to 60, 60 to 90, 90 plus and we get a whole bunch of different probabilities. And this makes sense, right? Because if you start in 90 plus days delinquent, then the probability of you going from 90 to 90 is probably higher than going from current to 90. And this is exactly what is captured these dynamics in the marov chain with this local conditional dependence and these state transitions. It is extremely impressive and very very cool. All of these questions that would otherwise be very difficult to answer, especially with some sort of naive model like this are very easily answered with what I mean this is only a couple lines of code. I do a matrix power and I get the correct index from the matrix. I mean it it doesn't get much easier than that. All of the hard work was done by marov the structure of the marov chain and the chapman kmagraovv equations. So now we can go ahead and answer really cool questions. So we looked at the probability ones. What about the idea of an expectation? Now I'm not just going to consider the expected value of a marov chain in that sense. I'm going to consider this idea of a state distribution vector. And what this is is a probability distribution across all states at time t. So this could be month zero, month one, so on and so forth. And this is going to tell us something about the likelihood of states at different points in time. If this sounds familiar, well, this is precisely what we were talking about in this naive example and in the context of modeling uncertain outcomes with random variables. So that is precisely what we're doing here when we use this state distribution vector. We're looking at this likelihood of being in any of these states at a given point in time. And of course as time progresses those distributions are going to change. And it turns out in this particular case they will actually converge to stable values. A very very important idea. Now until we actually converge to those stable values, the initial distribution is going to matter a lot and it's going to change the likelihood of observing different different distributions at different points in time. And this makes sense because if we think about it, let's look at this example here. I have 50% current, 20% 30 to 59 days, and then I have 30% 60 to 89 days, zero that are 90 plus days. And I can look at the expected proportion of loans that will be 90 plus days delinquent after 12 months. I can see it's roughly 7%. Okay. Well, what if I started with 30% 30% in 90 plus days? Does that mean that my expected proportion of loans is going to go up or down? Well, if I'm starting with more loans at 90 plus days, you better believe this is going to be higher. Let me rerun this. 27%. Now the expected proportion of loans that will be 90 plus days 27%. So this initial distribution certainly matters a lot. But what if we increase the amount of time? Then does it matter? What if I make this 100? Okay, that goes down to 13. What if I make this a th00and? That goes down to three 10,000. 3.8%. Okay. So, after 10,000 transitions, I'm seeing something like 3.8% here for 90 plus days delinquent. Okay. Well, what if I start in that original state? So, instead of 30% being 90 plus days delinquent, I'll go back down to 60 to 89 days being 30%. When I run this again, wait a second, we still see this 3.8% 8%. That is the convergence that I was talking about here. In the short run, there's going to be a degree of variability based on where you start in the proportion of time spent at each of these states. And that's going to be largely based on your initial distribution. Like I said, if you have more high-risk loans at the start, then yeah, in the short run, you're going to be at a higher risk. But in the long run, if we continue to observe the transitions that we saw in the diagram, then we actually observe this convergence. And we can actually see it in a chart here. We can see all of these state distribution vectors converging to these probabilities. And it doesn't matter the initial state vector. That's exactly what we saw here. We saw that it converged to 3.8% in the case that 30% was the 60 to 89 days or 30% was the 90 plus days. So that is a key property of this particular construction. Now there are other key properties of marov chains. Recurrence, irreducibility, this idea of a steady state, absorbing states. This is how you would likely model some sort of default periodicity, arrogicity, and transients, communication classes, so on and so forth. I would certainly need to do another video to discuss all of these. So, if you'd like to see another video on advanced marov chains and then maybe extensions in hidden marov models and regime switching models, please like, comment, subscribe, let me know down below and we will see what we can do in a future video. Nevertheless, we will continue on now to the application. How do we actually come up with states and probabilities if we're going to go about using these marov chains in practice? We saw an example above where it was relatively easy to define the states. They were almost just given to us, right? We know a loan is going to be characterized either by current 30 plus or 30 to 59 days delinquent so on and so forth, right? Those states are presented to us. It's very easy to construct states. I have a couple examples here of volatility, market, trend, um, liquidity. You could do like low, medium, high, bullish, bearish, sideways. It's very easy to come up with states. But how do we actually go about estimating probabilities? Because if we can define states and you know this could be for some sort of you know regime switching model if we're trying to to go into hidden marov models um whatever it may be that we are actually trying to develop a model for defining the states and the transitions and the subsequent transition matrix. That typically isn't the hard part. The hard part is how do we estimate these probabilities? Right? You could probably draw me a diagram like a very nice diagram and come up with the transition matrix. But how do we actually get these probabilities? That is the question that we have to answer. And to answer this, we need to understand this idea of maximum likelihood estimation. Maximum likelihood estimation is actually extremely intuitive. Whether or not it is actually effective, that's subject for discussion, especially after the animation that I showed you up here with the way that these distributions can change over time. Nevertheless, the idea of maximum likelihood estimation is to observe data. So for example, we observe this gray data and the gray data was generated by the red dashed distribution. And what we're going to do is we're going to use the MLE to fit the distribution to the data we've observed. It's going to produce the highest likelihood of generating that data. It's the distribution that has the highest likelihood of generating the data that we've seen. That's very intuitive, right? What other distribution do you want to use beyond the one that had the highest likelihood of generating the data that you've seen? And you can see in this example here, it does a pretty darn good job. It tracks that red line very very closely. So if we understand this idea in this setting, well, it's the exact same thing for marov chains. We can apply this maximum likelihood estimation and you can see it will actually converge to the true probabilities of our transition matrix. And this is only going to be true if a whole bunch of assumptions hold and so on and so forth. But nevertheless, it is a very effective way of producing these probabilities. And the maximum likelihood estimation has a whole bunch of very very nice properties. As we collect more samples, our empirical probabilities converge to true probabilities. That's what we observe here. These estimates are consistent, which means more data leads to better estimates. The convergence rate follows the law of large numbers, but early estimates can certainly be volatile due to small samples as we see here. So, we do have all of these very nice properties associated with this with this idea of maximum likelihood estimation. But again, whether or not it's actually appropriate, subject for discussion. Maybe there's different things we can do to make it more or less appropriate. Um, especially if we consider how these distributions change over time. But as a first pass, this is one way of going about estimating these probabilities. So how do we actually do this? Well, we we need to derive the maximum likelihood estimate estimator for a marov chain. To accomplish this, we have to maximize the log likelihood function. And I'm not going to walk through all the math. I actually have a video, again, another very old video on this topic. So, if you'd like to check it out, I'll leave you to it. But nevertheless, once we derive the maximum likelihood estimate of the transition probability, we can see it's extremely intuitive. All it is is the simple proportion of transitions out of the state to a different state from the original state. It is essentially how we would intuitively go about coming up with this proportion. Anyway, it's just like how if you wanted to come up with a raw probability estimate, you would just take what? The total number of favorable outcomes out of the total number of outcomes. It's effectively the exact same thing. And it's kind of cool to see such a heavily computational process spit out such an intuitive result and we can use that to actually develop the probabilities in our transition matrix from data. So what I have here is I actually have a synthetic loan transition data set. something that you could observe in practice a whole bunch of different loans and we can apply the MLE estimator to derive these probabilities and that is exactly what I do here. I'm actually using this data to produce these transition probabilities. And what you'll see here is I have my transition matrix that is estimated from the data and I have the true transition matrix. You can see it tracks very very closely and as I observe more data in this particular case it will converge to this original transition matrix thanks to the probability or thanks to the uh associated properties and assumptions of the MLE and the marov chains. So what are these assumptions that allow us to produce these probability estimates? Well, the Markoff property, the future state depends only on the current state, not past states. Time homogeneity, transition probabilities are constant over time. Arrogodicity, there's a unique stationary distribution, sufficient data. We have enough transitions that have been observed to estimate all probabilities and independence. Each loan in this case, the transitions are independent of other loans. And you know, two and five in this case are going to be our biggest concern. This goes into the idea of clustering, right? If there's a whole bunch of defaults, you know, maybe everybody's losing their job all at the same time, that independence assumption is likely going to be violated. Now, this isn't independence in the sense of states. This is independence in the context of a series of loans in our portfolio. Moreover, this idea of time homogeneity, whether or not these transition probabilities are constant over time, yeah, that's going to be subject for discussion. Again, this goes back to the animation that we looked at all the way at the start of this video, right? These are certainly going to change over time. So, in terms of the assumptions that are going to be violated in the MLE and the Marov chain, that is these probabilities that we're generating for this transition matrix. two and five are certainly some big concerns. Of course, this idea of the marov property that is also a concern because there could be this longer range dependence. Maybe we need to expand to more states so on and so forth. But nevertheless, if if we do make these assumptions, then we get all of these nice properties like I mentioned earlier, consistency, asmmptoic efficiency, asmmptoic normality. That's a huge one. The distribu the distribution approaches normality as the sample size grows. Invariance, maximum entropy, some phenomenal properties we get, which of course may or may not be true based on whether or not and how severely the assumptions are violated. Too long, didn't watch. Here is your executive summary. When modeling something as a random variable, independence is way too strong of an assumption that can lead to extremely inaccurate models. We saw how jumping from month zero to month one led to us modeling 90 plus day delinquent loans from all current loans. In 30 days, a loan became 90 plus days delinquent. That's not possible. So independence in that context made absolutely no sense. A major correction can be applied by considering the simplifying assumption of local conditional dependence rather than full independence. That is this idea of a marov chain. We can effectively model these dynamics these states and transitions to and from different states based on what's possible in this local dependency structure. This is governed by a transition matrix with corresponding probabilities that can easily be estimated from data using the result from the MLE we saw above. Now though marov chains aid in the modeling process offering more accurate or reasonable I think would be a more appropriate way to state it estimate. There are many assumptions that are still going to be violated in practice. So this is far from a cure all. Typically these Markoff chains are a first step in the modeling process. If we were to consider some sort of regime switching hidden markoff model for example, understanding these is a first step toward more comprehensive applications. Some future topics I would like to discuss technical videos and other discussions advanced marov chains. There is so much to discuss when it comes to marov chains. I'd like to break out the iPad, answer a whole bunch of really interesting questions. Maybe some some Citadel Jane Street interview questions, tackle these ideas of communication classes, arrogadicity, stationary distributions, initial state vectors, absorbing states, so on and so forth. If you'd like to see a video on these topics, please let me know in the comments below. I would certainly love to do one on advanced marov chains. of course other extensions too like hidden marov models and other implementations. I'd also like to get back to more quant builds. I still want to build that earnings event options trading dashboard. I'd also like to do a live cowman filter model with regime dynamics. I think that would be so cool along with this automated delta neutral trading system kind of algorithmically capitalizing on vol speculation. We could even use marovive chains and other time series models to aid in these speculative positions. I've been fleshing out this video for for quite some time, too. So, let me know in the comments if that is something that you would like to see. And that's going to do it for this video on Markoff chains for Quant Finance. I hope you enjoyed. I hope you learned something. This video certainly took a lot of effort to create. So, please like, comment, subscribe, share, check out channel membership. It helps me out tremendously so I can continue to create videos just like this one. Check out quankill.com to master your quantitative skills. Other than that, thank you so much for watching and I will see you in the next video.