Lecture 9: Principal Component Analysis in Finance

Key Takeaways

All right. I want to welcome Stfan Andrev uh as guest lecture today. He has been a significant part of this course over the past I think 11 years. Is that right, Stefan? A long a long period and >> count that far >> partly because his uh lectures are the most popular in the course and we really are glad to have him come back again. His background um is very interesting. He has a PhD in chemical physics from Harvard and he has worked at the top firms on Wall Street. Morgan Stanley Cedell and now to Sigma. So he's a really great person to uh share his expertise with us. So thank you. >> Thank you. >> Okay. >> Thank you. Thank you guys. Um very nice to to meet you all. Big class big class this year. That's exciting. Um I like I I'll start with this before everybody gets gets tired. But I wanted to say that like I we're going to talk about principal component analysis today and the some of the main things I want you to take away from this are this is a absolutely core data science tool in finance. That's number one. Number two, it's great to like this kind of stuff you only learn by doing it. So doing like actually doing some kind of project that where you use this toolkit is um is like essential for you to kind of relearn it. And um three if you ever want to get a job in finance like I think having having like some experience having like done a project with this and having some familiarity with what this is about might help you during interviews quite a bit. Um we're going to have lectures the slides are minute we're going to have some the slides are on the web on the classroom website. In addition to that there is um there is a notebook and a Jupyter notebook and a data set that I've used to prepare these slides to at least the first the first example we're going to talk about. I've um like I think that that's like that's a great reference if you ever want to like like see how you actually like how this stuff gets computed and if you want to do a final or like a project final project in the class. Um it would be something like that notebook but like a little more elaborate. So it gives you like a little bit of a flavor of what the starting point and uh what you know what the project would be like. Probably would be a little a little more extensive than than what we're talking about here, but but um like in the same same spirit. So yeah, that's so now that we got this over with like we're g let's talk about like what is PCA and where where does it fit within the world of um machine learning algorithms and it's it's basically um it's it's a un method for unsupervised learn learning. So it's it basically a method to cluster reduce dimensional dimensionality to make sense of the data. It it's not it doesn't it doesn't necessar it's not a regression so it doesn't like there's no causality implied when you use PCA but it is it's uh yeah it's a way to understand data better especially data that has a lot of dimensions where things are like many many of those dimensions are highly correlated here we have I've listed a bunch of examples of where PCA can be applied generally really it's it's widely applied in large data sets, multi-dimensional data, like a lot of data. Um it's mostly like the actual the actual calculations are mostly linear algebra. So it's efficient and fast and scalable. So and this is one of the reasons people generally like to to use it. I'm not going to go through all of these, but like it's it's very it's their applications are very widely they widely across across science and and in finance it's used like very extensively as well. Okay. So first thing I wanted to talk about is like kind of go through a visual example of um these are some of these things that I'm going to talk about are like my mental toy models to understand that I always refer to when I think about what is happening when I'm using PCA. In this example, uh do you guys see? Well, okay, good. In this example, I've prepared effectively three data sets, but they're really like one data set. Like it's one I've I've just made this up, right? It's just one it's like one dimensional data set. And all I've done is I've taken this onedimensional data set and um I've made it two dimensional by just adding an extra coordinate like the blue line the extra coordinate y I made zero and then I took this data set and I just like rotated it like 45 degrees and 90 degrees. And in the in this two dimensional world like it's it's two each point has two coordinates. So it looks like it's two dimensional, but I know it's like it's onedimensional because I I made it. And and the key thing that that PCA would do is like is PCA kind of would immediately tell you when it looks at this data that it's actually onedimensional. If just by like you download the data, you do the calculations, it's going to be onedimensional. So how how would it work? Um, PCA basically you can think of PCA as like a as a as a way to rotate the coordinate system. And we basically when when you apply PCA to this it will tell you if you apply to the blue blue data it will tell you that the PC1 the first principal component you guys have done uh Peter you guys have done the math part already right of PCA. Yeah. So the first principal component is going to be the x direction. When you do the the red line, it's going to tell you that it's like the the one one direction and and it's going to just discover this this um this dimensionality. And okay, the first data set was like very very simple, but there was no is purely onedimensional. This one here is like kind of the same thing, but I've added a little bit of noise in in the other dimension. And um now when I apply PCA to this, it's going to tell me basically the same thing. PC1 is going to be the same, but PC2 is is going to have a little bit of variance along PC2 along the perpendicular direction. So in both of these cases A and B, it will tell me that PC1 and PC2 vectors are the same. And it's what's going to be different is the there's going to be zero variance in case of B on the PC2 direction and a little bit of variance in in in the first in the in the example A along the PC2 direction. So it basically tells us the two thing it tells us like the directions along which the data is concentrated and the variance or the how important that direction is on a relative basis. We can generally what's nice about this like I've done this two-dimensional plots but PCA like when you actually do it it's just linear algebra and you can do it on you can do it on any number of dimensions you can do it on um yeah it so when you have high dimensionality this is a very robust and powerful tool what's what's really nice about that and the high dimensionality is like it's it's not a minimization, it's not an optimization, it's just a linear algebra calculation and it's fast. So you can you can process very large amounts of data very quickly. Like a very common like when you use this in practice like the answers that you get from like the things you kind of learn from PCA are going to be it tells you like a relationship between variables. Like in our first example, if you look at this diagonal line, it's going to tell you that like it's basically like y equals x is kind of the line, right? And then you can you can think, well, I can discover the same thing if I run a regression y regress y on x and and then it's going to tell me pretty much like the same if I if I look at my diagonal line, it's going to tell me the same thing. And that that is true in the one-dimensional example. But I wanted to kind of give an illustration of what is the difference between regression and and PCA. And the difference is like the PCA is is kind of um it's remember it's it's unsupervised learning. So there's no causality implied. So X and Y are kind of on the same footing. Each dimension is on the same footing. There's no special dimension that's dependent and one that is independent. So when you regress, this is like the same data set and you can see here the regression of Y versus X, the regression of X versus Y and the PC1 line, they're they're similar. They're not but they're not the same. it it like it's actually optimizing it's minimizing the distance to the line but like in a in a in a like this is this is like minimizing the distance in the perpendicular direction and the others are minimizing distance in the independent variable direction so when you do a regression so it's not exactly the same I think it's good to keep that in mind because sometimes if you really want to understand how you know like given an observation X how I should model I maybe PCA is not the right approach maybe you should run a regression but like if you want to understand the the correlation structure of X and Y of two variables there there is no you don't have necessarily one depends on the other you don't know anything about their relationship then then PCA is probably a more appropriate tool um yeah so I was just saying like if there's there's some data that is in some like if data is like in a particular line PC1 which is the red direction here would always discover where that direction is and if you actually map the the data into like PC coordinates it's just going to look like a straight line like that and a couple of things that I like about this example is like you can you can see how rotating this data set doesn't doesn't change the PC coordinate output so it look it's it finds a direction appropriately The second thing that's very important is robustness. In the real world, like when you do, it could be in science or in finance, you're not going to have perfect data. Everything is very noisy. Noise is a huge problem. And robustness to data noise is very important. So this kind of illustrates how robust things are. So if if I if I take one point and I just move it around my um you can see that my the red the red line which is the the direction of my PC1 doesn't it's like it's quite robust it's quite robust to to to noise in to one particular if everything else is aligned like just moving around one data is it doesn't really change things too much um if I make them a little bit more spread It's it's a little less robust, but it's still still pretty robust, right? And the thing that I wanted to kind of say like what happen people are just going to very often you just take some data and you just shove it into PCA and you don't actually care too much before you even know what the data is and you just put it into PCA and PCA always gives you something. But suppose the data doesn't really have a dominant direction then the PCA algorithm becomes not robust. So you can see now look I'm moving one data and look what happening to the coordinate one data point is I can just move that a little bit and like it's just completely giving me the output is like super sensitive to to any noise in the data. So it's kind of worthless. So it's very important when we're looking at the data if we don't really know what this data kind of looks like very sometimes it's high dimensional you can't even really visualize it you have to think how do I understand whether there is actual structure in the data that I'm trying to discover this will help me discover quantify the structure but it's also good to understand qualitatively how much the structure is um one thing that is that PCA also like part of the PCI analysis and the output. Yes. Do you have any like tool for like wiggling around the points to see how how robust it is to see if the data is actually alive or not? Well, um it's no but what it's not I don't have like you can write tools to do that but like I think what you want like one thing that you can do that is very important is to kind of when you do your PCI analysis is to look at the en values because people very often people like look at the vectors and like oh these are my directions and I'm going to project things and I'm going to assume everything is like factorized nicely and like we don't really look as much at the values but it's very important to look at the values and the reason is that in this example when there's no dominant direction the values which measure the variance along each direction would be very similar. So when you look at when you're look using a a principal component like the first principal component you want to make sure that your the value of that component you're using is is like quite distinct from the rest of the values in finance we'll see later like we're dealing with a time series of PCA like every day you construct the PCA model so everything anything that's a number here like a value is actually a time series of value so the thing I look at a when I'm doing analysis is I look at value plots in log space and I want to see that each distinct component that I use ideally you want to see it that it's like an order of magnitude different than the next one in in variance variance invariance basically which if that's true like that's great and that actually and that doesn't always happen but at least you want it to be like visually you want it to be distinct You want it to be like different. You don't want if you if you have a time series of things, you don't want like the first and the second one to cross. And if that's happening, then the first and second principal component if they're if the values are small changes are causing them to switch places, this basically means there isn't a lot of structure and the PC output is noisy and it's actually going to obiscate rather than clarify the thing you're looking at. I've put here like a little bit of math notation. You guys have um done your math lecture on PCA. So, we can be relatively quick here. But I do want to go through it. The the main thing is like you have um your we have your data data matrix is generally like there's n observations in pd dimensions and we think of that as a matrix and then uh if this matrix is x then uh everyday's observation are vectors of size p and then you have a coarian matrix which is xt * x and um um the and then like there is I I've here like defined the the W K or the orthonormals vectors representing the these are the vectors W K and then I've written down the math of how you do the projection of data onto principal components and um and how you compute the variance. How do you get these vectors? Well, there is you could you could do you could take your data compute coariance matrix feed it into an veg value calculator which is basically in in the in my work our data is not really the number of dimensions is not actually that high. So that that's pretty that's okay. it does kill as um like that thing is potentially maybe a little bit slow but it's not actually that slow. There is also a more efficient method called SVD um single singular value composition where you can just compute your projections without calculating the coariance matrix. I like to compute the coarian matrix because we generally need to compute the coarian matrix for other things in any case. So if you have it already then then it's then it's um you might as well use it. Some there's some I've listed here some properties of this vector matrix that are important that it's orthonormal which means that it's the transpose of the vector matrix is kind of also inverse. The coariance matrix is can be written as the in this in this form where lambda is the igen value matrix that's the svd de composition that there's total variance quantity you can think about like which is the variance of x which is just the trace of lambda so like the product of ien values is the variance of your data set and there is something called percent variance explained which is just defined as how much variance is is um along a particular PC direction divided by the total variance. So that adds up to 100%. Like when you're and um but that's very useful because remember how I was saying you want to look at the values to understand how important each direction is. Variance explained tells you that as a kind of a number you want to look at. You want to when you when you're trying to model your data set with as few dimensions as possible, you want to kind of add more PC directions until you get to a relatively high variance explain. If you're able to explain um if you have like a 10 dimensional data set and you're able to explain 80% of it using three dimensions, that's a pretty that's a pretty big win. And I've also like written out this out of sample variance explained or very often what we do is we we calculate a PCA we calibrated on historical data and then we apply it to future data that um we're trying to model like trying to model what will happen in the future and especially in finance we do that. So when when the data you're cal calibrating your your um PCA on is different than than the other data then this you hope that the structure of the two data sets are the same. So if if the structure is the same you'll get the same kind of variance explained structure but to the extent that is not true you you may not. So you may see that um your dominance of certain PC1 factor becomes like it starts to explain less variance as you as your um out of sample data becomes more different from the the data which you calibrate. So this is a little bit of like the modeling art of it where you're going to be using some data to to calibrate a model and then going to you're going to use this to to um predict the variance the correlation structure of what hap what happens in the future like future market moves or future returns. Okay. So um a typical how do we like what do we generally do when we apply PCA? We we're going to take our data set we're going to demean it and then maybe we're going to normalize the variance to one. Like when I say maybe and the reason this maybe is like sometimes you want to do that and sometimes you don't. It's um very often whether you want to do that or not depends on how exactly you're going to use your uh PC factors and uh your your dimension your dimensions. Generally I would say that most of the time you you want to normalize it in in a in a certain you you want to normalize in a way that you want the expected at least the expected variance along different along your dimensions to be unit to be similar. Then you're going to compute your coariance matrix and calculate the igen values and then you're going to order them by decreasing value. Um and um and you're going to have like a certain cut off where you say okay the only the top I'm going to use only the top X factors to model the the structure of my data set. >> Yeah. >> So you say like maybe you would normalize it. What would be like a concrete example of like where it would be better to not normalize it? >> Um well here's an example. So let let actually let yes I'll start with this. So this is an example of like it's a little bit of a toy example and I'll talk about a real example. So in this toy example there's the blue and the red data set and uh the red one looks that it's kind of one-dimensional and the blue looks like it's just a blob. There's no direct dimensionality. So when if I tell you okay should we use PCA is there what is PCA going to work on the blue data set is going is it going to reduce dimensionality and the answer will be no no because it's there is no dominant direction and in the red one it definitely will it will pick the right direction now what's fun about these two examples is that these two are the same data except I've normal in the blue data set I've normalized the variance along X and y and in the red one I haven't. So if if I if these two dimensions were you know like I don't know temperature versus um temperature in Seattle versus temp in Atlanta or something. They're like the kind of very distinct and generally like different things where you're only interested in correlation and the the relative variance doesn't quite is not very meaningful then you you would want to normalize it because you're you're that otherwise the units are not going to be you want to have like somewhat related units. So the units will be standard deviations if you normalize it. If you don't the units will be arbitrary and they don't it wouldn't really make sense. On the other hand, if um if this was like if x- axis was your grades on quiz one and y axis was your grades on quiz two, these two are like the same units now. And you you may not want to normalize. You you want to actually like these grades are these numbers are meaningful relative to each other. So there's no necessarily like you want to actually model the actual grades including the the variance of quiz one and quiz two. So there is no necessarily need to normalize it. >> That makes sense. Thank you. >> Yeah. >> But it's like that part is to be honest that part is very much the art of things. We've had a we have a lot of discussion work. Are we normalizing it? How are we normalizing it? What are we using to normalize it? Like that's like you'd be surp surprisingly this is sometimes cutting edge research is to actually understand how you normalize it. It's that these are the things that actually make a difference. The other thing I mentioned is we need to demean the data set. Why is that important? Because remember the principle of component analysis is a rotation. So it it always rotates around zero zero. So what zero is it matters a lot. And this is two examples of the same data set. These two blocks are the same data set, but one is like not the mean and the other is the means. So this one here is centered at zero. And PCA finds that there's a direction dominant direction. It finds it. It works like it's supposed to. This one here is not centered at zero. Like for example, let's say you do a bunch of stock prices. Like you just put prices into PCA. Stock prices are not centered at zero. They're always positive. So, it's going to have some direction, but it's going to be somewhere far away from zero. When you run PCA on it, it's going to be like it's going to try to rotate around zero, but it's not it's like it can't find anything because it's not you're not like you can see why like you can't rotate this thing. You can't there's nothing to find if you don't center it at zero. You cannot rotate around it's rotating always around the origin. So me mean that near zero or close to zero or exactly zero is is very important and surprising like it's surprising how often comes up somebody just does PCA on some on some numbers that are just like large positive numbers and like nothing comes out and like ah this doesn't work but you just have to normalize to zero. How do we actually do it? Well um this is just the examples of various places that have implementations of this value calculations. But I think when I started doing this um like 11 what I don't know how many years ago this actually was more relevant. Now it's like everybody uses Python, everybody knows Jupyter notebooks and like you guys um now now it's like standard toolkit. Okay. Now what's special about finance when it comes to BCA? It's mostly what is special about finance. This is not exclusive by the way, but like most of the time we are going to be doing PCA on time series in in quantitative finance like in in the quant world that I live in and I I work in. I would say it's our the general job is to discover patterns in the that are that have happened in the past and try to use that information to model the future and to somehow like it use that as part of your toolkit to make predictions and to place bets in the markets. So every day, every every trading session, every every observation is an opportunity to bet and to make predictions. And you can understand like the thing about time series is that the history now and the history on the next opportunity to bet is not that different. Like it has one extra data point maybe like later has an additional data point, but everything else is the same. So there is like there's a slow changing time series of information and what how we handle that there is a whole time series data science there class that one can take but in especially with regard to PCA it's important to um to understand like how what does what what does impact does that time series aspect of it have on PCA? A couple of things I think are are important. We we generally are going to be interested in in modeling the the changes of var of things of variables like not the levels. Part of that is that the thing I'm I was just talking about which is demeaning. And part of it is that when you make bets in the market, you make money based on how things change after your bet, not the price, the absolute level of where the price is at. So like returns are what matters, not the price of something, the absolute price. Um then the next thing is like okay so when we do when we model when we have our data uh in finance like generally the data is the historical returns or histo history history of something history of prices history of returns history of anything of of whatever you're modeling. It doesn't have to be just prices could be could be any anything that any any technical indicator and that means that every day there is a there is a history that is similar but a little different. So the history changes your your information about what has happened changes slowly and that means that when you're looking at your PCA you have to now define how am I going to define my data matrix which I'm using to calculate my vectors and values and the one of the main choices are how long of a history I want to use and how much how do I weigh How do I weight history? More recent history versus further away history. And this is these are some of the main hyperparameters of principal component analysis in finance. How many things I'm I'm including in my obser in my data data set. How far back am I looking? How am I waiting that those observations as I pass through time? And then the last thing is like the robustness of the data. Very often there is a trade-off between including more data points but each each marginal addition of data like each marginal dimension of data that I'm looking at could be less robust. Like there's like and we'll talk about in the examples later on, but there's um we have to think carefully about how how robust the data we're putting in is because if it's if it's too noisy, if it's not representative of where our bets are going to be, then it becomes less relevant to our modeling down the line. On the other hand, if we don't include enough, we're also going to be missing opportunities to to um understand the structure, the information. So again, that's part of the art of of um of actually of the practitioner is to actually kind of have some feeling of have some intuition about what I should include and what I should not include. And we'll we'll talk about an example here. The example first example and the main example I want to go through is um applying principal component analysis to the United States bond market. Yes. I feel like you mentioned that PCA in this case time series like historical historical but is there you kind of play with the like granularity of the time series as well like the different um I guess window sizes I would say of like how how granular the date is going to be when it's like >> Yeah. Yeah. That that's another that's a whole other dimension of how granular it is. Absolutely right. It's not just about how long of a history, but how granular it is. Like what like the examples I'm going to show you right now are based on daily observations. But why why did I decide to use daily observations? The short answer is like and we by the way very often in in my work we use daily observations. The reason is that daily observations are just a more available generally more histories is high quality data history is available for longer periods of time for daily observations. Second yes intraday observations are they represent additional data. However, the quality of the quality of that data, meaning how how much noise there is, can very often in practice be much higher for interday data than is for end of day data. That might not always be true. There are situations where that's not true, right? But and there's there's entire fields in finance that deal with high frequency trading. In those fields, end of day data is just like they their whole trading buying, holding and selling happens 10 times for the entire day. So they start and end the day with like no positions. And for them daily is like not relevant. It's too too slow. They need to understand how the market moves on a very fast time scale. So that how how often like how you sample time is a whole is a whole other story. And basically how we sample time is going to impact the units of volatility because volatility is measured in in units of square of time basically. So it's it's very important to like to know that and have to to to understand to keep to keep that in mind that whatever the frequency of sampling is that's going to give you effectively the unit of your volatility and then if you want to annualize it like you need to apply appropriate factors and comparing volatilities of different units is not going to be meaningful. So that that's important. Okay, going back to this example, the um US bond market. So why why am I like why is this an interesting example? Like I work in fixed income which is basically where we trade of bonds and various derivatives of bonds and those are the the biggest markets in the world like the most there's huge amounts of money being transferred. They're not necessarily the most volatile. Equities are much more volatile, but they are extremely deep, extremely liquid, and they also allow like people who trade in them to take large amounts of leverage. So even though they're not by themselves that volatile, if once you take leverage, you can increase volatility by taking leverage. So you can actually make um you can construct portfolios that are that are actually quite quite volatile and deliver substantial um substantial like market volatility and and returns. So there's a lot of there's a lot of huge industry of money managers, hedge funds that trade fixed income and um the US bond market is the kind of prototypical example of fixed income. It's the most liquid in the world by far, the most well structured. So, it's a great example to to play with. Another reason I'm using it is because there is pricing data from the Federal Reserve that is freely available and that's the data that we're going to use right now to do our analysis. this data is like it's it's actually quite high quality data and um yeah that like that's I think that um if you want to do a project like this will be a great great data set to play with because it's real and um it's it's goes on for a very long time and and you can apply you can apply our PCA techniques to it. So the first question to ask is like is there remember like we have our data set now what does it consist of? Well, in the United States, the US Treasury issues it just it issues a bunch of T bills, short maturity bills that they raise money. That's that's how they raise money for the debt. But, uh, the part we're going to concern with is going to be the US Treasury notes and bonds, which are kind of the longerdated instruments that US Treasury issues. They issue on a very regular basis. Every month there is a schedule of auctions that is very well known and repetitive and they always issue like a two-year bond, a three-year bond, fiveyear bond, seven-year bond, 10ear bond, and a 30-year bond. And there's actually now a 20-year bond that started about like three, four years ago, but we're not going to look at that right now because it's it's fairly it's very new and it's not not as liquid. So, we're just going to focus on these. And what are like the other thing about these bonds is that they're they're all debts of the US Treasury. So, they're they're very in a sense they're very similar. Like if you if you if I give you money for six months and I also give you a separate loan for like three years, I've I've given you two loans. The first thing that these two things are they're basically you still owe me money, right? Yeah. one is longer in maturity than the other, but like you're still wing money. So, in terms of the way the price of these of these securities evolves, they're all kind of they all depend on the interest rates, the short-term the path of short-term interest rates of the Federal Reserve and the expectations of that path. So, that it's they're all derivatives of that. The implications of that is that the returns or the um the changes of bond yields are very highly correlated. So here on the bottom right I've shown the um the the correlation matrix of changes and you can see this is like these numbers are very very high. So this is a data set where everything is very very highly correlated prime candidate for PCA. Whenever you see a correlation matrix that has high concentrations of high high correlations prime candidate of PCA. So what are these what are these numbers I'm looking at on the on the left chart I've shown the US yield curve. So on any given day we can observe the prices of two-year bonds, three-year bonds, five year bond, sevenyear bond, 10 year bond, 30-year bond. What is the price of that? And I don't know if uh if Peter have you have you um guys done bond math yet? >> A little. So so you we know the prices where these things buy. So they're very transparent prices and we can convert prices to yields. So basically a yield is like the implied rate of return of the bonds effectively given its price over the course of its maturity. And then when we plot these yields on on this plot here I've plotted the yields the yield is on the y- axis versus the maturity of the bond. And I've sampled five days from history. And this is what's known as the yield curve. that is a that is the most important input of macroeconomics in the world is the US the shape of the dollar yield curve. Um sometimes most of the time the yield curve is upward sloping so people people require a bigger return to lend money for longer and the US treasury like generally requires that. But sometimes that is not true. Like sometimes you can see this top blue line. Sometimes the yields actually like is is starts out downward sloping and then flattens out. So these yields can have a variety of shapes and this is just the level of yields. What we're betting on is the changes. So we're betting on how this curve evolves dayto day. And you can see from these from these um yield curves that you can imagine how if I just if I was to show you a movie of the yield curve, it's going to kind of flop around. It's going to be like a little string. It's like flop around, but it's going to be like going up and down. It's going to slope like this. It's going to have a whole set of dynamics and that that dynamics and that kind of those dynamics is what we're trying to kind of model with PCA here. Um bonus question is do you guys know what is the shape of the yield curve now? What does it look like? What are what are the is are short-term interest rates like higher than long-term interest rates? Have you has has anybody like we had a a very prominent Federal Reserve meeting recently and uh they they do do you remember what they did the action the Fed took was the action the Fed took was to cut interest rates by 50 basis points. So what the Fed is doing it's cutting the interest rate that is the leftmost point on this yield curve that that they're they're setting that point. Everything else, the rest of the curve is market expectations of what that point would do in the future. So they moved this point down by 50 basis points. 50 basis points means half a percent. So it went it went down by half a percent. Right now this blue curve, even though it's from 2023, that's like that's pretty recent, it's still the curve is still downward sloping. It looks a little bit more like like this right now. So it's not it it was for a very long time it was actually like really inverted. It was going like downward sloping like that. Now the back now the back is like starting to flatten out but it's still quite high in the front end and downward sloping. So that that unrelated to what we're talking about but fun kind of a fun fact. Okay. So now when we look at yield changes um when we look at the we take our data we compute your changes so the changes of yields on dayto day we we can we should demean that data set but generally demeaning it doesn't really help that much because the mean is going to be very close to zero because changes are sometimes the are going to go up sometimes they're going to go down you're not going to have over the history of of 20 30 years that we're talking about here, you're not going to have a large large mean. Still, it's always good practice to demean it. Won't change much the result. But then we take this in this example, I didn't actually normalize the variance. I think we just just left it in there. So we did PCA on coariance matrix. And the reason is that the units the yield changes are they're comparable units and there is um looking at the PCA without without normalizing the the variance is actually like it's a very meaningful output like it's it's we want to understand how volatile is going to tell us something about the relative volatilities of points. So when we do that and we look at the um what what I what I'm plotting here on the top are the the igen vectors. So the the principal component factors the shapes of the principal component factors on variance days. So this plot can be confusing right and I want to make sure that we're all on the same page what it is. This was this is the shape. This is the yields. Then we take these yields. We look at the changes of yields. Then we put them through PCA. And remember the output of PCA is IGN vectors and values. IN vectors means every every point like two year, three year, 5 year, 10 year, 30 year everything has a number. Now a loading we call that the number the vector coefficient. I call that loading. That's a very a term that we use. that loading is again there's a time series of those and I'm plotting here a select like five different dates in history of what that vector looks like what the first principal component looks like what the second principal component looks like what the third principal component looks like these are f these are like snapshots like five snapshots in time and I'm plotting maturity versus loading and there like there's actually three dimensions here. There's maturity, loading and time. So I can't like I can show you a threedimensional plot but that will just be like confusing. So I'm I'm showing here cross-sections. So these are these are various cross-sections on time. And you can see that there is a clear and persistent pattern in what the en vectors look like. The first one PC1 is called the level. That's basically means that the that that is like it all the loadings are positive. So it means when the yield curves move generally they move up and down together when in the same direction. So if 10 year rate is going up probably two-year rate is going up as well. Five year rate is going up as well. And hence the term rates are going up. rates are going up is like generally speaking everything is going up or down and that's the most that is the the move that that is explains most of the variance of the yield curve but on top of that now there's a second move which is we call this PC2 and we when we look at the shape of it we can say oh it starts out negative and the front end of the curve and it's positive on the back end of the curve that's the slope so that's kind of a steepening of the curve versus flattening the curve that is the second PC factor. So this first PC factor by the way explains around like 90% of the variance maybe like 85 or something and the second PC factor explains like between five and 10% of the variance like very huge difference and the third PC factor is like the curvature. So how how curvy what is the curvature of the of the like it's it's positive on the front end positive on the at the 30-year and negative at the belly of the curve 10 year point and that has the smallest amount of variance. So what does that mean? If you think mentally of the movie of what the yield curve is doing it's mostly going up and down mostly going up and down like that but it's also like it's also slope a little bit of slope. So it's mostly up and down but like a little bit of slope and a little bit of like curvature like that. So like that that's that's that's the that's what this thing tells us is the dynamics of the curve. Um now some of these things are like super the structure of this thing actually tells you a lot. You can see how the the PC1 level it like the the u the maturity at which it like it starts out steep upward sloping and then it levels out. The maturity at the maturity in years at which it levels out. I've plotted here what the maturity is at which it it kind of levels out where the short-term rate decoupled from long-term rates. And if you plot that in time in time series, you can kind of see that um it was very short like monetary policy was was basically only governing the short end of the curve up to two years up until right after the postfinancial crisis. And then the Fed became very very powerful. Post financial crisis the Fed said we're going to keep rates low for a very long time and all of a sudden this thing exploded. it became very very very high. So like basically um the short end rates were governing effectively the the the um rates up to up to four or five years and then up until co they started to decrease up until co happened. Fed came in and said we're going to bring the bazooka so we're going to everybody's like okay they're going to keep rates low for a long time as well. It exploded again and now it's back down again. So that is like that's that's already something that is kind of like a regime indicator. You can see how the market behavior is reflected in the shape of these PC factors. That tells you so much about what's going on in the market when you look at the these in the details of these factors. Something like where is the bottom of PC3 where is the belly you can see that sometimes is at at 10 years sometimes it becomes close to five sometimes close to 15. That's also a regime indicator PC loading stability. So here I'm taking this this graph PC1 graph but instead of plotting loading versus maturity I'm plotting loading versus time and each maturity point is line. So that tells me the stability of my loadings. you generally you want to look at that because you want you want factors that are stable and PC1 loadings are are pretty stable but these and the fact where the changes in PC1 loadings they're very often like they're quite meaningful like you can 2008 crisis two-year rate this blue line is a two-year rate the short end the short end becomes very volatile nobody knows what the Fed is going to do there's a lot of volatility and then collapses post 2012 like the Fed is not doing anything. So the to your point is not moving. Then it goes up. Maybe the Fed is going to move. And then after CO it also collapses and now it's back up again. The Fed is very active right now. This this ends in like 2023 this data set. If I make it a little bit longer, you see this line actually went very high in the recent history. So all of that is um these things are like they're they're they all come from this data but they're they tell you they they they tell you a lot about the behavior the regime and the uh dynamics of of interest rate market. This is the variance explained. I think this is like this is a way remember how I was telling you how we look at values and we want to make sure that they're well segregated. Well, this is a good this is a way to look at it. If you look at this chart and the first factor explains so much variance, it's like yeah, that that's that's a good factor. That's a good factor to that like that's a solid factor and you can um you know that this thing is persistent. You're very this thing is likely to continue is likely to be stable and it's a good part of your model for the future. If you look at variance explained by factor, this is a log plot. You'll see that how the first and second and third factor are separated in like by order of magnitude. So they never cross. So that means that these factors are robust and they they are like yeah they're they're very useful modeling factors for modeling. You want to see this kind of picture. What you don't want to see is like lines that are all entangled up with each other. That's you don't want to see that. You want to see something like this in general when you're looking at PCA. Again this is these are values I'm looking at variance explained um and this out of sample coariance is like when I'm this is basically when I when I calibrate my PCA remember my my in my if I take a single data set however I choose it and I do run my PCA the PC1 projections and the PC2 projections are going to be uncorrelated with each other by construction. But if I take the same loadings now and I look at the future returns, there is nothing that says that um if I project my future data, future returns onto the same loadings, those might not might actually become correlated. So here I'm computing like kind of future returns projected onto PC factors using that are calibrated on like past past history and and you see that it's not they're not fully orthogonal. However, they're quite orthogonal. So it means that like the dynamics are mostly preserved. They're the loading the PC factors are mostly stable. This is not like this is something you would expect you would hope to see. You don't want to see you don't want to see these numbers off diagonal numbers to become like 30 50% that will be like that means that your PCI model is quite unstable but if it's like if if your our sample correlation is is pretty small that's that's a good thing and now what do we do with this PCA well one of the key things is like okay I want to go back to this slide for one reason suppose you say, well, I really I want to bet on curve. I want to bet on the second factor or I have a model that on that I want to bet on like curve stippling or relative value of 10ear bonds versus fiveyear bonds or 30-year bonds versus 10ear bonds. I can use PCA to isolate effects that are idiosyncratic to particular points on the curve to particular factors. And I can basically say well if I if I hedge out the vast majority of the variance by hedging out PC1 and I look at like the residual PC1 residuals which is PC2 and above. What can I say about those uh about those returns? How can I model those and how do I and the key thing you can you can construct these things. You can say, "Oh, I want to model these things." But the key is how do if once you once you have a good model, how do you trade them? So being able to construct P to understand how PC loadings translate to portfolios is very important because it it tells you how I can use this methodology to construct specific portfolios that are uncorrelated to each other and um and give me like a new a new a new kind of target to model. If you if I didn't do that, I have I still have like eight things I can trade or six things. Two year, three year, five year, 10 year, and 30-year bonds, but they're so correlated where I'm trading 10-year bond or 30-year bond. Like, if I just trade 10-year bond outright or 30-year bond outright, I'm mostly getting the same return. So to be able to increase the diversity of what I'm of the target I'm capturing, I'm using PCA to construct more portfolios that are going to be uncorrelated and present me with more opportunities to model to make bets. That's like that's a key use case for PCA finance. Like we take these things that are very highly correlated and now I'm going to create out of these six highly correlated portfolios, I'm going to create three or four portfolios that are not highly correlated. What are like and these portfolios are going to be my PCA are going to be defined by my PCA factors. So remember the factors on any given day they give me a loading on every tenor on every maturity. Yeah. Did you have a question? Okay. Yeah. They gave me a loading on every maturity. So it tells me and I I can think of this loading as telling me how much to own of this particular bond on on that maturity. So PC1 means buy every bond in some in some equal amount. That would be that's PC1. PC2 is like oh you should like sell some two-year bonds, buy some 30-year bonds. Like basically the portfolio is going to be the portfolio is going to be defined like you're going to you're going to basically buy bonds according to the loading on every on every point on PC2. Why is this portfolio interesting? Well, it's interesting because PCA tells us that if I construct this portfolio like this, it would have no sensitivity to PC1. But PC1 is like 90% of my variance. So that means that by if I construct the PC2 portfolio, I have effectively hedged out 90% of the variance of individual bonds and I've constructed something that's new, less volatile, but uncorrelated to my original like outright tenure bond. Same thing with PC3. I can construct and I can construct like these PC3 portfolios that are like and these by the way these things actually trade in the market. They're they're they're called like curves PC2 things are like they're curves like you you buy two year you sell 10 year vice versa and then and there's flies which are like 10 and 30s where you sell 10 year and you buy two year and 30 year. Yes. Is there any application of the higher