Part 2: The Big Picture of Linear Algebra

Key Takeaways

okay in this second part I'm gonna start with linear equations a times X equal B and you see actually the first really the good starting point is a times x equals zero so are there any any solutions to the matrix any combinations of the columns that give 0 any solutions to a times x equals 0 now I'm multiplying the matrix a by a vector X in a in a in the way you'll know I take rows of x times it's called a dot product row of X row sorry rows of a times X so I have a row of numbers and X is a column of numbers I multiply those numbers and add to get dot product and I'm wondering can I get 0 for each is every is every row so having a 0 there is telling me gee in geometry that that Row is perpendicular orthogonal to that column if a row dot product with a column gives me a 0 then then in n-dimensional space that row is perpendicular 90-degree angle to that that column X so I'm looking to see is there are there any vectors X that are perpendicular to all the rows that's what ax equals 0 is asking for oh and that's what I've just said right there I've used the word orthogonal that's more of a high-level word than perpendicular so I'll stay with that sound a little cooler okay and now we can also look at that transpose oh do you know what the transpose of a matrix is I take those rows and flip the matrix so that those rows become the columns and the columns of a become the rows of a transpose so I'll look at a transpose x I'll call it Y for the new problem a transpose Y is all zeros and then the null space will be any vector any solutions any Y that's perpendicular to the rows of a transpose so I would need hours couple of hours of teaching to develop this properly because we're talking here about the fundamental theorem of linear algebra which tells me that the vectors in the null space like that our perpendicular to the vectors these guys are that's the row space oh but maybe I have told you I've said that from this equation that tells you the geometry that the row vectors are perpendicular to the X vector the thing in the null space so X is there the rows are there and they're perpendicular now if I transpose the matrix doesn't remember that means exchanging rows and columns so I have a new matrix new size even it will the same but it's a matrix the same will be true for it the the the rows become the columns and the solutions to the new equation with a transpose go in that space so so that then that little perpendicular sign is reminding us of the geometry so rows perpendicular to the X's columns perpendicular to the Y's that's the best I finally saw the right way to say that so I have two pairs and I know how big those each of those four things are those are the four fundamental subspaces to null spaces to two solution spaces with zero null means 0 so the these X's are in the null space because of that 0 those are the ends and then this is the column space and the row space so we got four spaces altogether two pairs and now you get to see the big picture of linear algebra we the four fundamental subspaces do their thing there you go you can die happy now it says the row space is there those are rows of the matrix independent rows of the matrix that's why I don't put in all the rows there are M rows but I only put in independent ones so that might be a smaller number our our the rank and here are the solutions the guys perpendicularly this is the rows of the matrix these are the vectors perpendicular to it these are the columns of the matrix these are the vectors perpendicular to the columns you see it's just a natural splitting of the of the whole spaces of vectors into two pieces and two pieces and I think of the matrix a when it multiplies stuff there it gives stuff here when a multiplies a vector X you get something you get a combination of the columns that was the very very first slide a times X is a combination of the columns and then we look at some X's if there are any that where a times X gives 0 and those there is 0 right there ok ok so that's the big picture and I'll just point to another little little point that's hiding in this picture you see that little symbol there that little thing and it's also here what that means is that those guys are perpendicular to these and these are perpendicular to these so we have four subspaces two pairs two perpendicular pairs and that's when you get the idea of knowing what they mean knowing how to find them at least for a small matrix you've got the heart of linear algebra part one this is the first half of linear algebra okay I'll just see what else there is oh here oh well this is this is a another comment I have hardly told you how to multiply two matrices the usual way is rose times columns but linear algebra being always interesting there's another way that I happen to like columns times rows now there is a column times a row now column times a row we've seen that once for for that rank one matrix do you remember I said that those rank one matrix one column times one row are the building blocks well here is the building those are n of those blocks a column times a row or column times a row and here is a reminder of oh we've only oh we're coming up to a güell Lu the first one get on with it professor Strang okay okay now we're solving equations now we're gonna get L times U so right so there's two equations in two unknowns solved in high school and how do you remember how that's the whole point if I take twice that equation so it's 4x plus 6y equals 14 and subtract from this one then I get an easy equation for only Y by itself so that's what I did that's called elimination I eliminated this 4x it's gone it's 2 times that right that's why I chose to multiply it by 2 then 1/2 x times this gives me 4 X's when I subtract it it's gone and I'm left with 1 y equal 1 so I know the answer y equal 1 and then I go backwards 2 X equal 2 because 2 X plus this is now 3 equals 7 2 X is 4 X is 2 and the real point about linear algebra done right is that all those steps can be expressed as a breakup another way to break up the matrix a into a lower triangular matrix you see that that matrix is triangular it's lower triangular and this one is upper triangular so those are called L&U yep yep so what we did here is expressed by that matrix multiplication you really want to express everything in the end as multiplying a couple of matrices then you know exactly where you are so that's the that's the idea of elimination and and now we only were doing a two by two matrix you remember our little matrix was pathetic two three four seven that was our matrix a we can't stop there so linear algebra it goes on to matrix of any size and this is the way to find the triangular factor L and the upper triangular factor you that would need more time so all I want to say is when you're doing elimination solving equations then in the back of your mind or in the back page you are producing an L matrix lower and a u matrix upper so that so yeah let me see yeah here we see them the Lommel matrix is all zeros above the U matrix is all zeros below and that's what is really happening so that's what that's what computer system totally focuses on okay that's the first slide of a new part so I'll stop here and coming back to orthogonal vectors good