Part 1: The Column Space of a Matrix

Key Takeaways

Okay, here's the well, the title slide. I since this year happened to be 2020 and that means clear vision. I thought I'd get that into the title of this these slides. And then you've seen in these six pieces as as a sort of look ahead. And I'm going to start on that first piece. A equals CR. That's the new way I like to start teaching linear algebra. And I'll tell you why. Okay. Oh, here we have a few examples. Well, that will lead to to our ideas. You see that matrix A zero. A matrix is just a square or rectangle of numbers. And but those are those numbers have special features. If you look closely, well, you see 132 as row one. And then what do you see for row three? 264 and there's like those are two Those are two vectors in the same direction. Why is that? Because 264 is exactly two times 132. And in the middle there four times 132. So, I have three rows in the same direction. And actually, also this is the magic. Can I tell you this right at the start? The columns. Look at the columns. 142. If I multiply that by three, I get 3126. If I multiply it by two, I get 284. So, sometimes somehow magically the columns are in the same direction exactly when the rows are in the same direction. They're different. That's what linear algebra is about, the relations between columns and rows. Okay, and here well, here's another one I'll look at. Uh there again, you see row one plus row two equal row three. So, there it's not quite like this where every row was a was in the same direction, but here's if I add rows one and two, I get row three. So, that's a matrix of rank two, we'll say. You'll see it. Okay. Then here here S is for symmetric matrices. Tho- Those are the kings of linear algebra, and here are a few small samples. And the queens of linear algebra are these matrices I call Q. Uh those are called orthogonal matrices. Orthogonal meaning perpendicular. So, uh there's and they tend to uh express a rotation. So, that's a rotation matrix, an orthogonal matrix that rotates the plane. And there's a pretty general matrix that we'll see at the very end. Okay, so I'm into uh the start of the column space. That's a word I don't use in the videos for quite a while. But here you see I'm using it in the first minutes. So, I look at a matrix. Well, first, let's just remember how to multiply a matrix by a vector. Okay. There's a matrix A. There's a vector X with three components. And the way I like to multiply them is to take the columns of A. That's what I'm focusing on, columns of A. There they are, one, two, and three. I multiply them by those three numbers, X1, X2, X3, and I add. And that's called a linear combination. Linear because nothing is squared or cubed or anything. And combination because I'm putting them together, adding them together. Okay. So, that's the idea. And now the big idea is in that top line. I want to think of all combinations. So, this is one particular combination with a particular X1 and X2 and X3. But now I let every I think of every X1 and X2 and X3. All the vectors that I could get. Well, of course I could get the first column by taking one and zero and zero. That would give me the first column. But it's really mixtures of the columns that this produces and it fills out it fills out in this case a whole plane in three dimensions. These vectors have three components. We're in three dimensions. And can you just imagine in your head a two lines meeting at 0 0 0. So, they they cross. But I just have two lines. And now I take I I fill in between those lines. Now, filling in between those two lines is taking the linear combinations. That's where they are. And the result is I get a plane. I do not get the whole space cuz nothing is going in a third direction for this matrix. All right. So, let's see more about this. So, that's that word column space. And I use the capital C for that. And it's all the vectors I can get that way. All the combinations of the columns. And now I ask Oh, only I just answered this question. Sorry. Uh I ask is the column space, all the combinations, is it the whole 3D space which everybody calls R3 for real three or is it a plane or is it just a line? Well, the answer is plane. That probably even gives us the answer. That's the good thing about this subject. The answer is a plane because I have two different lines that meet at the at zero. And when I fill in between them, I have a flat plane. I don't go in the third direction. Good. So, that's the column space for this matrix. And I And And And And here's a little more of saying about that. We kept column one and we kept column two because you remember those two columns, the first two, were different. They went in different directions. They go in different directions. We did not keep the third column cuz it was just a sum of the first two. It's on the plane, nothing new. So, it So, the real meat of the matrix A is in the column matrix C that has just the two columns. And what about R? Cuz this is my plan for the first few weeks, first two to three weeks of linear algebra, is is to understand So, that 553 would be called a dependent vector cuz it depends on the first two. Those were independent. So, those are the two that I keep in the matrix C. And then that matrix R, oh. Well, I Now I'm multiplying two matrices and you know how to do that, but I always have a another way to look at it. So, the way I look at it is by linear combinations. You remember those? So, the multiplying is a combination of these guys. First, I have one of the first column. That's my first column. The next time I have one of the second column. That's my second vector. And the third one is this guy, one of that and one of that. So, these two are the independent ones and that's dependent. And a full set of independent ones is called a basis. Really fundamental. So, I guess I think that linear algebra should just start with these key ideas. Just go with them. And we learn something that almost falls in our laps. It's a great first great and not obvious fact about linear algebra. It is I'm just amazed to have it here. Uh the number of independent columns in A which was two is equal to the number of independent rows in R, also two. You remember that we had two rows and two columns. So, two two columns first in C, two rows in R. And the point is that that's telling us uh and we just checked that those two rows were two columns were independent, the two rows were independent, they're a basis. And then we learned that the column space has dimension two, R equals two for this example. And the row space has the same dimension. So, the rank, the column rank R equals the row rank R. It's like you know, if you had a 50 by 80 matrix. Okay, that's 4,000 numbers. You couldn't see what those these dimensions are, but linear algebra is telling you the dimension of the row space and the column space, 50 of one and 80 in another, are equal. Okay. Uh so, this is again coming early and we'll see it again, but but it's good to start linear algebra from day one. And then, here's another great fact about equations cuz matrices lead to a lead to equations where x is the unknown and this equation has zero on the right-hand sides. So, you could How How could we get zero on the right-hand side? We could take one of that and let me change that to a minus sign and that to a minus sign. One of those minus one of those minus one of those would be zero zero zero. So, that one and minus one and minus one would tell us an x. And that's the solution where Yeah, in applying linear algebra in engineering and physics and in economics and business, you end up with equations. Things balance. And uh you want to know how many solutions there are. And linear algebra is created to answer that question. Okay. So, now I'm just going to say a little more about this starting method of the course. Oh, I want to focus here on these interesting method interesting matrices where every column is a multiple of the first column. Every row is a multiple of the first row. The instead of having two independent columns and rows, these matrices have only one. So, then C has one column and R has one row. And the rank is one. These are the building blocks of linear algebra. The these rank one matrices, column times row. Uh the previous matrix would would have one of those blocks and a second block. A big matrix from data science would have hundreds of blocks, but the great theorem in linear in linear algebra is to break that big matrix into these simple pieces. So, that's the goal for the end of the course. Okay, and finally, a last thought about these. So, this is C * R. I'm urging teachers to present that part at the early. So, one of the good things I've marked with a plus. First of all, the the columns we're looking at them in C. In and we see them from A. We take them directly from A. R turns out to be a famous matrix. Row reduced echelon form, it's called. So, to see that pop up here is terrific. And then this wonderful fact that row rank equal column rank is uh clear from this C * R. So, those are all terrifically good things. Uh the other thing I have to say is that C and R are not great for uh avoiding round off or being good in large computations. This is a uh a first factorization, but not the best one for computing big for big computing. Right. So, that's ill-conditioned means could they're difficult to deal with. And also, uh we often have a matrix with all the columns are independent, and it's a square matrix. All the columns are independent. We can solve Ax = b all the time. But then then if all the columns are independent, then our matrix C is just the same as A. We didn't get anywhere. And R would be the identity matrix, like a one, because A equals C. Uh so, um so, this is the starting point. Picking out the independent columns, but not the end, of course. And uh I'll stop here and and uh pick up on the next factorization uh right away. Thanks.