Rank and Null Space

The Column Space

When you multiply a matrix $A$ by all possible input vectors, which outputs can you produce? The answer is the column space of $A$: the set of all vectors that can be written as $A\vec{x}$ for some $\vec{x}$.

Since $A\vec{x}$ is a linear combination of $A$'s columns (weighted by the entries of $\vec{x}$), the column space is exactly the span of the columns. If the columns are independent, the column space is as large as possible. If they are dependent, it shrinks.

The dimension of the column space is called the rank of the matrix. A 2×2 matrix has rank 2 if its columns are independent (spanning the plane), rank 1 if they are parallel (spanning a line), and rank 0 only for the zero matrix.

The Null Space

The null space (or kernel) of a matrix is the set of all vectors that the matrix sends to zero:

Think of it as everything the transformation "flattens" or "destroys." If the null space contains only the zero vector, no information is lost—every input produces a distinct output. If the null space is larger, multiple inputs collapse to the same output.

The dimension of the null space is called the nullity. A matrix with a non-trivial null space (nullity greater than 0) is not injective: different inputs can produce the same output.

The Rank-Nullity Theorem

Here is a fundamental relationship: for any m×n matrix,

The number of columns splits into two parts: the dimensions you "keep" (rank) plus the dimensions you "lose" (nullity). This is the Rank-Nullity Theorem.

If you have a 2×2 matrix, rank + nullity = 2. Full rank means nullity = 0. Rank 1 means nullity = 1. The theorem tells you that you cannot increase the column space without shrinking the null space, and vice versa.

Column Space in 3D

In three dimensions, the column space becomes even more visually striking. Two independent 3D vectors span a plane through the origin. Three independent 3D vectors span all of 3D space.

When solving $A\vec{x} = \vec{b}$, you are asking: does $\vec{b}$ lie in the column space? If yes, a solution exists. If no, no combination of the columns can reach that target.

Connection to Invertibility

For a square matrix, these concepts all connect to invertibility:

• The matrix is invertible if and only if it has full rank
• Full rank means the null space is trivial (just the zero vector)
• Full rank also means the determinant is non-zero
• The columns are linearly independent if and only if the matrix is invertible

All these statements are equivalent ways of saying the transformation loses no information.

Why This Matters

Understanding rank and null space answers practical questions:

• Can I solve this system? The target must be in the column space.
• Is the solution unique? Only if the null space is trivial.
• How many free variables? Equal to the nullity.
• What dimension does the transformation collapse to? The rank.

In data science, the rank tells you the "true dimensionality" of your data—how many independent directions exist. In machine learning, the null space of a weight matrix represents redundant parameters. These concepts are everywhere.

Key Takeaways

• The column space is the set of all possible outputs $A\vec{x}$—the span of the columns
• The rank is the dimension of the column space
• The null space is everything the matrix flattens to zero
• The nullity is the dimension of the null space
• Rank-Nullity Theorem: rank + nullity = number of columns
• A square matrix is invertible if and only if it has full rank (trivial null space)
• Zero determinant, rank-deficient, and non-trivial null space all mean the same thing: the matrix is singular