Understanding Linear Independence Through \(Ax=0\)
This note follows the beginning of Gilbert Strang's Section 3.4, Independence, Basis and Dimension.
The central question is:
\[ \boxed{\text{Does every vector contribute a genuinely new direction, or is there redundancy?}} \]
1. Linear independence using \(Ax=0\)
Suppose vectors \(v_1,\ldots,v_n\) are placed as the columns of a matrix:
\[ A=[v_1\;v_2\;\cdots\;v_n]. \]
Then
\[ Ax=0 \]
means
\[ x_1v_1+x_2v_2+\cdots+x_nv_n=0. \]
The vectors are linearly independent when the only solution is
\[ x_1=x_2=\cdots=x_n=0. \]
Equivalently,
\[ \boxed{N(A)=\{0\}.} \]
If \(Ax=0\) has a nonzero solution, then the vectors are dependent.
The practical meaning is:
\[ \boxed{\text{independent}=\text{no redundancy}.} \]
2. Two almost parallel vectors can still be independent
Consider
\[ v_1= \begin{bmatrix} 1\\ 0 \end{bmatrix}, \qquad v_2= \begin{bmatrix} 1\\ 0.00001 \end{bmatrix}. \]
Put them into a matrix:
\[ A= \begin{bmatrix} 1&1\\ 0&0.00001 \end{bmatrix}. \]
To test independence, solve
\[ Ax=0. \]
Reducing \(A\):
\[ \begin{bmatrix} 1&1\\ 0&0.00001 \end{bmatrix} \longrightarrow \begin{bmatrix} 1&1\\ 0&1 \end{bmatrix} \longrightarrow \begin{bmatrix} 1&0\\ 0&1 \end{bmatrix}. \]
There is a pivot in both columns.
Therefore there are no free variables and
\[ x_1=x_2=0. \]
So
\[ \boxed{N(A)=\{0\}} \]
and the two vectors are independent.
This shows that independence is an exact property. Vectors can be extremely close to parallel and still be independent if one is not exactly a multiple of the other.
3. Why any set containing the zero vector is dependent
Consider
\[ v_1= \begin{bmatrix} 1\\ 1 \end{bmatrix}, \qquad v_2= \begin{bmatrix} 0\\ 0 \end{bmatrix}. \]
Then
\[ A= \begin{bmatrix} 1&0\\ 1&0 \end{bmatrix}. \]
Solve
\[ Ax=0. \]
This gives
\[ x_1=0, \]
while \(x_2\) is free because the second column is zero.
Choose
\[ x_2=1. \]
Then
\[ x= \begin{bmatrix} 0\\ 1 \end{bmatrix} \]
is a nonzero solution of \(Ax=0\).
Equivalently,
\[ 0 \begin{bmatrix} 1\\ 1 \end{bmatrix} + 1 \begin{bmatrix} 0\\ 0 \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}. \]
The coefficients are not both zero.
Therefore the vectors are dependent.
So:
\[ \boxed{\text{Any set containing the zero vector is automatically dependent.}} \]
4. Why three vectors in \(\mathbb R^2\) must be dependent
Take
\[ v_1= \begin{bmatrix} 1\\ 0 \end{bmatrix}, \qquad v_2= \begin{bmatrix} 0\\ 1 \end{bmatrix}, \qquad v_3= \begin{bmatrix} 2\\ 3 \end{bmatrix}. \]
Put them into a matrix:
\[ A= \begin{bmatrix} 1&0&2\\ 0&1&3 \end{bmatrix}. \]
Here
\[ m=2, \qquad n=3. \]
There are only two rows, so there can be at most two pivots:
\[ r\le2. \]
Therefore three pivots are impossible.
At least one column must be free.
Solving
\[ Ax=0 \]
gives
\[ x_1+2x_3=0, \]
\[ x_2+3x_3=0. \]
Set the free variable
\[ x_3=1. \]
Then
\[ x_1=-2, \qquad x_2=-3. \]
So
\[ x= \begin{bmatrix} -2\\ -3\\ 1 \end{bmatrix} \]
is a nonzero nullspace vector.
Thus
\[ -2v_1-3v_2+v_3=0. \]
Equivalently,
\[ v_3=2v_1+3v_2. \]
The third vector adds no new independent direction.
5. Why pivots cannot reach n when n > m
For an \(m\times n\) matrix,
\[ \boxed{r\le\min(m,n).} \]
A pivot needs its own pivot row.
Therefore the number of pivots can never exceed the number of rows:
\[ r\le m. \]
If
\[ n>m, \]
then
\[ r\le m<n. \]
Therefore
\[ r<n. \]
So not every column can be a pivot column.
Hence at least one free variable exists.
A free variable gives a nonzero solution to \(Ax=0\).
Therefore the columns must be dependent.
The complete logical chain is
\[ \boxed{ n>m \Rightarrow r<n \Rightarrow \text{free variable} \Rightarrow Ax=0\text{ has a nonzero solution} \Rightarrow \text{dependent columns} }. \]
Thus:
\[ \boxed{\text{Any }n\text{ vectors in }\mathbb R^m\text{ are dependent if }n>m.} \]
6. Geometric meaning of n > m
In \(\mathbb R^2\), two independent vectors can already span the entire plane.
For example,
\[ v_1=(1,0), \qquad v_2=(0,1). \]
Every vector
\[ (a,b) \]
can be written as
\[ a v_1+b v_2. \]
So after two independent directions, the entire plane is already available.
A third vector cannot introduce a new third direction while remaining inside \(\mathbb R^2\).
It must be a combination of the first two.
Similarly, \(\mathbb R^3\) can contain at most three independent directions. Any fourth vector must be dependent on the others.
7. Why Strang defines dependence using a zero combination
Suppose
\[ v_3=2v_1+5v_2. \]
Then \(v_3\) is clearly redundant.
Move everything to one side:
\[ 2v_1+5v_2-v_3=0. \]
The coefficients
\[ 2,\;5,\;-1 \]
are not all zero.
Therefore this is a nontrivial solution of
\[ Ax=0. \]
So the vectors are dependent.
But the same equation can also be rearranged as
\[ v_1=-\frac52v_2+\frac12v_3. \]
So there is no need to label one particular vector as the unique redundant vector.
The equation
\[ Ax=0 \]
treats all columns equally and simply asks whether there is redundancy anywhere in the set.
Core mental model
Linear independence asks:
\[ \boxed{\text{Does every vector contribute something genuinely new?}} \]
If yes, the vectors are independent.
If some nonzero combination produces zero, then there is a hidden relationship among the vectors and the set is dependent.
The matrix test is:
\[ \boxed{Ax=0.} \]
If the only solution is
\[ x=0, \]
the columns are independent.
If there is any nonzero solution, the columns are dependent.
Note metadata
- Note type: learning-note
- Subject: learning-notes
- Source: Introduction to Linear Algebra, 5th Edition — Section 3.4: Independence, Basis and Dimension