The Question Behind the Mathematics
While learning linear algebra, it is easy to understand how to perform a matrix calculation without understanding why the concept was needed in the first place.
A useful way to approach the subject is therefore:
What practical problem are we trying to solve? → Why is what we already know insufficient? → What new mathematical concept solves that problem? → Then learn the calculation.
This perspective gives a natural path from matrices to \(Ax=b\), inverse matrices and Gaussian elimination.
A Matrix Can Represent the Rules of a System
Consider a bakery that makes cakes and batches of cookies.
Suppose one cake requires 2 kg of flour and 1 kg of sugar, while one batch of cookies requires 1 kg of flour and 3 kg of sugar.
Those production rules can be stored in the matrix
\[A=\begin{bmatrix}2&1\\1&3\end{bmatrix}.\]
The columns represent the products and the rows represent the ingredients:
| Cake | Cookies | |
|---|---|---|
| Flour | 2 | 1 |
| Sugar | 1 | 3 |
This gives an important interpretation:
A matrix is not necessarily just a rectangular collection of numbers. It can encode the rules or relationships of a system.
Matrix Multiplication Answers a Forward Question
Suppose the bakery produces 10 cakes and 20 batches of cookies. Represent the production quantities by
\[x=\begin{bmatrix}10\\20\end{bmatrix}.\]
Then
\[Ax=\begin{bmatrix}2&1\\1&3\end{bmatrix}\begin{bmatrix}10\\20\end{bmatrix}=\begin{bmatrix}40\\70\end{bmatrix}.\]
So the bakery requires 40 kg of flour and 70 kg of sugar.
Conceptually, the matrix performs a transformation:
\[\text{production quantities}\xrightarrow{A}\text{ingredient requirements}.\]
This is one useful meaning of saying that A represents an action. The action is determined by the relationships encoded in its entries.
Why Does Ax = b Appear?
Now reverse the practical question.
Suppose we know that 40 kg of flour and 70 kg of sugar were used, but we do not know how many cakes and cookie batches were produced.
Now we know
\[A=\begin{bmatrix}2&1\\1&3\end{bmatrix}\]
and
\[b=\begin{bmatrix}40\\70\end{bmatrix},\]
but \(x\) is unknown.
The problem becomes
\[Ax=b.\]
Written as ordinary equations:
\[2x+y=40\]
\[x+3y=70.\]
So \(Ax=b\) expresses a very general question:
I know the rules of the system \(A\), and I know the resulting output \(b\). What input \(x\) produced that output?
Why Do We Need Elimination?
Once we have \(Ax=b\), knowing how to write the problem is not enough. We need a systematic way to find the unknown vector \(x\).
For the bakery equations,
\[2x+y=40\]
\[x+3y=70,\]
we can eliminate one variable. Multiplying the second equation by 2 gives
\[2x+6y=140.\]
Subtracting the first equation gives
\[5y=100,\]
so
\[y=20.\]
Substituting back gives
\[x=10.\]
This is the basic idea behind Gaussian elimination: systematically eliminate unknowns until the system becomes easy to solve.
With two equations this may seem unnecessary. With hundreds or thousands of simultaneous equations, however, we need a systematic computational procedure rather than ad hoc manipulation.
Why Does the Inverse Matrix Exist as a Concept?
There is another way to think about the same problem.
If \(A\) represents an action that takes \(x\) to \(b\), can that action be reversed?
We have
\[x\xrightarrow{A}b.\]
If the transformation is reversible, its inverse \(A^{-1}\) takes us in the opposite direction:
\[b\xrightarrow{A^{-1}}x.\]
Starting from
\[Ax=b,\]
multiplying by \(A^{-1}\) gives
\[A^{-1}Ax=A^{-1}b.\]
Since
\[A^{-1}A=I,\]
we obtain
\[x=A^{-1}b.\]
The important intuition comes before the formula:
The inverse matrix asks whether the action represented by \(A\) can be undone.
Not every matrix has an inverse. Understanding why some transformations cannot be reversed leads naturally to further ideas in linear algebra.
Elimination Matrices: Turning an Instruction into a Matrix
Gaussian elimination contains instructions such as
\[R_2\leftarrow R_2-2R_1.\]
Linear algebra can represent even this instruction as matrix multiplication.
For example,
\[E=\begin{bmatrix}1&0&0\\-2&1&0\\0&0&1\end{bmatrix}\]
performs the operation of subtracting twice row 1 from row 2 when it multiplies a compatible matrix or vector from the left.
This is another example of the same deeper idea: a matrix can represent an operation.
The Concepts Are Not Separate Tricks
Seen only as formulas, matrices, \(Ax=b\), inverses and elimination can appear to be unrelated topics that must be memorized one after another.
Seen through the problem they are solving, they form a natural chain:
\[\text{Real system}\rightarrow\text{encode its rules in }A\rightarrow Ax=b\rightarrow\text{find }x.\]
Gaussian elimination provides a systematic procedure for finding \(x\). The inverse asks whether the transformation represented by \(A\) can be reversed. Elimination matrices show that the elimination operations themselves can also be represented by matrices.
Where This Leads Next: LU Factorization
The next question is not to invent another unrelated matrix operation.
It is:
Can we organize the information produced during Gaussian elimination?
That question leads to the factorization
\[A=LU.\]
Here \(U\) is closely connected to the matrix produced by elimination, while \(L\) records the elimination multipliers in a structured form.
LU factorization deserves its own note after its practical meaning and mathematics are developed fully.
Connection to AI and Quantum Computing
This way of thinking about matrices becomes increasingly important beyond introductory linear algebra.
In AI, matrices can encode learned weights and transformations of data representations. In quantum computing, quantum states are represented by vectors and quantum operations by matrices (more specifically, unitary operators in the standard circuit model).
So the important habit is not merely learning how to multiply matrices. It is learning to ask:
What does this matrix represent, what does it do to a vector, what information does the transformation preserve or lose, and can the action be reversed?
Those questions provide a practical foundation for the linear algebra that follows.
Note metadata
- Note type: learning-note
- Subject: learning-notes
- Source: Introduction to Linear Algebra, Fifth Edition