import numpy as np
A = np.array([[1, 1],
[0, 1]])
print(A)[[1 1]
[0 1]]5 Matrices as Transformations
The itch
Everything we have done so far happens to one vector at a time. We measure a vector, add two together, and compare their directions. The vector remains stationary while we inspect it. However, almost no practical process in machine learning leaves data sitting still. The entire process relies on movement. We take every data point and shift it somewhere more useful, repeating this until tangled points finally separate.
Consider what a model actually does to its input data. As we established earlier, a face arrives as a long list of pixel numbers. This forms a vector in a space with far too many dimensions to visualize. That raw vector is entirely useless on its own because the numbers merely represent pixel brightness. A neural network transforms that vector layer by layer. It reshapes the mathematical space so that faces of the same person land near each other, while different people land far apart. The system did not add any new information to the photograph. It simply bent and stretched the space until the correct answer became easy to read.
This brings us to transformations. These are operations that take an entire space of vectors and move all of them at the exact same time. We need this operation to be something a computer can store and apply. As always, this means the process must translate entirely into numbers.
Here is the most important realization of this section. An enormous category of these space-moving operations can be captured completely by a simple grid of numbers. This is not an approximation. The complete behavior of the transformation is pinned down by a handful of numbers arranged in a box. This small box dictates exactly what happens to every single vector in the entire space.
That box is called a matrix. This chapter focuses on the single idea that makes linear algebra make sense. A matrix is much more than a random grid of numbers. A matrix is an instruction that moves space itself. The grid is simply how we write that movement down for the computer.
The picture
Moving an entire space sounds like it requires an enormous amount of data. It seems we would have to list the final destination for every individual vector, which is impossible. Fortunately, we do not have to do that.
Recall the two reference vectors from earlier. These are \([1, 0]\) and \([0, 1]\), which point exactly one step along each main axis. Every vector is built directly from them. The vector \([3, 2]\) is just three steps along the horizontal axis and two steps along the vertical axis. Every vector in the flat plane relies on a specific amount of \([1, 0]\) and a specific amount of \([0, 1]\). Because they are the fundamental building blocks of the entire space, they are called the basis vectors.
Now consider a well-behaved transformation. This specific type of transformation keeps all grid lines perfectly straight and evenly spaced, and it never moves the origin. For this type of operation, knowing where the two basis vectors land tells us exactly where every other vector lands.
The vector \([3, 2]\) requires three of the first basis vector and two of the second. After the transformation, it still requires three of wherever the first basis vector moved, plus two of wherever the second moved. The numerical recipe survives the transformation completely untouched. Only the underlying ingredients changed.
To describe a transformation of the entire plane, we do not need to track infinite vectors. We only need to track two. If we know the final locations of \([1, 0]\) and \([0, 1]\), the entire transformation is completely locked in. Every other vector simply rides along using the linear combination that originally defined it.
Imagine a transformation that leaves \([1, 0]\) exactly where it is. However, it sends \([0, 1]\) to \([1, 1]\), tilting the vertical direction over to the right. The basis vectors moved, so the entire grid moved with them. Every square stretches into a slanted parallelogram, much like pushing the top of a stack of paper. No individual vector was moved manually. They all simply followed the two basis vectors.
Notice the strict rule we applied earlier. The grid lines must stay straight and evenly spaced, and the origin must stay completely fixed. This is the exact definition of a linear transformation. These well-behaved transformations are the only kind a matrix can actually describe.
The math, built up
We established that a transformation is defined entirely by where the two basis vectors land. To write this transformation as raw numbers, we simply write down those two final landing spots. That is all a matrix truly is.
We stack those two landing spots as vertical columns. The destination of the first basis vector sits on the left, and the destination of the second sits on the right:
\[A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}\]
Reading this correctly removes all the mystery. The first column is \([1, 0]\) read downward. This is exactly where the first basis vector landed. The second column is \([1, 1]\), which is where the second basis vector landed. A matrix is not just a random grid of numbers. It is a strict record of what happened to the basis vectors, listing one landing spot per column. A matrix simply hands you the two new arrows side by side.
Applying this transformation to a vector is written mathematically as \(A\mathbf{v}\). This process carries \(\mathbf{v}\) along using its own recipe. If \(\mathbf{v} = [x, y]\), the vector relies on \(x\) amounts of the first basis vector and \(y\) amounts of the second. After the transformation, it needs \(x\) amounts of the first landing spot and \(y\) amounts of the second. The result is just a linear combination of the matrix columns:
\[A\mathbf{v} = x \begin{bmatrix} 1 \\ 0 \end{bmatrix} + y \begin{bmatrix} 1 \\ 1 \end{bmatrix}\]
Multiplying a matrix by a vector means scaling each column of the matrix by the matching number in the vector, and then adding them together. This is exactly the linear combination from the second chapter written in a new format. The matrix provides the ingredients through its columns, and the vector provides the scaling amounts.
We can apply this to the vector \([3, 2]\) to see the result:
\[A\mathbf{v} = 3 \begin{bmatrix} 1 \\ 0 \end{bmatrix} + 2 \begin{bmatrix} 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 0 \end{bmatrix} + \begin{bmatrix} 2 \\ 2 \end{bmatrix} = \begin{bmatrix} 5 \\ 2 \end{bmatrix}\]
The vector \([3, 2]\) successfully landed at \([5, 2]\). The shearing motion pushed it to the right by an amount equal to its physical height. We achieved this directly from the columns without memorizing any special formulas.
There is a mechanical way to perform this calculation, which is usually taught first. The top entry of the result is \(1\cdot3 + 1\cdot2 = 5\), and the bottom is \(0\cdot3 + 1\cdot2 = 2\). Each final entry is just a dot product of a matrix row and the vector. This row-by-row method is correct and often faster for manual calculation. However, if you only learn this mechanical rule, the matrix loses its geometric meaning. It becomes a grid of numbers you blindly shuffle. Viewing the columns as physical landing spots keeps the geometry completely alive. The row view is for computing, while the column view is for true understanding.
This concept scales perfectly to higher dimensions. A transformation of three-dimensional space is defined by three basis vectors, so its matrix has exactly three columns. A transformation in a three-hundred-dimensional space requires three hundred landing spots, creating a matrix three hundred columns wide. The operation \(A\mathbf{v}\) remains exactly the same. You scale each column by the matching entry in \(\mathbf{v}\) and add the results together.
Build it yourself
NumPy stores a matrix as an array of rows. Applying it to a vector requires a single operator. We will compute the transformation using both our manual column recipe and the built-in shortcut to verify they match.
First, we will build the shear matrix. NumPy requires us to enter the data row by row, even though we are thinking about vertical columns:
You must keep the columns in mind when reading the code. The first vertical column is [1, 0], which is where the first basis vector landed. The second vertical column is [1, 1], which is where the second vector landed.
Now we can apply this matrix to the vector \([3, 2]\) using our linear combination rule. We scale each column by the matching entry and add them together:
v = np.array([3, 2])
col1 = A[:, 0] # first column: where [1,0] went
col2 = A[:, 1] # second column: where [0,1] went
result = v[0] * col1 + v[1] * col2
print(result)[5 2]The answer is [5, 2], which matches our manual calculation. The code pulls out the columns and scales them exactly as we described. There is no hidden matrix magic happening. It is just a basic linear combination applied to the columns.
NumPy provides a built-in shortcut for this entire process using the @ symbol:
print(A @ v)[5 2]The output is exactly the same. The @ symbol represents matrix application. It automatically performs the column recipe we just wrote manually. We can prove they are completely identical:
print(np.array_equal(v[0] * A[:, 0] + v[1] * A[:, 1], A @ v))TrueThe system returns True. Applying a matrix is mathematically identical to combining its vertical columns.
We can apply this matrix to several vectors at once to see the physical motion happening:
points = np.array([[1, 0],
[0, 1],
[1, 1],
[2, 1]])
for p in points:
print(p, "->", A @ p)[1 0] -> [1 0]
[0 1] -> [1 1]
[1 1] -> [2 1]
[2 1] -> [3 1]Look at the arrows in the output. The first basis vector \([1, 0]\) stays perfectly still. The second basis vector \([0, 1]\) tilts over to \([1, 1]\). The corner at \([1, 1]\) slides further right to \([2, 1]\). Every point in the space shifts to the right by an amount equal to its height. This perfectly demonstrates the shearing motion across the entire grid.
None of this depends on the vectors being small. If you give A three hundred columns and v three hundred entries, A @ v still flawlessly combines the columns using the entries of v.
Where it lives in ML
We previously learned that a neural network layer runs many linear combinations on its input data. We can now give that collection of combinations its proper name. It is a matrix.
A network layer receives a vector from the previous neurons. Each neuron in the current layer runs one linear combination using its own row of weights. If you stack those rows together, you build a grid of numbers. That grid is a matrix. The entire layer applies a single matrix to the incoming vector, executing \(A\mathbf{v}\). This simple process of scaling columns and adding them together runs billions of times inside every modern model. When developers say a model has weights, they are simply talking about the numbers sitting inside these matrices. Training a model is just searching for the specific matrices that move the data in the most useful direction.
This completely reframes how a neural network operates. Each layer is a transformation that moves the mathematical space. The input starts as a raw point where the useful information is tangled and messy. The network pushes that point through a sequence of matrix transformations. Each matrix stretches and bends the space slightly more, until the data lands in a location where the final answer is obvious.
Matrix transformations handle almost everything in programming. Rotating an image, scaling a photograph, and flipping a graphic are all just matrices applied to pixel vectors. Whenever data is reshaped or reoriented, a matrix is doing the actual work.
We must also recognize a major limitation. A single matrix can only perform a linear transformation. It can stretch, rotate, shear, or flip the space. However, it can never curve or bend the space. Straight lines must always remain straight. If you stack hundreds of matrices together, the combined result is still just one massive linear transformation. It will never gain the ability to bend.
This is why a real neural network inserts a small non-linear step between every matrix. This extra step introduces the critical bending motion. The matrices handle the heavy movement, and the bending step allows those moves to untangle highly complex data. The matrix serves as the primary engine of the network, and the bending step makes it truly intelligent.
Common misunderstandings
Matrices naturally attract confusion. The mathematical notation makes it very easy to forget they are physical transformations.
In NumPy, the * symbol is not matrix application. This is a very common programming trap. The correct matrix operator is @. Using the standard * symbol performs basic element-by-element multiplication. It will happily return the wrong answer without generating an error:
import numpy as np
A = np.array([[1, 1],
[0, 1]])
v = np.array([3, 2])
print(A @ v) # correct matrix application: [5, 2]
print(A * v) # elementwise multiplication, incorrect math[5 2]
[[3 2]
[0 2]]The first output is the correct geometric transformation. The second output is mathematical nonsense in this context. You must actively use @ to apply a matrix correctly.
A matrix is defined by its action, not its numbers. It is easy to view a matrix as just a block of data. However, two matrices with completely different numbers can perform almost the exact same physical movement. The numbers only describe where the basis vectors land. When you read a matrix, you should look at the vertical columns and picture where they are pointing. The grid is just the written record of the transformation.
The order of operations matters. If you apply two transformations, the order completely changes the final result. Rotating a vector and then stretching it produces a different outcome than stretching it first and rotating it second. Actions in the real world follow this exact rule. Putting on socks and then shoes is very different from putting on shoes and then socks. Matrix math works the same way. You cannot freely shuffle the order of operations. This is the biggest difference between linear algebra and standard arithmetic.
A matrix cannot move the origin. Every linear transformation leaves the origin locked in place. You cannot use a single matrix to slide the entire space two steps to the right. A movement that shifts the origin is not linear, so a basic matrix cannot perform it. If we need to slide the space, we have to add an extra step to the process. For now, simply remember that a bare matrix will always pin the origin to the center of the grid.
Check your intuition
Try to answer these questions before expanding the answers. These ask you to picture the transformations directly.
1. A matrix has columns \([1, 0]\) and \([0, 1]\), making \(A = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}\). Where does it send the vector \([7, -3]\)? What is this transformation doing to the space?
2. A matrix sends the first basis vector to \([2, 0]\) and the second to \([0, 2]\). Write the matrix, and describe the transformation plainly.
3. A matrix sends \([1, 0]\) to \([0, 1]\) and sends \([0, 1]\) to \([-1, 0]\). Without computing anything, what motion is this?
4. Apply \(A = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}\) to \([1, 1]\) using the column recipe. What lands where, and why is this transformation not a basic rotation?
5. Someone claims a single matrix can take every point in the plane and slide it three units to the right, moving \([0, 0]\) to \([3, 0]\). Are they right?
TipAnswers
1. It sends \([7, -3]\) perfectly to \([7, -3]\). The columns perfectly match the original basis vectors, meaning the transformation leaves everything exactly where it started. This is called the identity matrix. It does absolutely nothing to the space. Applying it is the matrix equivalent of multiplying a normal number by one.
2. The matrix is \(\begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix}\). This transformation doubles the length of every single vector without changing its angle. It is a uniform scaling that stretches the entire flat plane outward from the origin. Every square on the grid doubles in size.
3. This is a perfect rotation by ninety degrees counterclockwise. The first basis vector points east and rotates north to \([0, 1]\). The second vector points north and rotates west to \([-1, 0]\). Because everything follows the basis vectors, the entire plane spins counterclockwise. We can see this physical motion just by reading the columns.
4. Applying the vector gives \(1\cdot[2, 0] + 1\cdot[0, 3] = [2, 3]\). The transformation stretches the horizontal axis by two and the vertical axis by three. This is not a rotation because it stretches different directions by completely different amounts. It distorts the physical shapes rather than spinning them gracefully. A rotation preserves all original lengths, while this operation clearly alters them.
5. No. Sliding every point three units to the right forces the origin to move. We know every linear transformation must leave the origin completely fixed. A basic matrix can never perform this sliding motion on its own. We will learn how to handle that specific movement later in the book.