Matrices and matrix algebra

The formula

A matrix acting on a vector:

y = A·x        y_i = Σ_j a_ij · x_j

Each output is a weighted sum of every input, and row i of A holds the weights. That one sentence is the entire job description — a matrix is a linear map written down as a table.

Multiplying two matrices:

(A·B)_ij = Σ_k a_ik · b_kj      (m×n)·(n×p) → (m×p)

Row of the left into column of the right. What it means: A·B is the single map that does B first, then A. The inner dimensions must agree — the outputs of B have to be legal inputs to A — and the shape rule is your first sanity check on any chain of transforms.

The order matters. With A = [[1,2],[3,4]] and B = [[0,1],[1,0]]:

A·B = [[2,1],[4,3]]     B·A = [[3,4],[1,2]]

Same two matrices, different answers. B on the right swaps columns of A; on the left it swaps rows.

Solving a system means undoing the map:

A·x = b   →   x = A⁻¹·b     where A·A⁻¹ = I  and  A⁻¹ exists only if det(A) ≠ 0

Read it as "find the input that produced this output." The identity I is the matrix that does nothing; the inverse exists exactly when A doesn't collapse some direction to zero. In practice you never form A⁻¹ — you factor and solve, which is faster and better behaved.

Where you meet it

The stiffness model at the design review. Every finite-element run behind a structural margin chart is one equation: K·u = f — stiffness matrix times displacements equals loads. Two springs in series (100 and 200 N/mm, tip load 50 N) give K = [[300,−200],[−200,200]], f = [0,50], and solving yields deflections of 0.5 and 0.75 mm. The million-DOF wing model on the reviewer's slide is the same equation with more rows.
Circuit nodal analysis. Kirchhoff's current law at every node of a network stacks into G·v = i — conductance matrix times node voltages equals injected currents. SPICE assembles and solves that matrix at every timestep; a voltage divider is the one-unknown special case you can do in your head.
Coordinate transforms in a pointing budget. Body frame to gimbal frame to nav frame to Earth frame — each hop is a 3×3 rotation matrix, and the full chain is one matrix product. A 90° rotation R = [[0,−1],[1,0]] sends [1,0] to [0,1]; chain five of those in the wrong order and the antenna stares at dirt. Graphics and robotics pipelines run the same math as 4×4 homogeneous transforms.
The bench, correcting a multi-axis sensor. A 6-axis load cell's calibration sheet is a 6×6 matrix C mapping true loads to bridge outputs; the cross-axis terms are the off-diagonal entries. Recovering the true loads from readings is F = C⁻¹·r — matrix algebra you run at every sample.

How it works

The mental model that makes everything else fall out: a matrix is not a spreadsheet of numbers, it's an action. Multiplication is composition of actions, which is why it's associative — A·(B·C) = (A·B)·C, so you can group a transform chain however is convenient — and why it is not commutative: rotate then translate is not translate then rotate, and no notation can fix that.

Matrix algebra keeps most of ordinary arithmetic and quietly drops the rest. You can add, scale, and distribute as expected. You cannot divide, and you cannot cancel: [[1,0],[0,0]]·[[0,0],[0,1]] is the zero matrix even though neither factor is zero, so A·B = A·C does not imply B = C. Engineers trained on scalar algebra rediscover this the hard way, usually inside a derivation that "should" have worked.

Solving A·x = b is elimination — subtract multiples of rows until the system is triangular, then back-substitute. Cost is about 2n³/3 arithmetic operations: roughly 7×10⁸ for n = 1000, which a laptop does in well under a second, but the n³ scaling is why a 10× bigger mesh is a 1000× bigger solve and why real FEA and SPICE codes lean hard on sparsity — a stiffness matrix is almost entirely zeros, because each node only touches its neighbors, and sparse solvers only pay for the nonzeros. Algorithms that beat n³ on paper exist (Strassen's recursion gets about n^2.81), but the constant factors and memory behavior mean the workhorse libraries mostly don't bother ^[10].

Two numerical gotchas earn their own paragraph. First: don't compute the inverse to solve a system. x = A⁻¹·b costs roughly three times the flops of an LU factorization, and it's less accurate; every serious library (LAPACK underneath MATLAB, NumPy, and your FEA code) factors instead ^[5]. Second: the determinant is a terrible singularity test. Scale a 100×100 matrix by 0.1 and its determinant drops by 10¹⁰⁰ while the system is exactly as solvable as before. The honest gauge is the condition number κ — the factor by which input error gets amplified — and the rule of thumb is that you lose about log₁₀(κ) digits of accuracy. A stiffness matrix with κ near 10¹² in double precision is telling you the answer has four good digits, no matter how many the printout shows. Pivoting during elimination (swap in the biggest available pivot) is the standard defense against the related roundoff failure ^[5].

The mistake that actually gets people is order-of-operations in transform chains. Whether vectors are columns (y = A·x, chains grow leftward) or rows (y = x·A, chains grow rightward) is a convention, different toolchains pick differently, and a chain assembled half in one convention and half in the other produces plausible-looking garbage. Check it with a known 90° rotation before you trust anything downstream.

History

The arithmetic is Victorian, but the arrays are ancient. Chapter Eight of the Chinese Nine Chapters on the Mathematical Art, compiled roughly two thousand years ago, sets up simultaneous linear equations as rectangular coefficient tables and solves them by the same elimination we still teach ^[1]^[5]. Determinants came next and came early: Seki Takakazu developed them in Japan in 1683 for elimination theory, working the formulas up through 5×5 arrays, while Leibniz was independently writing down the same idea in Europe in the 1680s and 90s ^[1]^[8]^[9]. Gauss put systematic elimination to work in the first decade of the 1800s reducing least-squares systems from asteroid observations — the method that carries his name predates him by nearly two millennia ^[1]^[5].

The matrix as an object with its own algebra took two friends walking laps around Lincoln's Inn. James Joseph Sylvester coined the word "matrix" in 1850 for the oblong array itself ^[1]^[2]. Arthur Cayley — then a barrister who treated law as a way to fund mathematics, publishing around 250 papers from chambers — supplied the algebra: his 1858 A Memoir on the Theory of Matrices defined addition, scalar multiplication, multiplication, and inverses as operations on the arrays themselves ^[1]^[3]^[4]. The same memoir states what we now call the Cayley–Hamilton theorem — every square matrix satisfies its own characteristic polynomial. Cayley proved it for 2×2, checked 3×3, and wrote that he had "not thought it necessary to undertake the labour of a formal proof of the theorem in the general case" ^[3]^[4]. He eventually traded the bar for Cambridge's Sadleirian chair and never looked back ^[3]^[4].

For sixty years matrix algebra stayed a specialist's tool. Then in July 1925 Werner Heisenberg sent Max Born a paper full of strange arrays of quantum transition amplitudes with a multiplication rule where p·q ≠ q·p. Born recognized the arrays as matrices — algebra he'd learned as a student — and within 60 days he and Pascual Jordan had recast quantum theory in matrix language; the three-man paper with Heisenberg followed before the year was out, with the non-commutativity that trips up every undergraduate sitting at the center of the theory as p·q − q·p = (h/2πi)·I ^[6]^[7]. Cayley's curiosity had become the bookkeeping of physics — and once digital computers arrived, of every stiffness model, circuit simulator, and navigation filter in the business.

Related tools

/tools/beam-deflection — the closed-form answers a stiffness matrix generalizes to arbitrary geometry
/tools/vibration-natural-freq — the single-DOF case of the matrix eigenvalue problem K·φ = ω²·M·φ
/tools/wheatstone-bridge — a four-node network you can solve by hand or by G·v = i
/tools/voltage-divider — nodal analysis with exactly one unknown
/tools/parallel-resistors — the conductance-addition rule that fills in the G matrix diagonal

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →