Condition number

The formula

The condition number of a square matrix A is the product of its norm and the norm of its inverse ^[5]:

κ(A) = ‖A‖ · ‖A⁻¹‖

Reading: how much the worst-case output error can be blown up relative to the input error when you solve A·x = b. It is always at least 1.

Using the 2-norm (the usual one), this collapses to a ratio of singular values ^[5]:

κ₂(A) = σ_max / σ_min

Reading: the longest axis of the ellipsoid A maps a unit sphere into, divided by the shortest. A fat, round ellipsoid is well-conditioned; a thin sliver is not.

The reason engineers care is the perturbation bound. If you perturb the right-hand side b, the relative error it drags into the answer x is bounded by:

‖Δx‖ / ‖x‖  ≤  κ(A) · ‖Δb‖ / ‖b‖

Reading: a 0.001% error in your load vector can become a κ(A) × 0.001% error in your displacements. The condition number is the amplifier's gain.

That gain turns into a digit count. With a numerically stable solver and inputs good to about 16 digits (IEEE double precision, machine epsilon ≈ 2.2×10⁻¹⁶), expect to lose roughly log₁₀ κ(A) digits ^[5]:

digits left ≈ 16 − log₁₀ κ(A)

If κ(A) = 10¹⁰, you keep about 6 good digits and the rest is rounding noise.

Where you meet it

• A fixture or bracket FEA that returns garbage displacements. You built a stiffness matrix K and solved K·u = F. If the model has a nearly unconstrained mode — a part that can drift or spin because a boundary condition is missing or a joint is almost a mechanism — K goes nearly singular, κ(K) climbs past 10¹², and the displacement vector u comes back looking physical but wrong in the fifth decimal that mattered. The solver didn't fail. Your matrix did.

• Least-squares curve fits at a review board. Fitting a high-order polynomial to sensor data builds a Vandermonde-type matrix that is famously ill-conditioned. The normal-equations form AᵀA squares the condition number, so a fit that looked marginal at κ = 10⁶ becomes hopeless at 10¹² once you square it. This is why the numerics people push QR or SVD instead of forming AᵀA — same fit, half the digit loss.

• A near-balanced bridge or divider on the bench. A Wheatstone bridge trimmed to the edge of null, or a resistor network where two large numbers nearly cancel, is the scalar version of ill-conditioning: the output is a tiny difference of two big quantities, and every bit of noise in the big quantities lands full-scale on the small one.

• Control and estimation on a test stand. A Kalman filter's covariance update, or a system-identification least-squares, will quietly diverge when the data matrix is rank-deficient in practice — two inputs that never moved independently during the test. The math says solvable; the condition number says you never excited enough to separate them.

How it works

The condition number is a property of the problem, not the solver. This is the part people get backwards. A stable algorithm gives you an answer whose backward error is tiny — it exactly solves a problem very close to the one you posed. But if the problem is ill-conditioned, "very close" still maps to a far-away answer. No amount of clever code fixes a κ = 10¹⁴ matrix; you have to fix the model.

A worked feel for the amplifier. Take the deliberately nasty 2×2:

A = [ 1    1        ]      κ(A) ≈ 4×10¹⁰
    [ 1    1+1e-10  ]

Perturb b by a relative 3.5×10⁻¹⁴ — smaller than a rounding error — and the solution x shifts by a relative 1×10⁻³. That is the perturbation bound firing at almost exactly the condition number: tiny in, huge out.

The Hilbert matrix is the classic demo because it looks harmless. The 6×6 Hilbert matrix (entries 1/(i+j−1)) already has κ ≈ 1.5×10⁷; the 8×8 hits ≈ 1.5×10¹⁰. Solve an 8×8 Hilbert system whose true answer is all ones and you get back roughly 6×10⁻⁸ relative error — about 7 good digits left, roughly 9 gone, close to the log₁₀ κ ≈ 10 worst-case prediction.

Gotchas that bite:

• Big κ doesn't mean big numbers. Scaling matters. A matrix with entries spanning many orders of magnitude can be perfectly conditioned; a matrix of numbers all near 1 can be terrible. Row/column equilibration (rescaling units so terms are comparable) sometimes drops κ by orders of magnitude for free. A stiffness matrix mixing meters and micro-radians is asking for trouble that is purely a units artifact.

• The determinant is not the condition number. A near-zero determinant does not by itself mean ill-conditioned — you can scale a well-conditioned matrix to have any determinant you like. Use κ, not det.

• Orthogonal / rotation matrices have κ = 1. They preserve lengths, so they never amplify error. That is exactly why QR and SVD factorizations, built from orthogonal pieces, are the numerically safe way to solve tough systems.

• κ is an upper bound, not a promise. A particular right-hand side may lose far fewer digits than the worst case. The condition number tells you the danger ceiling, which is the number you want at a design review anyway.

Rule of thumb worth taping to the monitor: if κ(A) exceeds 1/ε_machine (about 10¹⁶ in double precision), the matrix is numerically singular and the answer is noise. Anything past 10⁸ deserves a second look before you trust it.

History

The idea shows up alongside the first serious attempts to do linear algebra on early computers, when people discovered that arithmetic with finite digits could turn a clean textbook method into nonsense. In 1947 John von Neumann and Herman Goldstine published an error analysis of solving linear systems by elimination, one of the first honest reckonings with how rounding accumulates in matrix work ^[1].

The term itself, and the notion of a condition number attached to a matrix, came from Alan Turing (1912–1954) in a 1948 paper titled "Rounding-off errors in matrix processes," in the Quarterly Journal of Mechanics and Applied Mathematics, vol. 1, pp. 287–308 ^[2][3]. Turing had spent the postwar years at the National Physical Laboratory in London designing the ACE computer, and running actual elimination tests on real systems of equations sharpened his sense that some matrices were intrinsically treacherous no matter how careful the arithmetic ^[4]. In the same paper he wrote Gaussian elimination as an LU factorization — another idea now taught on day one of every numerical methods course ^[3].

The theory got its rigorous finish from James Wilkinson at the same laboratory across the 1950s and 60s. Wilkinson built the framework of backward error analysis ^[1] and showed that Gaussian elimination with partial pivoting is stable in the sense of producing a small backward error, with the forward error bounded by a constant times the condition number ^[6]. That is the result behind the modern rule of thumb: a good solver plus a known κ lets you predict your digit loss before you run the job.

Related tools

• /tools/beam-deflection
• /tools/euler-buckling
• /tools/vibration-natural-freq
• /tools/shaft-speed-critical
• /tools/wheatstone-bridge
• /tools/strain-gauge-bridge

Sources

1. https://nlagrouporg.wordpress.com/2019/02/18/wilkinson-and-backward-error-analysis/
2. https://turing.academicwebsite.com/publications/20-rounding-off-errors-in-matrix-processes
3. https://gauss.uc3m.es/fdopico/papers/arbor-2011-turing.pdf
4. https://mathshistory.st-andrews.ac.uk/Biographies/Turing/
5. https://en.wikipedia.org/wiki/Condition_number
6. Higham, N. J., "Accuracy and Stability of Numerical Algorithms," 2nd ed., SIAM, 2002, ch. 9 — https://epubs.siam.org/doi/book/10.1137/1.9780898718027