Vector and matrix norms

The formula

The three vector norms engineers actually use, for x = (x₁, x₂, …, x_n):

‖x‖₂ = √( x₁² + x₂² + … + x_n² )          (Euclidean norm, 2-norm)

Reading: straight-line length — the Pythagorean distance, and the square root of total energy in a sampled signal.

‖x‖₁ = |x₁| + |x₂| + … + |x_n|            (L1, taxicab norm)

Reading: add up every component's magnitude — the worst-case tolerance stack, where every error gangs up in the same direction.

‖x‖_∞ = max( |x₁|, …, |x_n| )             (L-infinity, max or Chebyshev norm)

Reading: only the single worst component counts; everything else is ignored.

All three are the p = 1, 2, ∞ cases of one family ^[1]:

‖x‖_p = ( Σ |x_i|^p )^(1/p),   p ≥ 1

Any function earning the name "norm" satisfies three axioms ^[1]: ‖x‖ = 0 only for the zero vector, ‖c·x‖ = |c|·‖x‖ for any scalar c, and the triangle inequality ‖x + y‖ ≤ ‖x‖ + ‖y‖.

For an m×n matrix A, two families matter ^[2]:

‖A‖_F = √( Σᵢ Σⱼ |a_ij|² )                (Frobenius norm)

Reading: unroll the matrix into one long vector and take its Euclidean length.

‖A‖ = max over x ≠ 0 of  ‖A·x‖ / ‖x‖      (induced/operator norm)

Reading: the worst-case gain — the biggest amplification A can apply to any input, measured in the vector norm you chose. Three of these have closed forms ^[2]: ‖A‖₁ is the largest absolute column sum, ‖A‖_∞ the largest absolute row sum, and ‖A‖₂ the largest singular value σ_max.

Where you meet it

Every RMS reading on the bench. For n samples, RMS = ‖x‖₂ / √n — the DMM's AC reading is a scaled 2-norm. Crest factor is a ratio of norms: ‖x‖_∞ over RMS. For a pure sine that's √2 ≈ 1.414; for the pulsed waveform your power analyzer chokes on, it's the number that tells you why.
Solver convergence in FEA, CFD, and SPICE. The iteration stops when ‖A·x − b‖ drops below tolerance. That default is almost always a 2-norm of the residual — which can read "converged" while one degree of freedom near a point load is still badly wrong. The ∞-norm is what catches it.
The error budget at the review board. RSS-ing independent error terms is taking their 2-norm; the worst-case stack is their 1-norm. When the customer asks why the budget grew from 12 to 19 units between charts, the honest answer may be that somebody switched norms, not that the hardware got worse.
Control and structural gain margins. The induced 2-norm of a MIMO transfer matrix at a given frequency is its worst-case gain at that frequency; the H∞ in H-infinity control is the peak of that curve — worst-case gain over every input direction and every frequency at once.

How it works

Picture the unit ball — every point with norm 1 — in the plane: the 1-norm gives a diamond, the 2-norm a circle, the ∞-norm a square. Same space, three different definitions of "distance 1," and the diamond fits inside the circle inside the square. That containment is the fixed ordering ‖x‖_∞ ≤ ‖x‖₂ ≤ ‖x‖₁. Concrete case, x = (3, −4, 12): ∞-norm 12, 2-norm 13, 1-norm 19. All legitimate sizes of the same vector; they differ by nearly a factor of 1.6 before anyone has made an error.

The norms bound each other both ways ^[1]:

‖x‖_∞ ≤ ‖x‖₂ ≤ ‖x‖₁ ≤ √n·‖x‖₂ ≤ n·‖x‖_∞

Mathematicians summarize this as "in finite dimensions, all norms are equivalent." Treat that phrase with suspicion. The constants grow with dimension: on a million-sample record, √n = 1000, and a bound that slack is not an engineering statement.

The choice of norm is a choice about which errors you tolerate. Take a 100-channel residual. Case one: every channel off by 0.1 — ‖r‖₂ = 1.0, ‖r‖_∞ = 0.1. Case two: 99 channels perfect, one channel off by 1.0 — ‖r‖₂ = 1.0 again, but ‖r‖_∞ = 1.0. Identical 2-norms, completely different situations. The 2-norm rewards spreading error around and will happily bury one dead channel in 99 good ones; the ∞-norm exists to prevent exactly that. This is why least-squares fits (2-norm) look clean while their worst point violates spec, why equiripple filter design minimizes the ∞-norm of the passband error instead, and why L1 fitting shrugs off outliers that would drag a least-squares line sideways.

Matrix norms add two traps of their own:

Frobenius is not the 2-norm. For A = |1 2; 3 4|: ‖A‖₁ = 6, ‖A‖_∞ = 7, ‖A‖₂ ≈ 5.465, ‖A‖_F = √30 ≈ 5.477. Close here, not in general — the gap obeys ‖A‖₂ ≤ ‖A‖_F ≤ √r·‖A‖₂ for rank r ^[2]. The identity matrix says it plainly: ‖I‖₂ = 1 (a wire with gain 1) but ‖I‖_F = √n. If the number is supposed to mean gain, use an induced norm; Frobenius measures total content, not amplification.
Norms cascade; that's the point. Submultiplicativity, ‖A·B‖ ≤ ‖A‖·‖B‖ ^[2], is what lets you bound a chain of stages by the product of their individual gains. It also powers the most consequential norm formula in numerical work: the condition number κ(A) = ‖A‖·‖A⁻¹‖, which bounds how much relative error in the data can grow into relative error in the solution. For the 2×2 above, κ₂ ≈ 14.9 — mild. At κ ≈ 10⁸, half your double-precision digits are spoken for before the algorithm makes its first mistake.

The recurring mistake is quoting a norm naked. "Residual = 0.003" means nothing without which norm and relative to what — ‖r‖₂/‖b‖₂ and ‖r‖_∞ can tell opposite stories about the same solve, and a review slide that compares this month's 2-norm against last month's ∞-norm is comparing a circle to a square.

History

The Euclidean length is as old as geometry; what the twentieth century added was the realization that length is a choice. The double-bar notation ‖x‖ traces to 1908, when Erhard Schmidt — of Gram–Schmidt — and Maurice Fréchet brought geometric language into infinite-dimensional sequence space, wrote the norm in its modern dress, and stated its triangle inequality ^[3]^[4]. The full p-norm family was legitimized earlier than that in one sense: the triangle inequality it needs is Minkowski's inequality, named for Hermann Minkowski, whose Geometrie der Zahlen began appearing in 1896 ^[5]^[6]. Frigyes Riesz built the L^p function spaces on that footing in 1910, opening the study of normed spaces that are not inner-product spaces ^[7]^[8].

The axioms as we teach them arrived in a burst: Stefan Banach set them down in his 1920 dissertation, and Hans Hahn and Norbert Wiener each published the general normed space independently within about two years — the idea, as one history of the period puts it, was in the air ^[3]^[9]^[10]. The complete spaces took Banach's name.

Matrix norms went from pure theory to shop-floor tool in 1948, when Alan Turing — fresh from wartime computing and thinking hard about what automatic machines would do to rounding error — published "Rounding-off errors in matrix processes," defining matrix norms for error analysis and coining the term condition number ^[11]^[12]. The entrywise root-sum-square norm had meanwhile picked up the name of Ferdinand Georg Frobenius ^[2]^[13]. Turing's framing — measure the problem's sensitivity with a norm before you blame the algorithm — is still the first move in every numerical post-mortem.

Related tools

/tools/rms-peak — RMS is the 2-norm of the sample record divided by √n
/tools/crest-factor — the ∞-norm-to-RMS ratio that flags pulsed and distorted waveforms
/tools/snr-enob — signal and noise powers are squared 2-norms; their ratio sets your usable bits
/tools/db-voltage-ratio — the dB scale most norm ratios end up quoted on

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →