Confidence intervals

The formula

The general pattern for any confidence interval:

estimate ± (critical value) · (standard error of the estimate)

Reading: take your best point estimate, then add and subtract a margin scaled to how noisy that estimate is.

For the mean of n measurements with unknown population sigma — the case you actually have on the bench ^[5]:

x̄ ± t(α/2, n−1) · s / √n

Reading: the sample mean, plus or minus a t-multiplier (looked up at your confidence level with n−1 degrees of freedom) times the sample standard deviation divided by root-n. The √n is why four times the data buys you half the interval.

For large samples, t collapses to the normal z-values:

90%: z = 1.645 95%: z = 1.960 99%: z = 2.576

For a proportion (pass/fail data, k failures in n trials, p̂ = k/n), the quick version:

p̂ ± z · √( p̂(1−p̂)/n )

Reading: same pattern, binomial standard error. This one has sharp edges at small n and extreme p̂ — see below.

For a variance, the interval comes from the chi-square distribution and is not symmetric:

(n−1)s² / χ²(α/2, n−1) ≤ σ² ≤ (n−1)s² / χ²(1−α/2, n−1)

Reading: variance estimates from small samples are lopsided — the upper bound sits much farther from s² than the lower one. With n = 10, the 95% interval for σ² runs from 9s²/19.02 up to 9s²/2.70 — the upper limit is more than three times the point estimate.

One-sided bounds use the same machinery with all the risk stacked on one side:

lower 95% bound on the mean: x̄ − t(0.05, n−1) · s / √n

Reading: when only one direction can hurt you — minimum strength, minimum margin, maximum leak rate — put the entire 5% miss probability on that side. A one-sided 95% bound is tighter than the corresponding side of a two-sided 95% interval, and it is the correct form for a requirement written as "shall exceed."

Sample-size planning inverts the interval. To hold the half-width to E:

n ≈ ( z · s / E )²

Reading: decide the margin you can live with before the test, plug in your best guess at the scatter, and the formula tells you how many runs to budget. Halving E costs four times the samples; that trade is worth knowing before the test readiness review, not after.

Where you meet it

The bench. You take ten readings of a voltage reference against the calibrator and report the offset. Ten readings, s = 2 mV: the 95% interval is x̄ ± 2.262 · 2/√10 = x̄ ± 1.43 mV. Anyone who quotes that offset to four digits without the ±1.43 is writing fiction.
The test stand. Qual report says mean thrust from six firings. Six firings is five degrees of freedom, and t(0.975, 5) is 2.571 — the margin is wide, and the review board should see it wide, not dressed up with a z-value that assumes infinite data.
Reliability demonstration. Thirty units through vibration, zero failures. Zero failures does not mean zero failure rate. The rule of three gives the 95% upper bound on the failure probability as roughly 3/n — for n = 30, about 10% (exact: 9.5%). That number belongs in the report next to the "0 failures" cell.
Regression slopes. Every calibration curve fit produces a slope with a confidence interval on it. If the interval on a sensitivity coefficient includes zero, the channel is not measuring what you think it is.

How it works

The confidence level describes the procedure, not the one interval in front of you. Run the experiment two hundred times, compute a 95% interval each time, and about 190 of those intervals will contain the true value — the other ten will miss, and nothing in the data tells you which kind you're holding. Saying "there's a 95% probability the true mean is in this interval" is the single most common misstatement in test reports; the true mean is a fixed number, not a random one, and it's either in your interval or it isn't ^[1]. What you're buying is a procedure with a known long-run miss rate.

The distinction matters in practice, not just in seminar. If a program makes a hundred go/no-go calls per year off 95% intervals, five of those calls rest on intervals that missed. The discipline the interval imposes is not certainty about any one number — it is a bounded, budgeted rate of being wrong across the whole campaign. That is a claim an engineer can defend in front of a customer, which is more than a bare point estimate offers.

The t-multiplier exists because with small samples you're estimating sigma from the same data you're estimating the mean from, and that double duty costs you. At n = 5 (4 degrees of freedom) the 95% multiplier is 2.776 versus 1.960 for infinite data — 42% wider. By n = 30 it's 2.045 and the penalty has mostly evaporated. Using z when you should use t is the classic way to publish intervals that are too narrow and to fail your own error budget later.

Width behaves predictably, and the behavior is worth internalizing:

Width scales with s — noisier process, wider interval, no way around it except fixing the process.
Width scales with 1/√n — diminishing returns; going from 10 samples to 40 halves the margin, going from 40 to 90 shaves another 33%.
Width grows with confidence level — a 99% interval is roughly 31% wider than a 95% interval (2.576/1.960) at large n, and worse at small n.
Width says nothing about bias. A miscalibrated instrument produces a beautifully tight interval centered on the wrong number. Confidence intervals bound random scatter only; systematic error walks right through them.

The gotchas that actually bite:

Independence is the load-bearing assumption. The √n improvement assumes each reading carries fresh information. Sample a drifting sensor at 1 kHz for one second and you have thousands of readings but nowhere near thousands of independent ones — autocorrelation quietly shrinks your effective n, and the interval you compute is far too tight. This is the most common way a "±0.01% measurement" turns out to be off by ten times its stated bound.
Confidence ≠ prediction ≠ tolerance. A confidence interval bounds the mean. A prediction interval bounds the next single reading — always wider. A tolerance interval bounds a fraction of the whole population. Specifying acceptance limits from a confidence interval on the mean, when the requirement was on individual units, is a documented way to ship hardware that fails in the field.
The Wald proportion interval fails where you need it most. At p̂ near 0 or 1 — exactly where reliability work lives — the p̂ ± z√(p̂(1−p̂)/n) formula gives intervals of zero width or ones that spill past 0 and 1. Use the Clopper–Pearson exact method or the Wilson score interval when failures are rare.
Twenty intervals, one liar. At 95%, one interval in twenty misses by construction. A dashboard with forty monitored parameters will average two "statistically significant" excursions from pure noise. Plan the review board conversation accordingly.
An interval without its pedigree is decoration. "±0.5" means nothing until the report states the confidence level, the n, and whether the bound is one-sided or two-sided. A 68% interval (one sigma) and a 99% interval on the same data differ by a factor of 2.6, and readers routinely assume whichever one flatters the result.
Normality matters less than people fear, for means. The central limit theorem makes x̄ behave near-normally even from skewed data once n gets past a couple dozen. But the variance interval and any tolerance interval lean hard on the normality assumption — check the distribution before quoting those.

History

The small-sample half of the story starts in a brewery. William Sealy Gosset joined Arthur Guinness & Son in Dublin in 1899 and spent his career doing quality-control statistics on barley and beer, where an "experiment" might be four fields or eight batches — nowhere near the large samples the theory of the day assumed ^[2]^[3]. He worked out the distribution of the mean for small samples partly by mathematics and partly by drawing random samples by hand — an early Monte Carlo run ^[2]. Guinness, burned by an earlier employee publication that leaked useful trade information, barred its brewers from publishing under their own names, so the 1908 Biometrika paper "The Probable Error of a Mean" appeared under the pseudonym "Student" ^[3]. Ronald Fisher later recast and extended the result, and traded more than 150 letters with Gosset over two decades ^[3]. That is why the multiplier in every small-sample interval is called Student's t and not Gosset's.

The confidence interval itself — the formal machinery and the "95% of intervals cover the truth" interpretation — belongs to Jerzy Neyman. By his own account the problem arrived around 1930 as a student's question about stating the precision of an estimated regression coefficient ^[1]. Working at University College London from 1934, Neyman published the first thorough general theory in his 1937 Royal Society paper, "Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability" ^[1]^[4]. Fisher had published his rival "fiducial" intervals around the same period, and while the two frameworks produced matching answers on several standard problems, the men did not produce matching temperaments — relations between Fisher and the Neyman–Pearson camp broke down even while they shared a building at UCL ^[1]^[4]. Neyman's frequentist formulation is the one engineering practice settled on, and the one every table of t-values quietly assumes.

Related tools

/tools/bearing-life-l10 — L10 life is a statistical estimate from scattered failure data; honest ratings carry confidence bounds.
/tools/arrhenius-rate — accelerated-life extrapolations inherit the uncertainty of the fitted activation energy, and it belongs in an interval.
/tools/strain-gauge-bridge — bridge measurements averaged over repeats are exactly the x̄ ± t·s/√n case above.

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →