Analysis of variance

The formula

The whole method rests on one accounting identity. For k groups and N total observations:

SS_total = SS_between + SS_within

Reading: total squared scatter about the grand mean splits exactly into the part explained by group-to-group differences and the part left over inside the groups. Nothing is lost; nothing is counted twice.

The two pieces, written out:

SS_between = Σᵢ nᵢ · (x̄ᵢ − x̄)² (k − 1 degrees of freedom) SS_within = Σᵢ Σⱼ (xᵢⱼ − x̄ᵢ)² (N − k degrees of freedom)

Reading: SS_between measures how far each group mean sits from the grand mean, weighted by group size. SS_within is pure repeatability — how much each reading wanders from its own group's mean.

Divide each by its degrees of freedom to get mean squares, then take the ratio ^[1]^[6]:

F = MS_between / MS_within = [ SS_between/(k−1) ] / [ SS_within/(N−k) ]

Reading: if the group means are truly equal, both mean squares estimate the same noise variance σ² and F sits near 1. If a factor is doing something, MS_between picks up an extra term — for equal group sizes n, its expected value is σ² + n·Σ αᵢ²/(k−1), where αᵢ are the true group offsets — and F climbs. Compare against the F-distribution with (k−1, N−k) degrees of freedom.

Two-way ANOVA runs the same books with more accounts. For factors A (a levels) and B (b levels) with replication:

SS_total = SS_A + SS_B + SS_AB + SS_error

Reading: the interaction term SS_AB, with (a−1)(b−1) degrees of freedom, is the part of the scatter that only appears when the two factors act together — the effect of temperature that depends on which vendor's part you bolted in. Each effect gets its own F ratio against MS_error.

Where you meet it

The test stand. Three lots of the same component fired five times each. The lot means differ by a few tenths — but shot-to-shot scatter is a few tenths too. ANOVA is the tool that answers "lot effect or noise?" before anyone writes a discrepancy report against the supplier.
Gauge R&R on the bench. The standard gauge repeatability-and-reproducibility study is a two-way ANOVA in work clothes: parts crossed with operators, the interaction term telling you whether certain operators fight certain parts. The output — how much of the observed variance belongs to the instrument versus the hardware — is a straight read of the variance components.
DOE debrief. After a screening experiment on a process — cure temperature, clamp pressure, primer batch — the ANOVA table is the deliverable. It ranks which knobs moved the response and which lines of the test matrix were spent measuring nothing.
Review board. Qual data taken across four test days shows a drift. Somebody asks whether day matters. Treating "day" as a factor and running the one-way table settles whether the environment contaminated the campaign or the eyeball found a pattern in static.

How it works

The logic is signal-to-noise, formalized. MS_within is a pooled estimate of the noise floor built only from scatter inside groups — the group means can differ all they want without touching it. MS_between estimates the same noise floor plus whatever the factor contributes. Their ratio is therefore a calibrated question: does the factor rise above the floor?

Small worked case: three fixtures, five readings each, means 12.18, 12.80, 12.40. SS_between = 0.988 on 2 degrees of freedom; SS_within = 0.348 on 12. So F = 0.494/0.029 = 17.0 against a 5% critical value of F(0.05; 2, 12) = 3.89 — the fixture effect is real (p ≈ 0.0003), even though every individual reading overlaps the ranges of the other fixtures. That overlap is exactly why eyeballing raw data fails and the partition doesn't.

Why not just run t-tests between every pair? Because with k groups that's k(k−1)/2 comparisons, and at 5% each the false-alarm rate compounds — six pairwise tests carry roughly a one-in-four chance of at least one bogus "significant" result on pure noise. ANOVA asks the question once, at the alpha you actually chose. For exactly two groups the two tools agree perfectly: F = t².

The gotchas that matter on real hardware:

A significant F says "at least one mean differs" — not which one. Follow up with a multiple-comparison procedure (Tukey's HSD is the workhorse) before naming a culprit in the report. Skipping straight from the ANOVA table to "lot 2 is bad" is the most common misread of the method.
Independence is the assumption that kills. Readings taken seconds apart on a drifting rig are not independent, and the F-test doesn't know that — it will happily convert autocorrelated drift into a phantom "significant" factor. Randomize the run order. That isn't statistical politeness; it's the mechanism that converts unknown drift into honest noise instead of a fake effect.
Equal variances and normality are assumed ^[1], but they don't bite equally. The F-test tolerates moderately non-normal residuals, especially with balanced groups. Strongly unequal group variances plus unequal group sizes is the dangerous combination. Check with Levene's test rather than Bartlett's — Bartlett's is itself so sensitive to non-normality that it flags the wrong crime ^[7].
Fixed versus random effects change the arithmetic. Three specific fixtures you own are a fixed effect. Three operators drawn from a pool of twenty are a random effect, and in two-way and nested designs the correct F denominator changes. Gauge R&R software gets this right; a hand-rolled spreadsheet frequently doesn't.
Statistical significance is not engineering significance. With enough replicates, ANOVA will flag a lot-to-lot difference of half a millivolt as real. Real and relevant are separate findings; the second one is your job, not the table's.

History

The word "variance" itself is younger than powered flight. Ronald Fisher coined it in a 1918 paper on Mendelian inheritance, where he also proposed its formal analysis ^[1]^[2]. The next year he turned down a post at Karl Pearson's Galton Laboratory in London and took a temporary statistician's job at Rothamsted Experimental Station instead ^[2]^[3] — an agricultural research farm sitting on decades of accumulated crop records. Field data is brutally noisy: soil, weather, and seed all vary, and the effects worth finding are buried in that scatter. It was exactly the right problem, and in 1921 Fisher published Studies in Crop Variation I, the first working application of the analysis of variance ^[1]^[2]. The follow-on paper, written with his assistant Winifred Mackenzie, became the template later ANOVA work followed ^[1]^[2].

Fisher's 1925 handbook Statistical Methods for Research Workers carried the method out of agronomy and into general laboratory practice ^[1]^[3], and his 1935 The Design of Experiments welded ANOVA to the ideas it can't function without — randomization, replication, and blocking ^[2]^[3]. The letter "F" isn't Fisher's, though. George Snedecor, who built the statistical laboratory at Iowa State, published the first extensive tables of the variance-ratio distribution in his 1934 monograph on analysis of variance and covariance, and attached the F to the distribution in Fisher's honor ^[4]^[5]. Farm-field statistics, in other words, is the direct ancestor of every DOE package on a test engineer's laptop — the noise just changed from weather to instrumentation.

Related tools

/tools/rms-peak — the RMS of a zero-mean signal is its standard deviation; ANOVA partitions the square of that same quantity.
/tools/snr-enob — signal-to-noise is a variance ratio; the F statistic is the hypothesis-testing version of the same instinct.
/tools/thermal-noise-floor — the physical origin of the MS_within your electrical measurements can never get below.
/tools/strain-gauge-bridge — repeat bridge measurements across setups and operators are the raw material of a gauge R&R ANOVA.

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →