Design of experiments

The formula

The workhorse model behind a two-level factorial design:

y = μ + (E_A/2)·x_A + (E_B/2)·x_B + (E_AB/2)·x_A·x_B + ... + ε

Reading: the response y is a baseline μ plus a contribution from each factor (coded x = −1 at its low setting, +1 at its high setting), plus interaction terms, plus noise ε. Fit this and the coefficients are half the effects.

E_A = ȳ(A at +1) − ȳ(A at −1)

Reading: a main effect is just the average response with the factor high minus the average with it low — averaged over every setting of everything else. That averaging is the whole trick.

N = 2^k (full factorial) N = 2^(k−p) (fractional factorial)

Reading: k factors at two levels each need 2^k runs to cover every combination. Seven factors is 128 runs. A fractional design gives up p factors' worth of combinations and gets the run count down — 2^(7−4) = 8 runs can still screen all seven factors, at a price paid in confounding (below).

SE(effect) = 2σ / √N

Reading: the standard error of any estimated effect shrinks with the square root of the total run count, because every run contributes to every effect estimate. Quadruple the runs, and effects half as large become distinguishable from noise — sensitivity improves only with the square root of run count. Note what else this says: the noise floor σ of your measurement chain sets the smallest effect you can ever resolve.

Where you meet it

Vibration or thermal test planning. You suspect fixture preload, sweep rate, and control-accel location all move the resonance you keep chasing. Three factors, two levels, 2³ = 8 runs — one shift on the shaker — and you know which one matters and whether any pair gang up on you.
The RF bench. Tuning a matching network or a solder-reflow profile for an RF assembly: temperature, dwell, paste volume, pad finish. A 12-run Plackett–Burman screens up to 11 candidate factors and tells you which three deserve the follow-up study ^[7].
The review board. Test readiness reviews are where run counts go to die. "Why 16 runs and not 6?" has an answer: 16 runs is a resolution-V half-fraction of five factors (2^(5−1)), which cleanly estimates all five main effects and all ten two-factor interactions. Six runs estimates nothing cleanly. DOE is how you defend a test matrix with arithmetic instead of adjectives.
Anywhere a process drifts. Screening a machining, potting, bonding, or plating process that went out of family — DOE finds the knob that actually moved, instead of the knob everyone remembers touching.

How it works

The instinct DOE replaces is one-factor-at-a-time (OFAT): hold everything, move one knob, record, repeat. OFAT feels careful and is quietly wasteful twice over. First, each OFAT comparison uses only two runs' worth of data per factor, while a factorial design reuses every run in every effect estimate — Fisher's people called it hidden replication. In an 8-run factorial, each main effect is a comparison of two 4-run averages; OFAT would need far more runs to buy the same precision. Second, and worse, OFAT is structurally blind to interactions. If sweep rate only matters at high preload, OFAT at low preload reports "sweep rate: no effect," and that conclusion is wrong, not just imprecise.

Three principles carry the method, and they are Fisher's originals ^[1]^[2]:

Randomization. Run order is drawn at random. Ovens warm through the day, operators tire, humidity climbs. Randomizing run order turns those drifts into noise instead of letting them masquerade as a factor. The classic self-inflicted wound is running all the "high temperature" trials Monday and all the "low" ones Tuesday — congratulations, temperature is now perfectly confounded with whatever else changed overnight, and no analysis can pull them apart afterward.
Replication. Repeat runs give you an honest estimate of σ, so "the effect is bigger than the noise" is a computed statement, not a vibe. Even a few center-point repeats do real work.
Blocking. When you know about a nuisance grouping — two test stands, three material lots, morning versus night shift — you don't randomize it away, you build the design so each block contains a balanced slice of the matrix. Block, then randomize within blocks. The slogan worth memorizing: block what you can, randomize what you can't.

Fractional designs pay for their small run counts with aliasing: some effects share an estimate and cannot be separated. The 2^(7−4) eight-run screen aliases main effects with two-factor interactions (resolution III); a resolution-IV design keeps main effects clean but tangles two-factor interactions with each other; resolution V untangles both. The gamble underneath is effect sparsity — out of k(k−1)/2 possible two-factor interactions (21 of them for seven factors), only a few are usually real. Usually. When a screening design flags a factor, the honest statement is "this factor or its aliases," and the follow-up runs exist to break the tie.

Limits of validity: a two-level design fits a straight line between the levels, so it cannot see curvature — add center points to check, and move to response-surface designs ^[8] when you're optimizing rather than screening. And no design rescues a measurement system that can't resolve the effect; if 2σ/√N is bigger than the change you care about, fix the instrumentation or grow N before burning test-stand time.

History

DOE was invented in the mud. Ronald Fisher — who had coined the term "variance" in a 1918 genetics paper ^[2]^[3] — joined Rothamsted Experimental Station in 1919 to make sense of decades of agricultural field trials ^[2]^[3]. Fields are a nightmare of nuisance variation: fertility gradients, drainage, last year's crop. Fisher's answer was to stop fighting the variation and design around it — randomize plot assignments, replicate, and block by field position — and to invent the analysis of variance to split the results back apart ^[3]. He put the working methods in Statistical Methods for Research Workers (1925) ^[2]^[3], laid out the field-trial principles in "The Arrangement of Field Experiments" (1926) ^[1]^[4], and wrote the definitive book, The Design of Experiments, in 1935 — opening with the famous test of whether a colleague could taste if milk went into the cup before the tea ^[1]^[2].

Frank Yates arrived at Rothamsted as Fisher's assistant statistician in 1931 and took over the statistics department in 1933 ^[5]^[6]; his 1937 monograph on factorial experiments, and the hand-computation algorithm that bears his name, made multi-factor designs practical in the pencil era ^[5]^[6]. In 1946, Robin Plackett and Peter Burman published the screening designs that carry their names — run counts in multiples of four, up to 11 factors in 12 runs ^[1]^[7]. The move from farm to factory finished with George Box, a chemist-turned-statistician at Imperial Chemical Industries, whose 1951 paper with K. B. Wilson, "On the Experimental Attainment of Optimum Conditions," founded response surface methodology — DOE aimed not at which factors matter but at where the optimum sits ^[8]^[9].

Related tools

/tools/strain-gauge-bridge — the measurement channel on most structural test matrices; its output resolution feeds straight into the σ in SE(effect) = 2σ/√N
/tools/snr-enob — quantifies the noise floor of the acquisition chain, which sets the smallest effect a design of any size can resolve
/tools/adc-resolution — quantization step size; if the effect you're hunting is under one LSB, no run count saves you

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →