Outlier detection

The formula

G = max|xᵢ − x̄| / s

Reading: the Grubbs statistic is the most extreme point's distance from the sample mean, measured in sample standard deviations ^[1]^[2]. Big G means the worst point sits farther out than its own dataset says it should.

Reject if G > ((n−1)/√n) · √( t² / (n−2+t²) ) , t = t-critical at α/(2n), n−2 d.o.f.

Reading: the cutoff is not a fixed "3 sigma" — it grows with sample size because the biggest of n honest points naturally drifts out as n grows. Two-sided at α = 0.05: the cutoff is 2.13 for n = 8, 2.29 for n = 10, 2.71 for n = 20 ^[1]^[2].

Chauvenet: reject xᵢ if n · P(|z| > |xᵢ − x̄|/s) < 1/2

Reading: compute how improbable the deviation is under a normal fit, multiply by the sample count; if you'd expect fewer than half an observation that far out, throw it out ^[5]. For n = 10 that works out to about 1.96·s; for n = 50, 2.58·s; for n = 1000, 3.48·s.

Q = gap / range = |x_suspect − x_nearest| / (x_max − x_min)

Reading: the Dixon Q statistic needs no mean and no standard deviation — sort the data, take the suspect point's gap to its nearest neighbor, divide by the full spread ^[9]. Built for the bench: n = 3 to 10, one suspect point, pencil arithmetic. Two-tailed 95% critical values run from 0.970 at n = 3 down to 0.466 at n = 10 ^[9].

Where you meet it

The static test stand. Thirty lug pull tests, twenty-nine cluster around 4,200 lbf, one breaks at 2,600. Before that number drags the reported minimum down, somebody has to decide whether it stays in the population. Grubbs on the thirty points is the calculation the stress report cites; the fixture inspection that found the misaligned clevis is the reason the exclusion survives review.
The cal lab. Ten repeat readings against a reference standard, one flyer during the run. A calibration certificate that quietly averages nine of ten readings is a finding waiting to happen; a certificate that cites ASTM E178 and shows the Grubbs computation is not ^[12].
The chemistry bench. Four titrations, one obviously off. With n = 4 there is no meaningful standard deviation to lean on, which is exactly the situation Dixon's Q was built for — gap over range, compare to the table, done ^[9].
The data review board. An interlab round-robin where one lab's mean sits away from the other seven. The outlier tests give the board a common, pre-agreed rule, which is their real job: moving the exclusion argument from "that point looks wrong" to a procedure everyone signed up to before the data existed.

How it works

Every classical outlier test answers the same question: assuming the data came from one normal population, how surprising is the most extreme point? The machinery differs — Grubbs standardizes against the sample sigma, Chauvenet against an expected count, Dixon against the raw spread — but all three are significance tests on the worst observation, with critical values that account for the fact that you went looking for the worst one. That last part is the whole trick. The biggest of twenty honest normal samples routinely lands near two sigma; flagging it at a fixed 2σ line would reject good data constantly. The Grubbs cutoff climbing from 2.13 at n = 8 to 2.71 at n = 20 is the correction for having searched.

Run NIST's worked example to see the scale ^[1]. Eight uranium-content measurements: 199.31, 199.53, 200.19, 200.82, 201.92, 201.95, 202.18, 245.57. Mean 206.43, standard deviation 15.85, so G = (245.57 − 206.43)/15.85 = 2.469, against a one-sided 5% critical value of 2.032. The 245.57 goes — and note that it took a point sitting 45 units above its neighbors to clear a cutoff of two-and-change, because the outlier itself inflated the standard deviation it was being measured against.

Dixon at the bench, same idea with less machinery. Five conductivity readings: 100.0, 100.2, 100.3, 100.5, 102.1. The suspect's gap to its nearest neighbor is 1.6; the range is 2.1; Q = 1.6/2.1 = 0.762, over the two-tailed 95% critical value of 0.710 for n = 5 ^[9]. The high reading flags without anyone computing a mean. Chauvenet on the textbook set 9, 10, 10, 10, 11, 50: the 50 sits 2.04 sigma out, the n = 6 threshold is 1.73 sigma, so it goes ^[5]. Note what the sigma already absorbed — that one point dragged the sample standard deviation to 16.3 on data whose honest scatter is well under 1, which is the inflation problem in miniature.

Choosing between the three is mostly a question of sample size and audience. Grubbs is the modern default and the one NIST's handbook recommends — proper significance level, exact critical values, extensions for two outliers ^[1]^[13]. Dixon earns its keep below about n = 10, or when a technician needs to screen dozens of small samples without a computer, which is exactly the use case ASTM E178 assigns it ^[9]^[12]. Chauvenet is the one you inherit from an old lab procedure; know how to read it, and know why the procedure should eventually be revised to name a significance level instead.

Two bookkeeping details that change answers. First, one-sided versus two-sided: if you would only ever exclude a high flyer (a contamination spike, a dropout reading pinned at rail), the one-sided critical value applies and it is lower — NIST's uranium example uses 2.032 one-sided where the two-sided value is 2.13 ^[1]. Decide which question you're asking before looking at the data, not after. Second, the significance level is per dataset, not per career: at α = 0.05, one clean dataset in twenty will hand you a false outlier, and a lab that runs the test on every batch will exclude good points on schedule unless the procedure says what happens next.

The rest of the traps, in the order they bite:

G has a hard ceiling. Algebraically, G can never exceed (n−1)/√n — that's 2.475 for n = 8. The NIST example's wildly aberrant point only reached 2.469, a hair under the ceiling. One consequence: for very small samples no point, however absurd, can look extreme enough, which is why Grubbs is not recommended below about n = 7 ^[2].
Masking. Two outliers on the same side prop up both the mean and the sigma, and each one hides the other — the test reports "no outlier" precisely because there are two. Dixon's ratios have their own version: a second high value sitting next to the suspect closes the gap in the numerator. This is why the standards allow variants that test pairs, and why "run Grubbs repeatedly until nothing flags" is not a licensed procedure — each deletion changes the distribution of the next statistic ^[2]^[9].
The normality assumption is load-bearing. All three tests calibrate "surprising" against the Gaussian tail. Feed them data that is legitimately heavy-tailed — vibration peaks, RF interference power, anything lognormal-ish — and they will flag real physics as blunders, forever. An outlier test is a test for measurement mistakes drawn from a process you otherwise trust, not a test of whether the tail is inconvenient.
Chauvenet trims by design. The 1/(2n) rule holds the expected false-rejection count near constant, so applied honestly it shaves roughly half a good point per dataset — and applied iteratively it keeps shaving. It survives in lab courses for its simplicity; NIST's guidance and modern practice route you to Grubbs or Dixon with an explicit significance level instead ^[1]^[13].
The mistake people make is treating a rejected H₀ as permission to delete. The defensible sequence is the one ASTM E178 codifies: flag the point statistically, then hunt for the assignable cause — the transposed digit, the loose BNC, the thermocouple that fell off. Statistical flag plus physical cause is an exclusion; statistical flag alone is, at most, a documented footnote with the analysis shown both ways ^[12]^[13]. Deleting the point that ruins your Cpk and citing Grubbs after the fact is the version that gets taken apart in the meeting.

History

The problem is older than the tests, and it started in astronomy, where every data point was a night at the telescope and one bad transit could poison a star catalog. Benjamin Peirce of Harvard published the first systematic rejection rule in the Astronomical Journal in 1852, and it got genuine production use — Peirce directed longitude work for the U.S. Coast Survey, and Survey computers applied his criterion as standard practice for decades ^[8]. William Chauvenet — who had helped found the Naval Academy in 1845 and would end his career as chancellor of Washington University in St. Louis — included Peirce's method in the second volume of his 1863 Manual of Spherical and Practical Astronomy, then offered the simpler expected-count shortcut that took his name instead of Peirce's ^[5]^[6]^[7]^[8]. The easier rule won the textbooks, a pattern engineers will recognize.

The modern tests arrived in one year, from two very different desks. Frank E. Grubbs — Alabama Polytechnic graduate, Ph.D. Michigan 1949 — spent his career at the Ballistic Research Laboratory at Aberdeen Proving Ground judging the ballistic consistency of projectiles and rockets, work where a wild round is either a manufacturing defect or a chart-reading error and it matters which. His 1950 paper in the Annals of Mathematical Statistics put outlier rejection on proper significance-test footing, and his 1969 Technometrics survey became the standard reference ^[2]^[3]^[4]. The same year, Wilfrid Dixon — Princeton Ph.D. under Samuel Wilks, later founder of UCLA's biostatistics program and of the BMDP statistical software — published his ratio tests in the same journal; the 1951 Dean and Dixon paper in Analytical Chemistry translated them for chemists, and Q = gap/range has owned the titration bench ever since ^[9]^[10]^[11]. ASTM's E178, which carries both men's tests, is where the whole lineage lands on a test engineer's desk today ^[12].

Related tools

/tools/snr-enob — SNR and effective bits assume the noise is well-behaved; one glitch sample in an ADC record is exactly the outlier these tests screen for
/tools/thermal-noise-floor — the noise floor defines what "normal scatter" means before any point can be called wild
/tools/rms-peak — the mean-and-sigma machinery the Grubbs statistic is built from, in signal clothing
/tools/strain-gauge-bridge — the channel producing most structural test data; a debonded gauge is the classic assignable cause behind a flagged point

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →