Chi-square tests

The formula

The whole family runs on one number. You compare what you counted against what a model says you should have counted:

chi2 = sum over all categories of (O - E)² / E

Reading it: for every bucket, take the observed count O minus the expected count E, square it so overshoots and undershoots both count against you, and divide by E so a miss of 5 on an expected-1000 bucket barely registers while the same miss of 5 on an expected-8 bucket screams. Add up the buckets. Big total means the data and the model disagree.

Two jobs, two ways to get E and two ways to count degrees of freedom.

Goodness-of-fit (does one set of counts match a claimed distribution) ^[6]:

E_i = N · p_i          df = k - 1 - m

N is your sample size, p_i the probability the model assigns to bucket i, k the number of buckets, and m the number of parameters you estimated from the data to build the model. Estimate nothing (probabilities handed to you) and df = k - 1 ^[6].

Test of independence / contingency table (does one categorical variable depend on another):

E_ij = (row_i total · col_j total) / N          df = (r - 1)·(c - 1)

Reading it: if supplier and failure mode were truly independent, each cell's expected count is just its row share times its column share of the grand total. The df counts how many cells you're free to fill before the margins force the rest.

You then compare chi2 against the chi-square distribution with that df. Exceed the critical value for your chosen alpha and you reject the null.

Where you meet it

• Incoming-inspection review board. Two suppliers ship the same connector. Supplier A had 10 failures in 100 units, Supplier B had 30 in 100. Is that a real supplier difference or a bad week? Build the 2x2 table [[90,10],[70,30]], and the test of independence returns chi2 = 12.5 on 1 degree of freedom, p ≈ 0.0004. That's not luck. The failure mode depends on the supplier, and now you have a number to put in front of the board.

• Bench: is my defect count Poisson? You're modeling solder-joint defects per board as a Poisson process for a yield prediction. Bin the observed boards by defect count, compute expected counts from the fitted Poisson, and run goodness-of-fit. If it fails, your defects are clustering — a fixture or reflow-profile problem — and the "random independent defect" assumption underneath your yield math is wrong.

• RF test stand: is the noise really Gaussian? Before you trust a Gaussian tail estimate for a rare bit-error event, histogram the measured noise samples, bin them, and test the fit against a normal with the mean and variance you estimated. Estimating those two parameters costs you two degrees of freedom (m = 2), which the formula above already accounts for.

• Reliability: does failure mode track a lot? Cross-tabulate failure modes against manufacturing lots. A significant chi-square says the failure mode isn't spread evenly — some lots carry a distinct signature, which points root-cause analysis at a process change instead of a random population.

How it works

The statistic works because, under the null hypothesis and with enough data, that sum of squared standardized deviations settles into a known shape: the chi-square distribution with the matching degrees of freedom. That's an asymptotic result — it's the large-sample limit, not an exact truth for small counts. Every gotcha below comes from leaning on that approximation when the counts are too thin to support it — except the first one, which is about where the counts came from in the first place.

Independent units, counted once. The math assumes every tally in the table comes from an independent observational unit, and that each unit lands in exactly one cell. This is the assumption engineers actually break: counting three defects on the same board as three separate entries (they share a board, they're not independent), pooling serially-correlated test-stand runs as if each were a fresh draw, or lumping per-lot data where units within a lot rise and fall together. Clustered or correlated counts inflate chi2 and your false-alarm rate — the test will "find" supplier differences that are really lot-to-lot correlation. And if your 2x2 table is paired data — the same units measured before and after a rework, pass/fail on two testers — the independence test is the wrong tool entirely; that's McNemar's test.

The expected-count rule. The working rule: every expected count should be about 5 or more. For a 2x2 table people want all four cells at 5+; for bigger tables the common guidance is 5+ in at least 80% of cells and no cell with an expected count of zero. Note that's expected, not observed — an observed cell of 0 is fine, an expected cell near 0 is not, because it sits in the denominator and blows a tiny deviation up into a huge contribution. When cells are too sparse, either pool categories or switch to Fisher's exact test, which computes the probability directly instead of approximating it.

The mistake people make. The number one error: running the test on percentages or rates instead of raw counts. The whole machinery is built on counts. Feed it "10% vs 30%" without the sample sizes and you've thrown away the only thing that makes the deviation meaningful — 10% of 20 units and 10% of 20,000 units are worlds apart in evidence. Always feed it integer counts.

Yates' correction. For a 2x2 table, some analysts subtract 0.5 from each |O - E| before squaring to compensate for approximating discrete counts with a continuous distribution. In the connector example it drops chi2 from 12.5 to 11.28 — more conservative, larger p-value. It matters most exactly when the counts are marginal, and it's debated whether it over-corrects. Know that it exists so a reviewer's "did you apply the continuity correction?" doesn't catch you flat.

What it does and doesn't tell you. Chi-square tells you whether observed and expected disagree. It does not tell you the direction or size of the effect, and it does not tell you which cell drove the result — for that, look at the standardized residuals (O - E)/√E per cell. A large table with huge N will flag differences too small to care about, so pair the p-value with an effect-size number (Cramér's V) before you spend money on the finding. And it's a test of association, never of cause: supplier and failure mode being dependent doesn't say the supplier caused the failures — it says something they don't share is real.

History

Karl Pearson published the goodness-of-fit test in 1900, and it's fair to call it one of the load-bearing walls of modern statistics — the first general method for asking whether a model fit the data at all ^[1][2]. In 1904 he extended the same idea to contingency tables, testing whether two categorical variables were independent, in his Drapers' Company memoir on the theory of contingency ^[2][3].

Then came the argument that fixed the tool. Pearson counted the degrees of freedom for a contingency table one way; R.A. Fisher, working through it in papers around 1922 and 1924, showed that the marginal totals you're forced to hold fixed eat into the degrees of freedom, and the correct count is (r-1)(c-1), not what Pearson had used ^[4]. Fisher's phrasing — that fitting to the margins "inhibits" the free variation — is essentially where the working notion of degrees of freedom in these tests comes from. Pearson rebutted in Biometrika, and the exchange hardened into the Fisher–Pearson controversy ^[4]. Fisher was right about the arithmetic, and every stats package computes the table's degrees of freedom his way today.

The last common piece is Frank Yates, who in 1934 published the continuity correction for 2x2 tables — subtract 0.5 before squaring — in a paper on contingency tables with small numbers, in the Journal of the Royal Statistical Society ^[5]. It's the fix you reach for when the counts are small enough that the smooth chi-square curve is a shaky stand-in for the lumpy reality of integer counts.

Sources

1. https://en.wikipedia.org/wiki/Chi-squared_test
2. https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test
3. Pearson, K. (1904). "On the theory of contingency and its relation to association and normal correlation." Drapers' Company Research Memoirs, Biometric Series I. https://archive.org/details/cu31924003064833
4. https://arxiv.org/pdf/0808.4032
5. https://en.wikipedia.org/wiki/Yates%27s_correction_for_continuity
6. https://www.itl.nist.gov/div898/handbook/eda/section3/eda35f.htm