Bayes' theorem

The formula

The core equation, read right to left:

P(A|B) = P(B|A) · P(A) / P(B)

P(A|B) — the posterior: probability of A given that you observed B. This is the number you actually want.
P(B|A) — the likelihood: probability of seeing B if A is true. This is what a spec sheet or a test report gives you (detection rate, hit rate).
P(A) — the prior: how probable A was before you saw anything. The base rate.
P(B) — the evidence: how probable the observation B is overall, across every way it can happen.

That denominator almost never comes handed to you. You build it with the law of total probability — split B across A and not-A:

P(B) = P(B|A) · P(A) + P(B|¬A) · P(¬A)

Substitute it back and you get the form you'll actually key into a spreadsheet:

P(A|B) = [ P(B|A) · P(A) ] / [ P(B|A) · P(A) + P(B|¬A) · P(¬A) ]

There's a cleaner form for arguing about it in a review, the odds form:

posterior odds = prior odds · likelihood ratio

O(A|B) = O(A) · [ P(B|A) / P(B|¬A) ]

Reading: the evidence multiplies your prior odds by the likelihood ratio — how much more often the observation shows up when A is true than when it isn't. Strong evidence has a big ratio. Weak evidence has a ratio near 1 and barely moves the needle, no matter how confident the tool looks.

Where you meet it

BIT and the false-alarm review. A built-in test on a line-replaceable unit flags a fault. The question the program office actually asks: pull the box or not? The BIT vendor quotes a detection rate — probability the BIT trips when the box is truly bad. That's the likelihood, not the answer. Bayes converts it to the probability the box is bad given the trip, using the fleet's real failure rate as the prior. For a good box on a rarely-failing unit, that number is often shockingly low.

Screening on the test stand. Any go/no-go screen on a population that's mostly good — a leak check, a continuity sweep, an acceptance vibe test — has the same shape. Sensitivity and specificity live on the spec sheet; the fraction of units that are genuinely defective is the prior. Skip the prior and you'll scrap good hardware every time the screen sneezes.

Radar and comms detection. Every detector runs on two knobs: probability of detection and probability of false alarm. When a threshold crossing fires, "is that a target or noise?" is a Bayes question, and the target's prior probability of being present is doing quiet, heavy lifting in the answer.

Fault isolation in a fielded system. Several failure modes can throw the same error code. Bayes ranks the suspects: each candidate's prior failure rate times how likely it is to produce this symptom, normalized across all candidates. That's the ordering a good troubleshooting tree is implicitly encoding.

How it works

The whole theorem is a bookkeeping identity — it just enforces that P(A and B) is the same number whether you factor it as P(A|B)·P(B) or P(B|A)·P(A). Set those equal, divide, done. Nothing controversial in the algebra. The trouble is always in the inputs.

The mistake that eats engineers alive is confusing P(B|A) with P(A|B) — the prosecutor's fallacy in a lab coat. "The BIT is 99% accurate" gets heard as "99% chance the box is bad." Those are different conditionals and Bayes is the machine that keeps them straight.

Run the numbers. Say a BIT trips on 99% of truly bad boxes (great detection) and false-alarms on 5% of good boxes. Sounds like a solid tool. Now suppose only 1% of the boxes coming across the bench are actually bad:

P(bad | trip) = (0.99 · 0.01) / (0.99 · 0.01 + 0.05 · 0.99)
              = 0.0099 / (0.0099 + 0.0495)
              = 0.0099 / 0.0594
              ≈ 0.167

A BIT hit means the box is bad about 1 time in 6. Five of every six pulls are good hardware. Not because the tool is broken — because the thing you're hunting is rare and the false-alarm channel is fed by a huge pool of good units. Make the fault rarer, 0.1% instead of 1%, and the same BIT hit drops to under 2% confidence. This is the low base-rate trap, and it's exactly why a diagnostic that looks fantastic on paper generates a maintenance queue full of retest-OK.

The odds form makes the fix obvious. Here the likelihood ratio is 0.99 / 0.05 ≈ 20. Prior odds of a bad box were 1-to-99. Times 20 gives 20-to-99 ≈ 0.2, which is a probability of 1/6 — same answer, less arithmetic. If you want the posterior high, you need either a rarer false-alarm channel (bigger ratio) or a higher prior (test a population you already suspect).

Limits and gotchas worth pinning down:

The prior is not optional. Refusing to name a base rate doesn't make the answer prior-free; it silently sets the prior to something, usually wrong. Argue about the number in the open.
Priors have to be honest, not convenient. Garbage prior in, garbage posterior out. A fleet failure rate pulled from actual reliability data beats a guess, and a guess beats pretending the base rate is 50-50.
The events must be exhaustive. The total-probability denominator only balances if A and ¬A cover every case. Miss a failure mode and the normalization is off.
Independence assumptions bite. Chaining multiple tests as if their errors are independent will overstate your confidence when the tests share a common-mode failure — same bad reference, same noisy bus.
It updates; it doesn't originate. Today's posterior is tomorrow's prior when the next reading lands. That's the strength — sequential evidence compounds correctly — but it also means one bad prior propagates.

History

Thomas Bayes was a Presbyterian minister and mathematician, born in London around 1701 and elected a Fellow of the Royal Society in 1742 despite having no mathematics in print at the time ^[1]^[2]. He worked out his result — a way to reason backward from observed outcomes to the probability of their underlying cause — and then, for reasons nobody has recovered, never published it. He died in 1761 with the essay sitting in his papers ^[1]^[2].

His friend Richard Price, a fellow minister with a serious head for insurance and probability, went through those papers, saw what he had, and edited the manuscript for publication. It was read to the Royal Society and appeared in the Philosophical Transactions under the title An Essay towards solving a Problem in the Doctrine of Chances, dated 1763 and in print the following year ^[1]^[3]. Price added the framing that made the case for why the result mattered; the theorem carries Bayes' name, but the world saw it through Price.

Then the piece the engineering version rests on: Pierre-Simon Laplace, working in France and apparently unaware of Bayes, derived the same inverse-probability principle in 1774 and spent decades generalizing it into the form we actually use, laid out in his 1812 Théorie analytique des probabilités ^[3]^[4]. For most of the next century the whole subject went by the name inverse probability, not Bayes' anything.

The label "Bayesian" is a twentieth-century arrival ^[4], and it started as an insult. R. A. Fisher, no friend of the method, used it dismissively around 1950; I. J. Good picked it up shortly after, and by the time Savage's Foundations of Statistics landed in 1954 the term had hardened into the name of a whole school ^[5]. The minister who never published his one big idea ended up with an adjective attached to a way of thinking he might not have fully recognized.

Related tools

/tools/thermal-noise-floor
/tools/noise-figure-temperature
/tools/snr-enob
/tools/radar-horizon

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →