Expected value and variance

The formula

Expected value, discrete case:

E[X] = sum over i of  x_i · p_i

Reading: multiply every possible value by how likely it is, add them up. The probability-weighted average of everything that can happen.

Expected value, continuous case:

E[X] = integral of  x · f(x) dx      (over the whole range of x)

Reading: same idea, but the probabilities become a density f(x) and the sum becomes an integral.

Variance:

Var(X) = E[(X - mu)^2]  =  E[X^2] - (E[X])^2      where mu = E[X]

Reading: the average squared distance from the mean. The right-hand form — second moment minus mean squared — is the one you actually compute.

Standard deviation:

sigma = sqrt( Var(X) )

Reading: the square root of variance, so it comes back to the same units as X. If X is in volts, sigma is in volts; variance would be in volts².

Sample estimators, when all you have is n measured points:

mean = ( sum x_i ) / n
s^2  = sum (x_i - mean)^2 / (n - 1)      s = sqrt(s^2)

Reading: the mean estimates mu, s² estimates the variance, and s is your standard deviation. The n - 1 — not n — is the part people get wrong.

Where you meet it

On the bench, characterizing an ADC or a voltage reference. You short the input, log ten thousand samples, and report two numbers: the mean (your offset / DC value) and the standard deviation (your noise). The datasheet's "RMS noise" spec is a standard deviation. ENOB, noise floor, and every dB of dynamic range trace back to that sigma.
On the test stand, stacking independent error sources. Thermal noise, quantization, reference drift, and fixture pickup are separate, uncorrelated contributors. You do not add their sigmas — you add their variances and take the square root. Three sources at 3, 4, and 12 microvolts RMS combine to sqrt(3² + 4² + 12²) = sqrt(169) = 13 microvolts, not 19. The big one dominates; the small ones nearly vanish under the square.
In a design review, defending a tolerance stack. Worst-case stacking assumes every part is simultaneously at its extreme — expensive and usually never happens. Statistical (RSS) stacking treats each dimension as a random variable with its own variance and combines them root-sum-square. The board buys the tighter number only if you can show the contributors are independent.
In a reliability or Cpk pull. Process capability is nothing but mean and sigma against spec limits. Cpk measures how many standard deviations fit between your process mean and the nearest limit. Shift the mean, or widen the sigma, and the yield number moves.

How it works

Expected value is linear, and that is the property you lean on hardest. E[aX + b] = a·E[X] + b always — no assumptions about the distribution, no independence required. Sums pass straight through: E[X + Y] = E[X] + E[Y] whether or not X and Y are related. This is why you can chain offsets and gains through a signal path and track the mean by hand.

Variance is not linear, and this is where people trip. Two rules:

Var(aX + b) = a^2 · Var(X)          (a constant offset b changes nothing)
Var(X + Y)  = Var(X) + Var(Y)        ONLY when X and Y are independent

The a² is the reason a gain of 10 multiplies your noise power by 100 and your noise voltage by 10. The additive offset b drops out entirely — shifting a signal up by a volt moves the mean but not the spread.

The independence caveat on the second rule is the one that bites. If your error sources are correlated — same clock, same ground, same temperature swing driving two of them — the variances do not simply add. There is a covariance term you left out, and your combined sigma will be wrong, usually optimistically. Before you RSS anything, ask whether the contributors truly move independently.

The n - 1 in the sample variance is Bessel's correction. When you estimate the mean from the same data you are using to estimate the spread, the sample sits a little too snug around its own mean, and dividing by n gives a variance that reads low. Dividing by n - 1 corrects the bias. For n = 10 the difference is real: ten readings averaging 10.0 with a spread give a sample sigma of about 0.183 but a population sigma of 0.173 — a five-percent gap that matters when you are reporting uncertainty. Know which one your instrument or your spreadsheet is computing. STDEV and STDEVP in a spreadsheet are not the same function.

Two honest limits. First, both numbers assume the mean and variance exist and are finite — true for the Gaussian and everything you meet on a bench, false for heavy-tailed distributions like the Cauchy, where the sample mean never settles no matter how many points you take. Second, mean and variance describe center and spread and nothing else. Two distributions can share both and look completely different — one symmetric, one skewed, one with a fat tail hiding the failure that gets you. Mean and sigma are the first two moments; they are a summary, not the whole story. When the tail is the risk, plot the histogram.

One more that costs money: variance is in squared units. Sigma is what you quote to people, because a noise spec in "volts squared" means nothing to the tech installing the box. Do the math in variance, report in sigma.

History

The center came first, and it came from gambling. In 1654 Blaise Pascal and Pierre de Fermat traded letters over the "problem of points" — how to split the pot when a game of chance is cut short — and in working it out they reasoned, for the first time, that a future payoff is worth its size times its chance ^[1]^[2]. Three years later the Dutch physicist Christiaan Huygens turned that reasoning into the first published treatise on probability, De ratiociniis in ludo aleae (1657), which laid down expectation as a formal quantity you could calculate ^[1]. Expected value is, quite literally, the mathematics of a fair bet.

Spread took another two centuries to get its name. Astronomers and surveyors had long worked with what Gauss called the "mean error," but the term everyone now uses arrived in a lecture Karl Pearson gave on 31 January 1893, where he proposed "standard deviation" as a cleaner substitute for the mouthful "root mean square error" ^[3]^[4]. It reached print in his 1894 paper "Contributions to the Mathematical Theory of Evolution," and Pearson pinned the symbol σ to it ^[3]^[4].

Variance — the square of that sigma — got promoted to a headline quantity by Ronald Fisher. In his 1918 paper "The Correlation Between Relatives on the Supposition of Mendelian Inheritance," Fisher argued it was cleaner to analyze the square of the standard deviation directly, coined the word "variance" for it, and kept σ² as its symbol ^[3]^[5]. That choice seeded analysis of variance and most of the statistics an engineer runs today. So the two numbers you carry around were built by three different people over 264 years, for three different reasons — dividing a bet, measuring evolution, and breeding better crops — none of them a test engineer, all of them yours now.

Related tools

/tools/rms-peak
/tools/snr-enob
/tools/thermal-noise-floor
/tools/friis-cascade
/tools/adc-resolution

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →