Correlation and autocorrelation

The formula

Cross-correlation of two sampled signals:

R_xy[k] = Σ_n x[n] · y[n+k]

Reading: shift y by k samples, multiply point-by-point against x, add it all up. Where the signals match, the products reinforce and the sum is large; where they don't, positive and negative products cancel.

Autocorrelation is the same operation with a signal against itself:

R_xx[k] = Σ_n x[n] · x[n+k]

Reading: how much does this signal resemble a shifted copy of itself? Any periodicity in x shows up as repeating peaks in R_xx, spaced one period apart.

The normalized version for two data sets is the Pearson correlation coefficient:

r = Σ (x_i − x̄)(y_i − ȳ) / √( Σ(x_i − x̄)² · Σ(y_i − ȳ)² )

Reading: subtract the means, correlate, then divide by the two signal spreads so the answer lands between −1 and +1 regardless of units or amplitude.

One identity worth memorizing: R_xx[0]/N = mean-square value = RMS². Mind the units here: the raw zero-lag sum Σx² is the signal's energy, and it grows with record length; divide by the number of samples N and you get power — the mean square. Compare the printed sum straight against RMS² and you are off by a factor of N. And by the Wiener–Khinchin theorem, the Fourier transform of R_xx(τ) is the power spectral density — autocorrelation and the power spectrum are the same information in two coordinate systems ^[8]. This holds for wide-sense-stationary signals — if the statistics drift over the record (run-up, thermal transients), detrend or segment into locally stationary blocks first, or R(τ) and the PSD are not well defined.

Where you meet it

Time-delay estimation on a test stand. Two accelerometers on a structure, one disturbance, and you want the propagation delay between channels. Cross-correlate the two records; the lag of the peak is the delay. Sampling at 1 MHz with the peak at lag 137 means 137 µs — no curve fitting, no phase unwrapping.
GPS ranging. A GPS receiver holds a local replica of each satellite's C/A code — a 1023-chip Gold code repeating every millisecond — and correlates it against what the antenna hears. The code offset that produces the correlation spike is the signal transit time, and one chip of misalignment is about 293 m of range ^[14]^[15]. Every position fix your phone has ever produced was a correlation peak first.
Radar and sonar detection. The matched filter in a radar receiver is cross-correlation of the return against the transmitted pulse shape — provably the best possible detector for a known signal in white noise ^[11]. A target at 15 km hands you the transmit waveform back 100 µs later, buried in noise; correlation digs it out.
Periodicity hunting in noisy test data. A bearing signature or a flow oscillation that is invisible in a raw strip chart shows up as clean repeating peaks in the autocorrelation, because the noise decorrelates in a few samples while the periodic part keeps agreeing with itself. NIST's engineering statistics handbook uses the autocorrelation plot as the standard check on whether residuals are actually random ^[3].

How it works

Correlation is a sliding dot product. At each trial offset you are asking one question — do these two records agree right now? — and the sum turns thousands of weak per-sample agreements into one large number. That averaging is where the noise immunity comes from: noise samples are as likely to cancel as to add, while true signal alignment adds every time. This is why a GPS correlator recovers a signal sitting well below the thermal noise floor.

Behavior worth knowing:

Autocorrelation always peaks at zero lag, and no other lag can beat it — that is the Cauchy–Schwarz inequality doing structural work. Normalize by R_xx[0] and everything lands in ±1.
White noise autocorrelates to a single spike at zero lag. For a finite record of N samples the off-peak values don't vanish; they scatter with a standard deviation near 1/√N — about 0.03 for a 1000-sample record. Anything poking well above that band is structure, not luck.
A sine wave A·sin(ωt) has time-averaged autocorrelation (A²/2)·cos(ωτ) — that is the sliding sum divided by N, the same normalization as the RMS² identity above. Note what survived: amplitude and frequency. Phase is gone. Autocorrelation is deaf to phase, which is exactly why it pairs with the power spectrum and not the full Fourier transform.
In practice nobody computes the sliding sum directly for long records. Wiener–Khinchin gives the shortcut: FFT, magnitude-squared, inverse FFT ^[8] — with the same wide-sense-stationarity condition attached. The trap is that FFTs give circular correlation — the record wraps around and correlates with its own tail. Zero-pad to at least twice the record length or the wraparound contaminates every lag.

The mistakes people actually make:

Correlating raw signals with DC offsets. A constant correlates perfectly with a constant, so two signals riding on offsets show a huge "correlation" that means nothing. Subtract the means first.
Correlating two drifting signals and believing the number. Any two records that both trend upward correlate strongly — that is Yule's nonsense-correlation trap, and it has been embarrassing engineers and economists since 1926 ^[6]. Detrend before you correlate, and remember that correlation is a similarity score, not a causation certificate.
Trusting the biased estimator at large lags. Divide the lag-k sum by N and the estimate shrinks artificially at long lags; divide by N−|k| and the variance blows up where few samples overlap. Either way, don't read anything into the last 10–20 percent of the lag axis.
Reading delay finer than one sample. The raw peak location is quantized to the sample interval. If you need sub-sample delay, interpolate around the peak — a three-point parabolic fit is the usual move — and understand that your bandwidth, not your sample rate, sets the real resolution.

History

The correlation coefficient came out of biology, not signal processing. Francis Galton, chasing heredity patterns in the 1880s, worked out that the relationship between two variables could be captured in a single number; Karl Pearson, who ran the mathematics in Galton's orbit at University College London, put the product-moment formula on rigorous footing in 1896, which is why the normalized r carries his name ^[4]^[5].

The cautionary chapter came in 1926, when G. Udny Yule used his Royal Statistical Society presidential address to show that Church of England marriage rates and national mortality from 1866 to 1911 correlated at 0.9512 — a number both enormous and meaningless. His diagnosis was that internally correlated time series routinely produce spurious agreement, a result sharp enough that mathematicians were still publishing rigorous treatments of it in the 2010s ^[6]^[7].

The engineering version arrived through Norbert Wiener, whose 1930 Generalized Harmonic Analysis connected a signal's autocorrelation to its power spectrum, with Aleksandr Khinchin supplying the stationary-random-process form in a 1934 paper — jointly the Wiener–Khinchin theorem that every spectrum analyzer leans on ^[8]^[9]^[10]. War work made it hardware: in 1943 D. O. North at RCA's Princeton laboratory showed, in a then-classified report, that correlating a radar return against a replica of the transmitted pulse is the optimum detector in noise — the matched filter ^[11]^[12]. And in 1967 Robert Gold published a family of code sequences with deliberately engineered correlation properties — sharp autocorrelation peaks, uniformly small cross-correlation between family members — which is precisely what GPS adopted for its C/A ranging codes ^[13]^[14]^[15]. When your phone locks onto eight satellites sharing one frequency, Gold's correlation mathematics is what keeps them from talking over each other.

Related tools

/tools/rms-peak — the zero-lag autocorrelation, divided by record length, is the RMS squared
/tools/doppler-shift — the other half of radar and GPS measurement: correlation finds range, Doppler finds rate
/tools/thermal-noise-floor — sets the noise the correlation gain has to climb out of
/tools/snr-enob — quantifies the SNR that correlation processing improves
/tools/frequency-period — the lag of the first autocorrelation peak is the period; convert it

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →