Fitting the mean to your data costs exactly one observation
Eight marks, one sum of squared deviations: 52. Divide by 8 and you get 6.50; divide by 7 and you get 7.43, which is 14% larger. Every course says the second one is right, and the reason usually given is a name rather than a mechanism.
Open Spread and Deviation on its first preset — eight marks, 4, 6, 7, 7, 8, 9, 10, 13 — and the squared deviations add up to 52. In population mode the tool divides by 8 and prints 6.50. Press the sample button and the same 52 becomes 7.43. Nothing about the marks changed.
Every introductory course explains the second number by saying that the sample mean has consumed a degree of freedom, so only n − 1 of the deviations are free to vary. That is true, and as an explanation it is a closed loop: the answer is n − 1 because there are n − 1 of something. It does not say why the raw figure was wrong to begin with, or why it is wrong in one direction rather than either.
A fitted mean makes the data look tight
Take the sum Σ(xᵢ − c)² and let c vary. It is a parabola, and its lowest point sits at c = x̄. That is not a fact about your particular numbers; it is what the mean is for. Of every value you could have measured the spread around, the sample mean is by construction the one that returns the smallest answer.
The true mean μ is one of the values you did not pick, and the difference between the two is exact:
Σ(xᵢ − μ)² = Σ(xᵢ − x̄)² + n(x̄ − μ)²
The last term is a square multiplied by n, so it can never be negative. The spread a sample shows around its own mean is therefore never larger than the spread that same sample shows around the truth, and the two agree only when x̄ lands exactly on μ. Over all 16,807 samples of five that can be drawn from the population in the tool's sampling panel, that identity holds to within floating-point noise; the worst discrepancy is 4 × 10⁻¹⁴. The two sides come out exactly equal in 1,451 of them, 8.6% — the samples that happened to average 10 on the nose.
The cost is one observation, at every sample size
So how big is n(x̄ − μ)² on average? The sample mean has variance σ²/n, which is the one thing everybody remembers about averaging and the fact the central limit theorem is built on. Multiply by n and you get σ². One population variance. Not a fraction of one, and not something that shrinks as the sample grows: the same at five values and at five thousand.
Σ(xᵢ − x̄)² therefore averages nσ² − σ², which is (n − 1)σ², and dividing that by n − 1 hands σ² back. The whole derivation is an identity and a variance everyone already knows.
A degree of freedom is a name for the observation you spent locating the mean. Spending it is what makes the deviations too small.
Checked without a simulation
The tool's sampling panel draws repeatedly from a population it knows the answer for: 4, 6, 8, 10, 12, 14, 16, mean 10, variance exactly 16. At its default of five values per sample and two thousand samples it prints 12.96 for the n divisor against 16.20 for n − 1.
Those last digits wobble because the draws are random, and with a population this small they do not have to. There are 7⁵ = 16,807 possible samples of five, all equally likely, so the average can be taken over every one of them rather than over a sample of them:
- two values per sample: 8.000000 dividing by n, 16 dividing by n − 1
- three values: 10.666667, and 16
- five values: 12.800000, and 16
- seven values: 13.714286, and 16
The first column is 16 × (n − 1)/n to every digit shown. The second is 16.0000000000, which is not a rounding of anything.
What the correction does not do
Unbiased means the average over all possible samples is right. It says nothing at all about the sample in front of you, and the distance between those two claims is bigger than the correction itself.
Run the tool's experiment two hundred times with different seeds. The n divisor lands below the truth in 200 runs out of 200, averaging 0.8004 of it, one-sided every single time, exactly as the identity demands. The n − 1 divisor lands below the truth in 97 of 200. A coin flip is what unbiased looks like from outside.
Now stop averaging and take one sample. Across all 16,807 samples of five, s averages 3.855 against a true σ of 4, and its own standard deviation is 1.068, which is 26.7% of σ. Changing the divisor moves s by 1 − √(4/5), or 10.6%. The correction is two and a half times smaller than the scatter it is sitting in, and that ratio gets worse rather than better as samples grow: at fifty values it is 1.01% of σ against a scatter of 6.32%.
The correction fixes an average over the samples you did not take. It cannot fix the one you did.
And it does not fix the standard deviation
s² is unbiased for σ². s is not unbiased for σ, and the square root is what breaks it. A square root is concave, so the average of the roots falls below the root of the average — Jensen's inequality, strict unless the quantity being rooted is a constant, which s² never is. E[s] < σ for every population that has any spread at all, at every sample size, and no choice of divisor repairs it, because the defect is in the root rather than in the sum.
On the tool's population, s runs 3.6% below σ at five values, 1.3% below at ten, 0.20% below at fifty. How large the shortfall is depends on the shape of the population. Which way it points does not.
Almost nobody corrects it, and that is a defensible position rather than an oversight. The t distribution behind a confidence interval was derived for exactly this s, bias and all, so the interval covers what it claims to cover. Where s gets reported as a number in its own right the correction does get made: an s-chart in statistical process control divides the average sample standard deviation by a tabulated constant, c₄, before anyone calls it σ. Gurland and Tripathi gave a simple approximation for it in 1971.
Unbiased is a choice, and not the obvious one
Ask a different question of the same sum of squares. Not "which divisor is right on average" but "which divisor lands closest" — smallest mean squared error across all the samples — and neither n nor n − 1 is the answer. Divide by some c and minimise, and the optimum comes out as
c* = n + 1 + γ₂ (n − 1) / n
where γ₂ is the excess kurtosis of the population. For a normal population γ₂ is zero and the best divisor is n + 1, which is larger than n and two larger than the one you were taught. The population in the tool is flat-topped rather than bell-shaped, γ₂ = −1.25, and at five values per sample c* comes out at exactly 5.0000. Over all 16,807 of those samples the mean squared error is 64.00 with the n − 1 divisor and 51.20 with n. Bessel's estimator is right on average and 25% worse at being close.
Nobody should switch divisors on the strength of that. γ₂ is a property of the population you are trying to describe, and you have five numbers. Searls and Intarapanich set the kurtosis-using estimator out in The American Statistician in 1990 and it has stayed a note rather than a practice. What it settles is that n − 1 answers one narrow question well, and the question is narrower than the way it is taught.
Bessel, 1838
The divisor carries the name of Friedrich Bessel, who published thirty-six pages titled "Untersuchungen über die Wahrscheinlichkeit der Beobachtungsfehler" — investigations into the probability of observational errors — in Astronomische Nachrichten in 1838. Karl Pearson would not be born for another nineteen years. The question was how far to trust a reading taken through a telescope, and readings came in twos and threes. At two observations the two divisors differ by a factor of √2, which is 41% on the standard deviation. Nobody at that sample size is arguing about a rounding.
Go back to the eight marks. If those eight people are the whole class, 6.50 is the right number and n − 1 would be an error: you are describing them, and there is nobody else to be wrong about. If they are a sample, 7.43 is your estimate of the spread of the population they came from, right on average and almost certainly not right here. The arithmetic cannot tell you which of those you are in. That is what the toggle on the tool is asking.
References (3)
- the 1838 memoir the divisor is named after Bessel, F. W. (1838). Untersuchungen über die Wahrscheinlichkeit der Beobachtungsfehler. Astronomische Nachrichten, 15, 369–404.
- the correction nobody applies: s is still biased low for σ after Bessel Gurland, J., & Tripathi, R. C. (1971). A Simple Approximation for Unbiased Estimation of the Standard Deviation. The American Statistician, 25(4), 30–32.
- the divisor that minimises mean squared error rather than bias, and why it needs the population’s kurtosis Searls, D. T., & Intarapanich, P. (1990). A Note on an Estimator for the Variance That Utilizes the Kurtosis. The American Statistician, 44(4), 295–296.