Spread and Deviation

The centre is one number; spread says how far the data live from it.

Loading interactive simulation...

One fitted number costs one freedom 🖖

The sample mean is the one number that makes the sum of squared deviations as small as it can possibly be. So a sample measured around its own mean always gives a smaller total than the same sample measured around the population mean you cannot see, and dividing by n inherits that shortfall. In the experiment above, samples of five leave the n divisor about a fifth low, and it comes out low on every redraw rather than on average. Dividing by n βˆ’ 1 is the factor that scales the shortfall back.

Standard deviation is not average distance 🖖

Squaring the deviations, averaging, then taking the root is not the same operation as averaging the distances, because squaring makes a value twice as far from the mean count four times as much. The difference only ever runs one way. Standard deviation is never smaller than mean absolute deviation, for any list of numbers at all, and the two are equal only when every value sits the same distance from the mean. The classroom scores read 2.55 and 2.00; the one-outlier preset, which loads the sample divisor, reads 5.15 and 3.13.

IQR deliberately ignores the tails 🖖

The interquartile range throws away the lowest quarter and the highest quarter of the data before it measures anything, which is exactly what makes it unmoved by one wild value. It also makes it deaf. The tight cluster and the one-outlier preset on this page both report an IQR of 2.00, while their standard deviations are 1.22 and 5.15. An IQR on its own has said nothing about how far the data reach.

Problem solved in full

  1. The obvious outlier that a three-sigma rule cannot flag 9 steps

    The one-outlier preset holds seven values between 4 and 7 and one at 20. Work out the z-score of that last value from the sample standard deviation, then decide whether a rule that flags anything past three standard deviations could ever have caught it.

    1. Add the eight values and share them out: 60 across 8 is 7.50. Seven of the eight sit below that, so the single high value is already moving the centre before it touches the spread.

    2. Subtract 7.50 from each. The seven crowded values land between 0.5 and 3.5 away; the eighth is 12.5 away. The panel prints that column, and it adds to zero, which every deviation column does.

    3. Squaring makes the total lopsided. The seven together contribute 29.75, and the value at 20 contributes 156.25 on its own, more than five times the rest of the sample put together. That is the squaring doing the job it is there for.

    4. This preset loads sample mode, so the divisor is 7 rather than 8. The panel prints the fraction as well as the answer: 186.00 over 7 is a variance of 26.57 and a standard deviation of 5.15. Carry that division one place further and s is 5.1547, which is what the next step needs.

    5. A z-score counts standard deviations from the mean, so here it is 12.5 over 5.1547, which is 2.4249. The most conspicuous value in the set does not reach three, and it is not close.

    6. Now ask whether that is a fact about this list or about eight numbers in general. One thing decides it: the deviations sum to zero, so whatever the other seven are, they add to exactly minus the eighth.

    7. Numbers with a fixed sum have their smallest possible sum of squares when they are all equal. So the seven remaining deviations cannot all be small: the most economical they can be is to share the load evenly, and that puts a floor under the squared total.

    8. A floor under the squared total is a floor under s, and a floor under s is a ceiling on z. No value in a list of n can sit further than (n βˆ’ 1)/√n sample standard deviations from the mean. At n = 8 that ceiling is 2.4749. Raise the 20 to 100 and the panel gives a deviation of 82.50 against a standard deviation of 33.35, so z is 2.474. Raise it to a million and z reaches 2.4749 to four places, and there it stops.

    9. The ceiling depends on n and on nothing else, and it first clears 3 at eleven values: ten can reach 2.846, eleven can reach 3.015. The list here holds twelve, so the extreme case is within reach. Type ten zeros and one 1000, still in sample mode. The panel gives a mean of 90.91 and a standard deviation of 301.51, and the largest deviation, 909.09, divided by 301.51 is 3.015, which is the ceiling itself.

    Answer

    z = 2.42, and no three-sigma rule could have flagged it, because eight numbers cannot produce a z of 3 however extreme one of them is.

    The rule was not wrong here so much as unusable, and that is the part worth carrying away: a threshold written in standard deviations is a statement about the sample size as much as about the value, because the value being tested is inside the standard deviation it is being measured against. Three standard deviations needs at least 11 observations before it is reachable at all, four needs 18, and below those sizes a contaminated sample looks exactly like a clean one to the test. Something that does work on eight numbers is already on this page. Tukey's fence puts the boundary at Q3 + 1.5 Γ— IQR, the panel gives Q3 = 7.00 and IQR = 2.00, so the fence sits at 10 and the value at 20 is well outside it. The quartiles are positions in the sorted list rather than averages of it, which is why the point being judged does not get to move the ruler.

Learning path

One number for many: summary statistics

Example problems

  • classroom scores - Eight test scores from 4 to 13. Read the middle column down and the deviations cancel to zero, which is why the third column squares them first. Those squares total 52.00, and over n = 8 that is a variance of 6.50, a standard deviation of 2.55 and a mean absolute deviation of 2.00.
  • tight cluster - Eight readings between 48.00 and 52.00. Spread does not care how large the numbers are, only how far apart they sit: the squared deviations come to 12.00 and the standard deviation is 1.22, against a mean absolute deviation of 1.00. Load the outlier set next and watch that pair separate.
  • one outlier - Seven values crowd between 4 and 7 and one sits at 20. That single point supplies 156.25 of the 186.00 total in the squared column, so the standard deviation reaches 5.15 while the mean absolute deviation stops at 3.13. The interquartile range never sees it: Q1 and Q3 are 5.00 and 7.00, so the IQR stays 2.00 however far out that last value goes. This is the one preset that loads the sample divisor.