Ensemble Weather Forecasting

Launch fifty forecasts from almost the same starting point and watch them agree, then stop agreeing. The page measures what each one is worth, what their average is worth, and whether their disagreement warned you in advance.

Worked example — Ensemble Weather Forecasting

Ensemble spread	3.30 K
Error of the ensemble mean	2.91 K
Error of a typical member	4.35 K
The split: mean error and spread	19.131
Members worse than their own mean	50 of 50
Where a forecast stops being worth issuing	3.60 K
The ensemble mean lasts	10.9 days
One forecast alone lasts	9.4 days

These are the values the calculator shows when the page opens. Change any input above to recompute them.

Loading interactive simulation...

Lesson

The theory — Ensemble Weather Forecasting

An ensemble forecast runs the same model many times from slightly different starting states, because the starting state is never known exactly. The ensemble mean is the average of those runs and is the best single estimate. The spread is how far the members sit from that mean, and it is the forecast’s own statement of how much it should be believed. The model here is Lorenz 96, which Edward Lorenz wrote in 1996 as a toy atmosphere rather than a chaos demonstration: 40 values around a latitude circle, each pushed by advection from its neighbours, damped, and driven by a constant forcing. One model time unit is read as five days, which is Lorenz’s own calibration, and one model unit as one kelvin of temperature anomaly, which is this page’s convention for making a threshold concrete.

What each symbol means

N: how many members the ensemble runs. 50 is an ordinary operational number; the page lets you go down to 2 and up to 100
σ: the spread: the root-mean-square distance of the members from their own mean
E: an error: the root-mean-square difference between a forecast and what actually happened. Only ever computable after the fact
λ: the growth rate of a small error, 0.3154 per day here, which is a doubling time of 2.20 days — close to what the real atmosphere does
P: a probability read straight off the members: the fraction of them that cross a threshold

Assumes: A perfect model. Every member here integrates the same equations that produced the truth, so the only thing that separates them is where they started. The analysis — the forecaster’s best estimate of today — is the truth plus Gaussian noise, and the members are that analysis plus another draw of the same size, which makes the truth statistically one more member. That last detail is what makes the probabilities mean anything: perturbing the truth itself instead puts it at the centre of the cloud, and measured that way the ensemble said 24% for events that happened 15% of the time, and 65% for events that happened 80%.
Breaks when: On everything a forecasting centre worries about. A real ensemble is under-dispersive — its spread is smaller than its error — because every member carries the same imperfect physics, and centres spend a great deal of effort adding model perturbations to repair it. Real perturbations are not drawn at random either: they are chosen along the directions that grow fastest, so that fifty members explore the error rather than sampling it. And 40 numbers on a circle is not an atmosphere; a forecast model carries on the order of 10⁹ of them.

The average of fifty forecasts beats all fifty of them 🖖

Press An ordinary week and read day 16. A typical member is out by 4.35 K; their average is out by 2.91 K; and the count of members beaten by that average is 50 of 50. This is not a run of luck, and the results table shows why. Expand the members’ squared error about their own mean and the cross term vanishes, leaving an identity: mean(Eᵢ²) = E(mean)² + (N−1)/N · spread². The average of the members’ squared errors is always the mean’s squared error plus their disagreement, which is why the typical member cannot do better and why an individual one only manages it by luck — one did, on the quiet week. The more the members disagree, the larger the gap. At day 16 that reads 19.1308 = 19.1308 on both sides.

The spread is the forecast 🖖

Press A quiet week, then A stormy week. Same model, same fifty-member ensemble, same observation error of 0.3 K. On the quiet week the ensemble mean is still worth issuing at 15.7 days; on the stormy one it is spent at 6.9. The atmosphere is not equally predictable from one week to the next, and the useful part is that you can tell which week you are in without knowing the answer. The members disagree early on the days the forecast turns out badly, often enough to bet on: sixty launches verified at six days, plotted in the third panel, put spread against error at r = 0.80. That is what an ensemble is for, and it is the reason a forecast that says "I do not know" is doing its job rather than failing at it.

The best single forecast is a map of weather that could not happen 🖖

Watch the last column of the table on An ordinary week as the lead time grows. It is the variance left in the ensemble mean, against the variance of a member: day 0 reads 99%, day 8 reads 57%, day 16 reads 21%. Averaging fifty maps that disagree flattens them, and a flat map is not a weather pattern. Of the 601 days in this model’s record, the smoothest still carries 70% of the average variance and not one falls below half; the day-16 mean map carries 21%. So the ensemble mean is simultaneously the best single estimate available and a picture of something that will not occur, which is why what a forecasting centre publishes is a probability of rain rather than the mean map.

Problem solved in full

Will it freeze on Thursday? 7 steps
It is Wednesday and you are being asked whether Thursday week — eight days out — will fall below freezing at place 19. The ensemble mean says +1.44 K. Decide what to publish, and then find out what happened. Press Will it freeze? to put the tool in this state.

Open this problem in the tool
1. Read the mean first, because that is what a single forecast would have given you. The ensemble mean at place 19 on day 8 is +1.44 K, comfortably above zero. A forecaster with one model run publishes "no frost" and goes home.
  
  $\overset{x}{ˉ}_{19} (d = 8) = \frac{1}{N} i = 1 \sum 50 x_{i, 19} = + 1.44 K$
2. Now count instead of averaging. Of the 50 members, 20 are below zero at that place and that day. The ensemble’s answer to the question asked is 40%.
  
  $P (x_{19} < 0) = \frac{# { i : x _{i, 19} < 0 }}{N} = \frac{20}{50} = 40%$
3. Those two statements are about different things, and no amount of care with the mean would produce the second. An average says nothing about how many of the numbers under it lie below a line. Fifty members all sitting at +1.44 K and the fifty members here, which run from −4.95 K to +11.25 K, both average to +1.44; the first ensemble gives frost no chance at all and the second gives it four in ten.
  
  $\overset{x}{ˉ} < 0 ⟺ \frac{1}{N} i \sum x_{i} < 0 \neq ⟺ # {i : x_{i} < 0} > \frac{N}{2}$
4. What happened at place 19 on day 8 was −3.86 K. The mean was wrong about the sign of the answer. The ensemble was not wrong: it said the event had a four-in-ten chance, and a four-in-ten event happening is not a failed forecast.
  
  $T_{19} (d = 8) = - 3.86 K$
5. That last sentence is only defensible if 40% means 40%, and one case cannot establish it. The fourth panel is the measurement: 1,600 place-days at this lead, sorted by the probability the ensemble issued. The calls in the 30%–40% band — 130 of them — verified 29% of the time, and the 60%–70% band verified 66%. The ensemble’s probabilities are worth taking at face value because they have been checked, not because they come out of a model.
  
  $Pr (T < 0 ∣ P \in [0.3, 0.4]) = \frac{38}{130} = 29%, Pr (T < 0 ∣ P \in [0.6, 0.7]) = \frac{57}{87} = 66%$
6. The choice of case is honest about what it is: this is a day picked to show the failure, and on most days the mean and the ensemble agree about which side of a threshold the weather falls. The point is not that the mean is usually wrong. It is that the mean cannot answer a question of this shape at all, and the members can.
7. Where the page stops: a real frost warning weighs the cost of a wrong quiet day against the cost of a wrong warning, and that calculation needs the value of the two mistakes, which no atmosphere supplies. Our own calibration page is where the 40%-means-40% question gets its proper treatment.
Answer

Publish 40%, not "no frost". The mean was +1.44 K, 20 of the 50 members were below zero, and the outcome was −3.86 K. The mean answered a question nobody asked.

$P = \frac{20}{50} = 40% \overset{x}{ˉ} = + 1.44 K T = - 3.86 K$

Learning path

How a weather forecast is made

References (5)

The seminar the model comes from, where Lorenz wrote it down as a toy atmosphere rather than as a chaotic curiosity: E. N. Lorenz, "Predictability: A problem partly solved." Proc. ECMWF Seminar on Predictability, Vol. 1, Reading, ECMWF, 1996, pp. 1-18.
The paper that put the model into general use, and the source of the K = 40, F = 8 settings this page runs: E. N. Lorenz and K. A. Emanuel, "Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model." Journal of the Atmospheric Sciences 55(3), 1998, pp. 399-414. DOI 10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2
What an operational ensemble actually is, including why real ones are under-dispersive and why their perturbations are chosen rather than drawn at random: M. Leutbecher and T. N. Palmer, "Ensemble forecasting." Journal of Computational Physics 227(7), 2008, pp. 3515-3539. DOI 10.1016/j.jcp.2007.02.014
Where the fourth panel comes from: the decomposition that separates "were the probabilities honest" from "did they say anything useful", which is why a reliability curve is read as a curve rather than scored as a number: A. H. Murphy, "A New Vector Partition of the Probability Score." Journal of Applied Meteorology 12(4), 1973, pp. 595-600. DOI 10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2
And the 1963 paper the whole subject starts from, which is a different model and a different lesson: E. N. Lorenz, "Deterministic Nonperiodic Flow." Journal of the Atmospheric Sciences 20(2), 1963, pp. 130-141. DOI 10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2

By the 42Math team · Reviewed against published sources. Corrections welcome.