The Evolution of Cooperation

Set the starting mix of a population playing Axelrod's tournament against itself. Each generation multiplies every strategy's share by how well it scored against the population it actually met, so the winners breed and the losers stop playing.

Worked example — The Evolution of Cooperation

Still above a tenth of a percent	3 of 8
Lowest mean payoff	1.00 at generation 27
Mean payoff at the end	3.00
One defector among reciprocators	1.02 against their 3.00
One reciprocator among defectors	0.9950 against their 1.00
Defectors at their peak	99.2% at generation 21
Defectors at the end	0.0%
Generations	200

These are the values the calculator shows when the page opens. Change any input above to recompute them.

Loading interactive simulation...

Lesson

The theory — The Evolution of Cooperation

The replicator equation is what turns a tournament into an ecology. Each strategy holds a share of the population; its fitness is what it earns per round against the population it actually meets, which is every other strategy weighted by how common that strategy is; and the next generation multiplies each share by its own fitness divided by the population mean. A strategy above the average grows, one below it shrinks, and nothing else happens — no strategy is ever invented, chosen or taught.

What each symbol means

pᵢ: the share of the population playing strategy i
fᵢ: what that strategy earns per round against the whole population
f̄: the population mean, the bar every strategy is measured against

Assumes: An infinitely large, well-mixed population: every strategy meets every other exactly in proportion to its share, with no neighbourhoods and no luck. That is what makes the shares smooth numbers rather than counts of individuals, and it is also why insight block 2 can follow a share down to 10⁻¹⁷⁶ without the model noticing that no such population exists.

The defectors are killed by their own success 🖖

Press A trusting world, run to the end and watch the red line. One percent of the population defects; 97% of it cooperates with anyone. For eight generations almost nothing happens. Then defection passes half the world in generation 9 and 90% of it in generation 12, and by generation 14 the table calls the unconditional cooperators extinct. That is also the moment the defectors' income dies, because it was the same thing: a defector who meets another defector earns 1 a round. The mean payoff sits at 1.00 from generation 15 to generation 120, a hundred generations of everyone taking from everyone. Meanwhile Tit for Two Tats, Tit for Tat and Grudger came through the famine on about 0.17% each, and when two of them meet they still earn 3. Their three shares together compound at about one and a half percent a generation, which looks like nothing for a century and then stops looking like nothing: the defectors hold 97.1% at generation 120, 6.5% at 150, and are gone by 155. Axelrod called this the ecological round of his tournament. No new strategy enters it. The only thing that changes is how many of each there are.

Cooperation comes back; the cooperators do not 🖖

At generation 200 the population earns 3.00 points a round, which is the most it can earn, because every pair that meets cooperates. The trusting world has been restored without the trusting strategy in it. Always cooperate fell below a tenth of a percent in generation 14, and its share at generation 200 is 3.55 × 10⁻¹⁷⁶. It is not even still falling: it bottoms out in generation 147 and recovers 23% from there. That is the generation its fitness passes the population average, 1.63 against 1.57, because by then most of what a cooperator meets is a reciprocator paying three, while the average is still weighed down by defectors taking one off each other. Replicator dynamics multiply, and multiplication has no floor, so the model can shrink a strategy to 10⁻¹⁷⁶ and hand it back. A population cannot: there are about 10⁸⁰ atoms in the observable universe, and a share of 10⁻¹⁷⁶ is not a rare strategy but an absent one. What ends up earning three a round is Tit for Two Tats on 38.3%, Tit for Tat on 31.0% and Grudger on 30.7%. All three of them open by cooperating and go on cooperating with each other for all 200 rounds, so they collect exactly what Always cooperate would have collected. The difference between them and it shows up only when a defector arrives, and it is the whole difference: they stop.

A quarter of one percent is enough to start 🖖

Read the three cards under Can cooperation get started at all?. A world of unconditional defectors pays everyone 1.00 a round. Drop a single Tit for Tat into it and the newcomer earns 0.9950: it cooperates once, gets nothing for it, and defects for the remaining 199 rounds. Half a hundredth short, and it dies out. Now let the newcomers arrive together, so that a fraction q of their interactions is with each other and pays 3.00. The cluster earns q × 3.00 + (1 − q) × 0.9950, which passes 1.00 at q = 0.2494%, or one interaction in 401. That is Robert Axelrod and W. D. Hamilton's answer, published in 1981, to the question of how cooperation ever gets going in a world that punishes it. The reciprocator's whole loss is one exploited round out of 200, so the threshold falls like 1/n as the relationship lengthens. An unconditional cooperator is exploited in every round rather than the first, and its threshold is a third of all its interactions at any length of relationship whatsoever.

Problem solved in full

How small can the first cluster be? The tool prints one answer: a cluster of reciprocators needs 0.2494% of its interactions to be with its own kind before it can hold against a world of defectors. That figure belongs to a 200-round match at Axelrod's payoffs. Where does it come from, and what happens to it when the relationship is shorter?

Open this problem in the tool
1. In a world of unconditional defectors every meeting is mutual defection, so a resident earns the punishment payoff P in every round.
  
  $π_{res} = P = 1$
2. A lone Tit for Tat cooperates on the first round and collects the sucker payoff S, then defects for the remaining n − 1 rounds. It is robbed once and spreads that loss over the whole match, so at 200 rounds it keeps 199/200.
  
  $π_{lone} = \frac{( n - 1 ) P + S}{n} = \frac{199 \times 1 + 0}{200} = 0.9950$
3. Now give the newcomers a cluster: a fraction q of their interactions are with each other, paying the reward R, and the rest are with the residents. The cluster holds when that average beats what the residents earn.
  
  $q R + (1 - q) π_{lone} > π_{res} = P$
4. Solve for q. The n under both differences cancels, leaving the cost of one exploited round over what n rounds of mutual cooperation gain on mutual defection, plus that same cost.
  
  $q^{*} = \frac{P - π _{lone}}{R - π _{lone}} = \frac{( P - S ) / n}{( n ( R - P ) + ( P - S ) ) / n} = \frac{P - S}{n ( R - P ) + ( P - S )}$
5. At 5, 3, 1, 0 and 200 rounds that is 1/401, the 0.2494% the tool prints. The threshold falls like 1/n: at ten rounds it is 1/21, or 4.76%.
  
  $q^{*} = \frac{1}{2 n + 1} = \frac{1}{401} = 0 . 2494%, n = 10 \Rightarrow \frac{1}{21} = 4 . 76%$
6. An unconditional cooperator is exploited in every round rather than only the first, so its lone payoff is S itself and the n disappears from its threshold altogether. At one round the two strategies are the same strategy and the two thresholds agree at a third. Every round after the first is a discount only the reciprocator collects.
  
  $q_{AC}^{*} = \frac{P - S}{R - S} = \frac{1}{3}$
Answer

A cluster of reciprocators needs q ≥ (P − S)/(n(R − P) + (P − S)) of its interactions internal, which at Axelrod's payoffs is 1/(2n + 1) and at 200 rounds is 1/401 = 0.2494%. A cluster of unconditional cooperators needs (P − S)/(R − S) = 1/3, at every n. Both clusters are doing the same thing, arranging to meet their own kind, and the reciprocator needs a hundred and thirty times less of it, because the only round it can be robbed in is the first one.

$q^{*} = \frac{1}{2 n + 1} vs q_{AC}^{*} = \frac{1}{3}$

Learning path

The best reply, and what happens when nobody computes one

References (4)

The cluster result this page's third block and worked problem are about: a lone reciprocator cannot enter a world of defectors, and a cluster of them can. Also the source for the 1981 date and for the two authors' names: R. Axelrod and W. D. Hamilton, "The Evolution of Cooperation." Science 211(4489), 1981, pp. 1390–1396.
The second tournament, from which this population's eight strategies and the 200-round match are drawn, written up by the man who ran it: R. Axelrod, "More Effective Choice in the Prisoner's Dilemma." Journal of Conflict Resolution 24(3), 1980, pp. 379–403.
"The ecological round" — the same strategies in changing proportions, with no new entrant — and the four maxims that came out of it: R. Axelrod, The Evolution of Cooperation. Basic Books, New York, 1984.
The generation step on this page is the discrete replicator equation, and this is where it and the evolutionarily-stable-strategy connection were set out: P. D. Taylor and L. B. Jonker, "Evolutionary stable strategies and game dynamics." Mathematical Biosciences 40(1–2), 1978, pp. 145–156.

By the 42Math team · Reviewed against published sources. Corrections welcome.