How much can we trust the Living Planet Index?
The Living Planet Index is the headline number for the state of the world's vertebrates. But it's built from patchy data, and there's no answer key to check it against. So we built one.
The problem: no answer key
The Living Planet Index (LPI) combines tens of thousands of population time series (counts of birds, mammals, fish, reptiles and amphibians) into trends running from 1970. It's split into regional groups, like "freshwater fishes in the Neotropics", and those groups are combined into the global index.
The data behind it is uneven. Most time series are short. Some regions and animal groups have thousands; others have a handful. The obvious question is how close each trend is to reality. The obvious answer is that we can't know, because nobody has a complete count of the world's animals to compare against.
Our way around this was to simulate the world. We generated thousands of complete virtual datasets in which every population is known in every year, sampled them the way real data is sampled (few populations, short and gappy series, counting errors) and measured how far the sampled trend strayed from the true one. You can try it below.
1. Build a world, then sample it
This simulates 4,000 populations of 400 species over 50 years, then draws a sample like the ones in the Living Planet Database. The teal line is the true trend of the whole virtual world. The orange line is the trend you'd calculate from your sample. The gap between them is measured with the Jaccard distance, which the paper calls the trend deviation value (TDV): 0 means a perfect match. Any single sample is partly luck, so the panel also shows the typical TDV across 20 draws at the same settings.
Bottom: how many sampled populations were actually counted in each year. Notice how thin the data gets at both ends of the timeline, which is why the start and the most recent years of a trend are the most fragile.
The simulation follows the paper's design (stochastic exponential growth, a hierarchy of population, species and group growth rates, random degradation of series, 15% average counting error, Jaccard distance against the true index) but simplifies two steps: every series is gap-filled by log-linear interpolation (the LPI uses GAMs for series with 6+ points), and the split of growth-rate variation between species and populations is approximate. The reliability threshold shown is the paper's 0.176. Expect your numbers to match the paper's pattern, not its exact values.
2. What makes a trend reliable?
We ran this experiment on 3,000 virtual datasets, sampling each one 20 times, and fitted a regression to the results. Five properties of the sampled data, all measurable on the real database, explained 62% of the variation in TDV. Sample size mattered most. The calculator below uses the paper's fitted model (Table 2).
Slider ranges match the ranges found across regional groups in the Living Planet Database (paper, Table 1).
The curve shows predicted TDV against sample size, holding the other sliders fixed. The dashed line is the reliability threshold of 0.176, chosen with the concordance probability method: the point beyond which extra populations buy less and less accuracy. A trend counts as reliable at 100% or more of the required populations, data deficient at 50–99% and severely data deficient below 50%.
One surprise: growing groups need more data than declining ones, because populations can only fall as far as zero but can grow without limit, so increases vary more.
3. The verdict, group by group
Applying the model to every regional group in the Living Planet Database gives each one a reliability score: the populations it has, as a percentage of the populations it needs. Select a cell for the numbers.
The pattern is clear. Bird trends are the most reliable overall, but weak in the tropics, especially Africa. Reptiles and amphibians are data deficient almost everywhere. Freshwater trends are the least reliable of the three systems, and fewer populations are needed in the global north (median 213) than in the global south (median 354).
Two results cut against intuition. First, the share of species represented in a group says little about its reliability: some groups that look well covered by species count are data deficient, and some that look poorly covered are fine. Second, the regional groups with the most potential to undermine the global index, those with a high weight in the index and a low reliability score, are all tropical.
4. What's the fastest fix?
We tested three ways of improving a typical weak group (200 populations, mean length 14 years).
Improvement in trend accuracy (TDV) relative to the control group. Only adding existing time series gave a statistically significant improvement (p < .001). The paper reports a mean for the 10-year blitz and medians for the other two.
The best investment is gathering data that already exists, from studies, reports and agency records that never made it into the database. Where that's exhausted, going back to recount populations that were studied before is nearly as useful as a ten-year monitoring campaign, and takes one year instead of ten.