Source: The Economist
November 4, 2004

A statistically based study claims that many more Iraqis have died in the conflict than previous estimates indicated

THE American armed forces have long stated that they do not keep track of how many people have been killed in the current conflict in Iraq and, furthermore, that determining such a number is impossible. Not everybody agrees. Adding up the number of civilians reported killed in confirmed press accounts yields a figure of around 15,000. But even that is likely to be an underestimate, for not every death gets reported. The question is, how much of an underestimate?

A study published on October 29th in the Lancet, a British medical journal, suggests the death toll is quite a lot higher than the newspaper reports suggest. The centre of its estimated range of death tolls—the most probable number according to the data collected and the statistics used—is almost 100,000. And even though the limits of that range are very wide, from 8,000 to 194,000, the study concludes with 90% certainty that more than 40,000 Iraqis have died.

Numbers, numbers, numbers

This is an extraordinary claim, and so requires extraordinary evidence. Is the methodology used by Les Roberts of the Johns Hopkins University School of Public Health, in Baltimore, and his colleagues, sound enough for reliable conclusions to be drawn from it?

The bedrock on which the study is founded is the same as that on which opinion polls are built: random sampling. Selecting even a small number of individuals randomly from a large population allows you to say things about the whole population. Think of a jar containing a million marbles, half of them red and half blue. Choose even 100 of these marbles at random and it is very, very unlikely that all of them would be red. In fact, the results would be very close to 50 of each colour.

The best sort of random sampling is one that picks individuals out directly. This is not possible in Iraq because no reliable census data exist. For this reason, Dr Roberts used a technique called clustering, which has been employed extensively in other situations where census data are lacking, such as studying infectious disease in poor countries.

Clustering works by picking out a number of neighbourhoods at random—33 in this case—and then surveying all the individuals in that neighbourhood. The neighbourhoods were picked by choosing towns in Iraq at random (the chance that a town would be picked was proportional to its population) and then, in a given town, using GPS—the global positioning system—to