In an exclusive for IraqMortality.org Milan Rai, Author of War Plan Iraq, Regime Unchanged, and Chomsky's Politics gives indepth analysis of the three major mortality studies conducted in Iraq; Iraq Body Count, The Lancet, and The UNDP Report. This document is presented to help activists more fully understand the differences and similarities between these studies.
For five days, begining on October 24th, almost 100 grassroots groups and individual activists in the US, UK, and Switzerland will toll a bell in their communities for Iraqis who have lost their lives in this war and for the families and loved ones they have left behind. This tolling of bells will also usher in the one year anniversary of the publishing of The Lancet Study on October 29th which estimates 100,000 Iraqi deaths due to the war and occupation.
INTRODUCTION
As the death toll in Iraq continues to grow, one question haunting the debate over the occupation is the scale of this loss. Supporters of the continuing war seek to confuse and obscure the issue by presenting existing estimates as in conflict with each other. However, when we examine the best-known Iraq mortality estimates, we find that they tend to support rather than contradict each other.
All known estimates agree that the death rate in Iraq, especially the rate of violent death, has increased dramatically since the US/UK invasion in March 2003. They all indicate that number of ‘excess deaths’ (deaths that would not have occurred if not for the war) is staggeringly high.
IRAQ BODY COUNT
The first authoritative, and still constantly-updated, estimate of war-related deaths in Iraq was compiled by Iraq Body Count (IBC). In July 2005, IBC issued a dense, readable analysis of recorded civilian deaths due to the invasion and occupation of Iraq from March 2003 to March 2005. Careful and conservative work by IBC principal researchers Hamit Dardagan, John Sloboda, Kay Williams and Peter Bagnall, showed that there had been 24,865 civilian war-related deaths, almost all of them as a direct result of violence, reported between 20 March 2003 and 19 March 2005.
In order to provide irrefutable, minimum figures for the death toll, IBC only records civilian deaths which have been reported by two reputable English-language sources.
IBC observed in its June 2005 report: ‘The population of Iraq is approximately 25,000,000, meaning that one in every thousand Iraqis has been violently killed since March 2003.’
THE LANCET
Another much-publicised estimate of war-related deaths in Iraq was published in the Lancet, the world’s premier medical journal, in October 2004. The study was co-authored by Les Roberts, Riyadh Lafta, Richard Garfield, Jamal Khudhairi and Gilbert Burnham, and carried out under their supervision by trained Iraqi medical survey teams. Their random survey of 988 households throughout Iraq was used to derive a conservative estimate that 98,000 Iraqis (civilians, soldiers and insurgents) had died directly and indirectly of war-related causes in the period March 2003 to mid-September 2004. (This estimate was made after excluding the results from Fallujah, the most war-torn area surveyed.)
Careful reading of the Lancet Study, and the exclusion of all results from Anbar province (where Fallujah is located), which is the most violent province in Iraq, indicates that the 98,000 figure is very likely to be an under-estimate.
The Lancet Study commented: ‘Violence accounted for most of the excess deaths and air strikes from coalition forces accounted for most violent deaths.’
THE UNDP REPORT
Another survey-based estimate was published six months after Lancet Study, in April 2005. This estimate was based on research by the United Nations Development Programme (UNDP) and the new Iraqi Government into people’s living conditions. The study was co-organised by Norway’s FAFO Institute for Applied International Studies and the Iraqi Government’s ‘Central Organization for Statistics and Information Technology’.
Called the ‘Iraqi Living Conditions Survey’, this study is sometimes referred to as the ‘ILCS Report’, or the ‘Iraqi MIRA (Multiple Indicator Rapid Assessment) Report’, but we will use the term ‘UNDP Report’.
The survey found that in the period March 2003 to April 2004, there were 24,000 war-related deaths in Iraq. (‘War-related’ was defined much more narrowly than either in the IBC Report or in the Lancet Study, as we shall seee.) Rather than the 988 households surveyed by the Lancet group (on a minute budget), the UNDP surveyed 21,668 households, which makes the resulting estimate much more precise, statistically.
On the other hand, the central focus of the survey was not on deaths in Iraq (the focus for both the IBC Report and the Lancet Study), but living conditions in Iraq; hence the name ‘Iraq Living Conditions Survey’. Only one page out of a 60-page questionnaire was on deaths in the family, as opposed to ‘relatives living abroad’ (the preceding page) and ‘livestock and agriculture’ (the following section) and so on. There were just five questions on adult deaths, in a wide-ranging 83-minute interview with each household.
In fact, the survey organisers were so worried about the results for infant death rates that they ordered a re-survey of the entire 21,000 households on this question, which found that there was a substantial under-count of infant deaths. [1] It is peculiar that questions to do with adult deaths were not also included in the repeat survey, as they may well have been subject to the same pressures that produced the under-count in relation to infant deaths.
The UNDP Report authors comment: ‘As the data on infant mortality make clear — as does the data on malnutrition, presented elsewhere in this report — the suffering of children due to war and conflict in Iraq is not limited to those directly wounded or killed by military activities.’
CONFLICTING RESULTS?
It is common to find officials and commentators using these figures against each other. When they are examined closely, however, they tend to converge rather than contradict each other. Before considering how this is so, we must clarify the different categories of deaths being assessed.
What follows is not a technical analysis of the strengths and weaknesses of the three studies, but a clarification that while there are certain tensions, there is no glaring conflict between the three estimates.
One question we leave to one side, for example, is the correct technical presentation of the various estimates. The estimates in the Lancet and UNDP reports are not given as simple point estimates of 98,000 or 24,000 deaths. They are given as spectrums of possibility with what is called a 95 per cent confidence, and the most likely figure identified within that range.
In the case of the UNDP Report, the data collected ‘indicates 24,000 deaths, with a 95 percent confidence interval from 18,000 to 29,000 deaths’. In the Lancet Study, the confidence interval is 8,000 to 194,000, with the most likely figure within this range being 98,000. Within such confidence intervals, however, not all values are equally likely in reality (a common misconception). Statistically, the central value (24,000 for the UNDP report, 98,000 for the Lancet report) is the most likely value.
Having made this fundamental point, we focus now on the reasons why the various estimates converge more than they diverge.
CATEGORIES
When we consider the number of Iraqi deaths that have occurred because of the invasion and occupation, the most obvious category takes in those Iraqi civilians, soldiers and insurgents directly killed by US-led forces (by gunfire, air-launched missiles, artillery, mines, cluster bombs, and so on).
Then there are other violent deaths that have occurred as a result of the conflict – those deaths of both armed and unarmed Iraqis that are the result of the rocketing rate of violent crime, for example; the deaths of both civilians and uniformed Iraqis caused by anti- occupation or sectarian forces.
Then there are ‘nonviolent’ deaths that have been caused by the conditions created by the invasion and occupation. For example, there is known to be an increased rate of disease caused by war-related damage to sewage, water purification and other facilities vital to the public health infrastructure. (US forces often deliberately cut off vital services to urban areas they are about to attack.)
It might also be that for a variety of reasons (travelling at high speeds to avoid getting caught up in an ambush or a robbery) the rate of lethal traffic accidents has increased. Lives are also lost as families flee US invasions of their cities, for example, and then suffer from exposure and malnutrition as internal refugees.
In total, taking both violent and ‘nonviolent’ causes, there is an overall figure for ‘deaths which would not have occurred if the invasion had not happened – or if the occupation had ended once Saddam’s regime fell apart.’
The technical term is ‘excess deaths’ – deaths that are over and above those that would have otherwise happened.
From another point of view, there are those deaths which have been recorded – in media reports and in morgue records – and there are deaths which have not been recorded – because those who die have been buried without referral to a hospital, or because the incident was not reported to the media. The first kind of death will be picked up by ‘surveillance’ of the situation; the second can only be estimated by a ‘survey’ of the area under consideration.
DIFFERENT MEASURES OVER DIFFERENT PERIODS
Each of the three national estimates (IBC, Lancet, UNDP) has a different time period, and a different set of categories is being measured in each case.
The IBC Report period is the longest at two years. It measures reported deaths of civilians. By definition, this leaves out those deaths which were not reported in the media or recorded at morgues, and it leaves out the deaths of soldiers and insurgents. The IBC Report does include an estimate for the extra level of crime-related deaths caused by the war and occupation, but this once again is only of reported civilian deaths. Violent crime accounts for 36 per cent of reported civilian deaths in the IBC report.
The Lancet Study period is eighteen months, and measures all excess deaths (civilian, military and insurgent) from all causes, due to military action, the insurgency, crime, increased disease, increased accidents, and so on.
The UNDP Report period is the shortest, at one year. The estimate is a measure of all deaths (civilian, military and insurgent) that are ‘war-related’ in the narrowest sense. The choices in the section of the questionnaire devoted to ‘mortality’ are: ‘disease, traffic accident, war-related deaths, during pregnancy/childbirth/within 40 days of childbirth, other (specify)’. [2]
Thus the ‘war-related deaths’ estimate specifically excludes all deaths caused by raised levels of disease, maternal or infant mortality, or accidents due to the war and occupation, or crime-related deaths (which are presumably filed under ‘other (specify)’). (Incidentally, violent crime including murder is the subject of an entirely separate Question PM06 on page 16 of the UNDP questionnaire, 32 pages away from the Mortality section.)
Note that both the Lancet and the UNDP estimates are based on sampling and estimation, using known and proven statistical methods, not on reported deaths, as in the case of the IBC estimate.
UNDP-LANCET CONVERGENCE
If we try to match like with like, what is the figure in the Lancet Study data most similar to the UNDP estimate? Using figures given in the Lancet Study, Australian academic Tim Lambert derives a figure of 33,000 excess Iraqi deaths due to military action by insurgents and US-led forces in the period up to September 2004 [3].
Each death recorded in the Lancet study represents 3,000 deaths in Iraq in the period under consideration. Outside Fallujah, there were nine deaths caused by coalition forces and two by the insurgents. By simple multiplication, this results in 33,000 ‘war-related’ deaths according to the UNDP definition of this term, for the period March 2003-September 2004.
This compares to the UNDP estimate for narrowly-war-related violent deaths of 24,000 for the period March 2003 to April 2004 (13 months compared to the Lancet’s 18 months). If we crudely scale up the UNDP figure to take account of the longer Lancet time period, we reach a figure (33,000) which is exactly the Lancet-derived figure of 33,000 violent deaths due to military action.
So, rather than there being a conflict between the two survey-based estimates, as often suggested by officials and apologists for the war, we see mutual confirmation.
SURVEY-SURVEILLANCE CONVERGENCE
What about the difference between the UNDP and Lancet estimates, which are based on surveys, and the IBC estimate, which is based on ‘surveillance’ or reporting?
The IBC Report estimated that around 25,000 civilians died because of coalition, insurgent, criminal or ‘other’ (perhaps sectarian) violence. But this was for the two years after the invasion, a period longer than either the Lancet or UNDP estimates.
If we take a nearly comparable figure from the IBC record, the recorded maximum civilian death toll as of Saturday 7 February 2004 was 10,079 [4], compared to the UNDP estimate of 24,000 civilians, soldiers and insurgents in the period up to April 2004.
On first principles, as IBC itself readily concedes, a surveillance-based system like IBC will produce lower figures than survey-based estimates like UNDP and the Lancet. The reason is simple. Not all deaths will be reported in the media, or recorded at morgues. In addition, this proportion is likely to decrease as the severity of violence increases.
When there is mounting violence, there is less access to roads and health facilities, and fewer bodies will reaching morgues, and fewer journalists will make journeys to areas of conflict. Those responsible for surveillance studies have fully recognised this fact.
Iraq Body Count note in their Report, ‘even our max[imum death toll] figure is likely to under-represent the full toll, given that not every death is officially recorded or reported’.
When supporters of the war – the British Government, in particular – claim that the IBC estimate represents the total level of war-related deaths, they are misrepresenting the IBC itself.
The fact that the IBC estimate is well below that of the UNDP/Lancet estimates of violent deaths does not indicate a fundamental contradiction. The IBC estimate sets an irrefutable base line of reported violent civilian deaths. The UNDP/Lancet estimates of reported and unreported violent deaths are bound to be larger, and they are larger. Note that both the UNDP and Lancet estimates also include violent deaths of insurgents and Iraqi soldiers, though the UNDP figure does not include deaths due to violent crime, which the IBC estimate does include. (See tables for a graphical representation of these differences.)
The IBC surveillance method and the Lancet/UNDP survey methods produce results which are pointing in the same direction, rather than diverging.
CONVERGENCE, NOT CONTRADICTION
In summary, when we take account of the different categories being measured by the UNDP’s Iraqi Living Conditions Survey, the Lancet Study, and the Iraq Body Count reported death toll, and the different time periods that they measure, the three estimates seem to be mutually reinforcing rather than mutually contradictory. The Iraq Body Count estimate is based on credible, multiply-sourced reports, and is the only national estimate focussed on the violent deaths of civilians due to military action and criminal murder during the invasion and occupation of Iraq. The immense value of the IBC estimate is that it is an irrefutable baseline figure, described as an under-estimate by its authors.
The other two estimates include conventional and unconventional Iraqi combatants in their reckonings. The Lancet Study is based on the only national survey so far that has focused on the total Iraqi death toll associated with the war, including increases in accidents, crime, disease and other indirect effects of the conflict. That is its unique contribution.
The UNDP survey is the largest national survey conducted of living conditions inside Iraq, which confirms the validity of the Lancet estimate (while still being likely to be an under-estimate for the design reasons discussed above). This confirmation of the scale of the death toll is the particular contribution that this huge project has made to the debate.
In summary: there is no inherent contradiction between the three best-known estimates of the war-related death toll in Iraq. Each estimate can be used confidently, so long as it is made clear that it is an estimate over a particular time period, assessing a particular category of deaths due to the conflict, in its own particular way.
Interestingly, as Les Roberts, lead researcher for the Lancet Study, has pointed out, both the IBC and Lancet figures demonstrate that US-led forces have killed more Iraqis than anti-occupation forces. [5] The occupation is part of the problem, not part of the solution.
Those of us concerned at the enormous loss of life due to the invasion and occupation of Iraq owe the analysts involved – in Iraq Body Count, in the Lancet Study team, and in the UNDP/FAFO/COSIT survey – an enormous debt.
We also owe it to them not to misrepresent their work.
Milan Rai 14 October 2005
Table 1 Comparing the IBC, Lancet and UNDP estimates: Unadjusted
Table 2 Comparing the IBC, Lancet and UNDP estimates: Violent Deaths
Table 3 Comparing the IBC, Lancet and UNDP estimates: Violent Deaths Crudely Adjusted For UNDP Period
NOTES
- IBC Report (pdf)
- Lancet Study
- UNDP Report (pdf)
[1] Iraq Living Conditions Survey Volume II: Analytical Report, page 50. The report can be downloaded from http://www.iq.undp.org/ILCS/population.htm
[2] Subsection HM: Mortality, FAFO, ‘Iraq Multiple Indicator Rapid Assessment’ Questionnaire, page 48.
[3] Tim Lambert, ‘Lancet Study Vindicated’, 14 May 2005
[4] IBC, ‘Civilian deaths in “noble” Iraq mission pass 10,000: We need a tribunal to administer justice for the victims’, 7 February 2004
[5] Personal communication, 14 October 2005.