Source: Znet by Stephen Soldz November 03, 2004

[corrected 11/5/04*]

One justification for the Iraq war was to remove the barbarous regime of Saddam Hussein, thereby freeing Iraqis from the threat of death at the hands of his regime. Yet, from the first days of the war, accounts have surfaced of Iraqi civilian deaths at the hands of "coalition forces" and from the increased crime and chaos that have swept much of the country.

The United States and its British and other allies claim they do everything in their power to prevent civilian casualties. Yet, repeatedly accounts have appeared of civilians dying at checkpoints, in passing American convoys, in house searches, and in the relentless bombing campaigns that are allegedly precision strikes on known terrorist hideouts. Reports have also surfaced about increased murder rates.[1] If the rates of Iraqi civilian deaths increased significantly since the invasion, it would undercut the last remaining rationale for the war.

So, how many Iraqis have died since the invasion in March 2003 and the subsequent occupation and war? The United States has repeatedly insisted that it doesn't keep track of civilian deaths. In the infamous words of General Tommy Franks, "We don't do body counts"[2] (though, claims remain that the US does do secret body counts[3]). Furthermore, when the Iraqi health Ministry attempted to count civilian deaths, they were summarily ordered to stop by the US occupation authorities.[4]

While the US military and occupation authorities may profess a lack of interest in Iraqi civilian deaths, decent people around the world are concerned to know whether Iraqis have died post-invasion at rates that would substantially undermine any alleged humanitarian benefit of the war.

Perhaps the best known estimate of civilian deaths from the fighting is that of the Iraq Body Count project.[5] This British-based group of researchers has systematically examined the western press and collated all accounts of civilian casualties. They tabulate all deaths that are independently reported by two sources. Based on this rigorous methodology, they estimate civilian casualties from the invasion until October 29, 2004 at between 14,181 and 16,312. Other estimates have come from the Brookings Institution[6] (between 15,200 and 31,400 "killed as a result of violence from war and crime between May, 2003 and September 30, 2004." Some of their estimates are based on Iraq Body Count data), and the Iraqi People's Kifah[7] (through a household survey they identified 37,000 deaths between March and October 2003).

With the exception of the People's Kifah estimates, which might be considered suspect as they are an anti-occupation organization and they have published no details about their methods (and which only covers the first eight months of war and occupation), these estimates largely are based on western press accounts. As is acknowledged by Iraq Body Count, such accounts likely underestimate deaths as many, perhaps most, battles and other military actions, and resultant Iraqi deaths, are often not reported unless coalition forces suffered casualties.[8] Additionally, in recent months western reporters have been unable to move about Iraq independently, meaning that even such high-profile claims of mass civilian deaths from US bombing as the killing by US bombing of upwards of 45 Iraqis at a wedding party in the town of Mogr el-Deeb in May[9] could not be independently verified. Thus, all previous estimates of Iraq civilian deaths since the invasion are probably on the low side.

In order to address the question of how many Iraqi deaths have occurred, a team of public health researchers from the Johns Hopkins Bloomberg School of Public Health, the Columbia University School of Nursing, and the College of Medicine at Al-Mustansiriya University in Baghdad undertook an epidemiologic survey of "excess Iraqi" deaths since the March, 2003 invasion.[10] This high-powered research team combined epidemiologic expertise with a background in studying people in disaster and emergency situations and an in-depth knowledge of Iraq. Members of the team have carried out research and consulting in many parts of the world, including Iraq, sub-Saharan Africa, and Eastern Europe and have worked with such organizations as the World Health Organization and the U.S. Centers for Disease Control.[11-13]

The results of the research by this team have surprised many. The researchers estimated that there were 98,000 more deaths in the 18 months after the invasion than there would have been if Iraqis had died at the same rate as during the 15 months prior to invasion.

This report has stirred up quite a storm. The British Government has challenged it.[14] The Washington Post quoted a "senior military analyst" at Human Rights Watch as saying "The methods that they used are certainly prone to inflation due to overcounting.... These numbers seem to be inflated".[15] On the other hand, numerous critics of the war and continuing US occupation have latched on to these results as further evidence of the destructiveness of the US-British invasion.[16-20]

So how should one evaluate this study? While it may be tempting for those outraged by this war from the beginning to accept these results uncritically, those of us who also believe our politics should be guided by facts and a search for the truth should approach these findings with caution. All controversial research needs to be carefully examined for evidence of methodological problems or other flaws.

As an activist who is also a psychological and public health researcher with experience conducting prevalence surveys, and a teacher of statistics and social research methods, I'd like take a look at this study to help readers judge it for themselves. Researchers early learn the folly of latching onto results simply because they support our preexisting beliefs.

First I'll briefly describe the methodology of the study. Then I'll evaluate it.

The Study Methodology

The researchers used a traditional epidemiologic technique called a clustered sample survey. Without getting into technical details, the country was divided into a number (33 in this instance) of subgroups and a community was randomly selected from each cluster. In each community, Global Positioning System (GPS) devices combined with random numbers were used to select a particular point in the community. Then the nearest 30 household were surveyed; these 30 households are referred to as a cluster.

The important points in the previous paragraph are that the procedure was systematically designed to represent the population of the entire country and that random numbers were used whenever choices had to be made. The combination of a systematic procedure along with random numbers when choices must be made are techniques used by survey researchers to avoid any systematic bias due to the researchers intentionally or unconsciously selecting certain communities or households. If carried out successfully, these procedures allow a fairly small sample to accurately represent the population of an entire country. Thus, while we may have questions about the accuracy of election polls, they are usually at least in the ballpark of representing the vote-getting potential of a candidate. Seldom does a candidate get 10 percentage points more or less than an immediate pre-election poll.

The researchers ended up with information on 7,438 people pre-invasion and 7,868 people post-invasion. These people represented 988 households in 33 clusters.

After households were selected, the interviewers, Iraqis (five of the six of whom were medical doctors), had a carefully designed procedure to ask about the population in the household from January 1, 2002 (about 15 months before the invasion) on. Any changes in people living in the household were carefully noted so that births, deaths, and changes in residence could be identified. In order to control for potential bias in recall, the interviewers asked to see death certificates in a certain predetermined fraction of the households; in the vast majority of instances these were provided, or their absence had a reasonable explanation. Thus, people's accounts of deaths of family members appeared to be accurate and showed no evidence of bias.

The researchers then used standard statistical techniques to estimate the rate at which Iraqis died before the invasion and at which they died post-invasion. They report a number of checks on their data analysis, all of which supported the reasonableness of their analyses.

Discussion and Critique of Results

By the luck of the draw, the Iraqi city of Falluja was included in their sample. The researchers found an enormous number of deaths in that besieged city, much higher than in any over cluster they sampled. Researchers call such an observation an outlier and have developed a number of techniques for dealing with them. These researchers used the most conservative approach: namely, they presented many of their data excluding Falluja. With this exclusion they found that mortality (deaths) rose from 5.0 deaths per 1000 people per year to 7.9 deaths.

Public health researchers usually report such data as the relative risk of death post-invasion, compared to pre-invasion. A relative risk greater than 1.0 means that more people died after the invasion than in an equivalent time period before the invasion. With Falluja included, the relative risk post-invasion was 2.5, whereas with Falluja excluded it was 1.5. so they estimated that roughly 50% more people died post-invasion than had died in the same time period before.

The researchers then estimated how many more people died in the post-war period than would have been expected from the pre-war mortality (death) rate. The figure they came up with was 98,000.

In interpreting this estimate, the researchers, like all good researchers, give an estimate of its precision. This is done by providing what are known as 95% confidence intervals. These mean that, 95% of the time, the true value is between the lower and upper limits of the confidence interval (remember, these are estimates). Given the nature of the study, the confidence intervals for this 98,000 estimate are broad, from 8,000 to 194,000. Thus, it is 95% likely that the true rate of excess deaths is between 8,000 and 198,000.

From these results alone, it could be that the true value is nearer the lower estimate of 8,000 than at the middle or upper end of this interval. On this basis, one could conclude that the headlines about 100,000 excess deaths are unwarranted. But is that all there is to it? Fred Kaplan in Slate seems to think so.[21] A closer examination of this crucial issue is needed, however.

In contradistinction to the beliefs of the poor souls suffering through statistics classes, when researchers analyze data, they don't just run a set of canned statistical procedures on their data to derive an answer. Rather, they use statistics as part of a process of examining data to try and extract meaning, to tell an empirically justifiable story about what is happening in the world the data came from. Similarly, when we researchers read a research paper, we examine the data presented to form a judgment of whether they are consistent with the authors' interpretations.

So let's examine the data in this article. Of course, we don't have the raw data in the researchers' computers. But we do have the wealth of data they present in the article. These data go considerably beyond the estimates of relative risk and excess mortality.

One issue to examine is: are the deaths localized in one cluster that may be atypical of the country as a whole? Of course, the death rate in Falluja was far higher than that for the rest of the country. (More on this below). The authors provide information on the distribution of deaths in their Figure 1, which illustrated the pre- and post-invasion death rates in the randomly selected clusters in 11 of Iraq's Governorates. One of these Governorates is Falluja, with mortality rates over 10 times higher than the rest of the country. For another Governorate, Sulaymaniya, the post-invasion rate was considerably lower than the pre-invasion rate, while for two others, Karbala and Ninawa, the rates were essentially equal. Now, Sulaymaniya is in the Kurdish north of the country, under the firm control of the Kurdistan Regional Government, where there is no insurgency and no Coalition attacks. The other seven Governorates all exhibit considerable increases in mortality rates post-invasion. Thus, 8 of 10 sites in non-Kurdish Iraq exhibit a pattern of increased mortality.

These data help explain the large confidence intervals (mentioned above) around the mortality estimates. Most likely a major reason the confidence intervals are so large is the inclusion of the Kurdish region and the reversed pattern in Sulaymaniya. It would have been reasonable to have analyzed the data for non-Kurdish Iraq separately, which would undoubtedly have resulted in a higher estimated relative risk and confidence intervals for estimated deaths with a considerably greater lower bound (above the 8,000 reported in this paper). The fact that the authors did not do so, as well as their exclusion of Falluja from the excess deaths estimate, is a sign that they were conservative in their data analysis; that is, they did not make post hoc (knowing how they would affect the results) decisions in order to inflate the mortality estimate.

In addition to noting that the Kurdish region is included in analyses, it is important to examine areas of the country that were not included. By excluding Falluja from the estimate of excess deaths, the authors are in fact excluding all of Anbar Governorate. Now, Anbar Governorate includes both Falluja and Ramadi, as well as some of the areas near the Syrian border where "foreign insurgents" are alleged to have entered Iraq. An examination of news accounts from Iraq, or of US casualties[22] will show that a large proportion of all fighting in Iraq occurs in Anbar, as this Governorate has been in the forefront of fighting since the first month of the occupation.[23] Thus, excluding this Governorate could easily lead to a serious underestimation of Iraqi deaths.

It should also be noted that Najaf, scene of fierce fighting and massive US bombing in April and August 2004,[24, 25] was also not sampled (see Table 1 in the paper). Further, while the Baghdad slum known as Sadr City, a stronghold of Moqtada al-Sadr -- and site of furious fighting and US bombing for months -- was included in the sample, the area sampled there "by random chance was in an unscathed neighborhood with no reported deaths from the months of recent clashes" (p. 7). Thus, many of the areas with the most intense fighting were either excluded from key analyses (Falluja) or were not sampled (Ramadi, Najaf), or while part of a sampled area did not have their violent section sampled. This fact suggests that the mortality rate for the country as a whole may, in fact, be far higher than the authors estimate (the authors in fact state this possibility). In any case, these exclusions, combined with the inclusion of the Kurdish region with the Sulaymaniya outlier, increase the likelihood that the rate of excess deaths is NOT near the lower level of the stated confidence interval of 8,000.

One of the surprising findings is that, of the 61 violent deaths attributed to Coalition forces in this study, only three involved actions by ground forces. The other 58 deaths were attributed to "helicopter gunships, rockets, or other forms of aerial weaponry" (p. 7). These results strongly suggest that the air war has been even more intense than is often suggested.

On the other hand, it is possible that at least a few of these deaths may actually be due to insurgent actions. There have been a number of reports of Iraqis blaming American helicopters or missiles for attacks actually carried out by insurgents.[26-28] Thus, it is possible that not all the violent deaths attributed to Coalition actions may be due directly to those actions. Of course, one could still argue that the US and its allies, by invading Iraq on false pretenses and continuing a long-term occupation of the country, bear primary moral responsibility for deaths occurring as a consequence. Nonetheless, one should exercise some caution in attributing all these deaths to Coalition actions. It should, however, also be remembered that even official statistics from the Iraqi Ministry of Health (before the US-installed government ordered them to stop releasing statistics on Iraqi casualties[29]) document that the majority of reported casualties are due to Coalition actions, rather than those of the insurgents.[30]

Conclusion

This study is an extremely well-conducted and analyzed piece of research. Like most high-quality research, it has potential limitations and the paper's Discussion section details possible interferences with the accuracy of the results. The authors argue convincingly that none of these limitations invalidate their basic findings of high excess deaths following the invasion and occupation of Iraq. In fact, they argue, based on arguments somewhat different from those I present here, that the real number of excess deaths may be even higher than their 98,000 estimate.

So, have excess 100,000 Iraqis died since the invasion? I don't know for sure. But this study convinces me that it is extremely likely that many tens of thousands of Iraqis have died, far more than the Iraqi Body Count estimate that I had previously relied upon. Noted Middle East scholar Juan Cole came to a similar conclusion.[31]

We researchers never consider a single study to be the definitive word on a topic. We always like to see a number of studies, using somewhat different methodologies and carried out by researchers with different biases. The authors clearly recognize this desirability and do not present their study as the last word on Iraq mortality rates. (See the excellent interview with one of the study authors from the New Republic Online.[32]) Near the end of their paper they call for "confirmation by an independent body such as the ICRC, Epicentre, or WHO" (p. 7). The British-based NGO, Medact has endorsed the call for independent investigations.[33] Medact further points out that this study only examines Iraqi deaths and that "experience suggests that at least three times as many people are injured as are killed in conflict." Thus, it is likely that hundreds of thousands of Iraqis have been wounded in the last 18 months.

In the absence of this confirmation, this study remains the best estimate of Iraq deaths. Its finding are truly horrifying. Recent reports indicate that the US is placing a far greater reliance on air power as a way of reducing Coalition casualties.[34] If this study's findings are at all accurate, the result of these policies will be even higher Iraqi civilian casualties. The continued US war in Iraq cannot be justified on any conceivable humanitarian grounds when many tens of thousands of Iraqis are being killed and many more injured. Surely, this study should be a wake up call for all those, regardless of their opinions about the original justifiability of the war, who sincerely are concerned about the fate of the Iraqi people. The looming attacks on Falluja and Ramadi suggest that, in the absence of world outrage restraining this Coalition action, the death and injury toll will soon be rising far higher.

* [An earlier version of this article wrongly attributed the location of the Governorate of Ninawa to the Kurdish region. Although I gave no numerical estimates, this correction would make the confidence interval slightly larger and the case for excluding the Kurdish region stronger.]

References

1. Fisk R. Baghdad is a city that reeks with the stench of the dead. Independent. July 28, 2004.
2. Epstein E. How many Iraqis died? We may never know. Some observers are pressuring Pentagon to put forth an informed estimate. San Francisco Chronicle. May 3, 2004.
3. Whitaker R. Pentagon Suppresses Details of Civilian Casualties, Says Expert. Independent. October 31, 2004.
4. Price N. Iraq's Health Ministry ordered to stop counting civilian dead from war. Associated Press. December 10, 2003.
5. Iraq Body Count. Iraq Body Count [Web site]. Accessed October 29, 2004.
6. O'Hanlon ME, Albuquerque ALd. Iraq Index: Tracking variables of reconstruction & security in post-Saddam Iraq. Brookings Institution [pdf version of report]. October 29, 2004. Accessed October 30, 2004.
7. Janabi A. Iraqi group: Civilian toll over 37,000. Aljazeera. July 31, 2004.
8. Fisk R. Secret slaughter by night, lies and blind eyes by day. Independent. September 14, 2003.
9. Faramarzi S. Video Contradicts US Military, Shows Iraq Wedding Celebration. Associated Press. May 24, 2004.
10. Roberts L, Lafta R, Garfield R, Khudhairi J, Burnham G. Mortality before and after the 2003 invasion of Iraq: cluster sample survey. Lancet. October 30, 2004.
11. Presenter Biographies [Web page].
12. Richard Garfield, DrPH, RN: Henrik H. Bendixen Clinical Professor of International Nursing. Columbia University School of Nursing [Web page]. Accessed October 30, 2004.
13. Faculty: Gilbert Burnham. Johns Hopkins Bloomberg School of Public Health [Web page]. October 30, 2004.
14. Wintour P, Norton-Taylor R. No 10 challenges civilian death toll. Guardian. October 30, 2004.
15. Stein R. 100,000 Civilian Deaths Estimated in Iraq. Washington Post. October 29, 2004.
16. Laurance J, Brown C. Revealed: War has cost 100,000 Iraqi lives. Independent. October 29, 2004.
17. Tirman J. 100,000 Dead In Iraq. AlterNet. October 30, 2004.
18. Chelala C. The Most Tragic Victims of the Iraq War. CommonDreams. October 30, 2004.
19. Burgess R. By Our Hand And In Your Name: 100,000 Dead Iraqi's. Information Clearing House. October 30, 2004.
20. Johnson P. 100,000 Civilian Women And Children Killed In Iraq By Coalition Forces. Scoop. November 2, 2004.
21. Kaplan F. 100,000 Dead -- or 8,000: How many Iraqi civilians have died as a result of the war? Slate [Web magazine article]. October 29, 2004.
22. Iraq Coalition Casualty Count. Iraq Coalition Casualty Count [Web page]. November 1, 2004.
23. Soldz S. Iraq: What Went Wrong? ZNet. July 26, 2004.
24. Burns JF, Berenson A. 8-Day Battle for Najaf: From Attack to Stalemate. New York Times. August 18, 2004.
25. Wilson S. Over 60 Days, Troops Suppressed an Uprising. Washington Post. June 26, 2004.
26. Rosen N. Everyday Chaos. MotherJones.com. April 22, 2004.
27. Fairweather J. Iraqis blame US for bomb attacks on army recruits. London Telegraph. Febrauary 12, 2004
28. Martinkus J. Blood, smoke and the bitterness of blame. The Age. March 3, 2004.
29. Onishi N. How Many Iraqis Are Dying? By One Count, 208 in a Week. New York Times. October 19, 2004.
30. Youssef NA. Forces linked to more deaths than insurgents: Operations are killing twice as many civilians as militants are, statistics show. Knight Ridder Tribune News. September 25, 2004.
31. Cole J. US Has Killed 100,000 in Iraq: The Lancet. Juancole.com [Web blog entry]. October 29, 2004.
32. Ackerman S. Here we have a conflict going on in a densely populated urban setting and populations are at risk. New Republic Online. October 28, 2004.
33. McElderry E. Iraqi toll. Guardian. November 1, 2004.
34. Krane J. Copters maxed in counterinsurgency war. Associated Press. November 1, 2004.

Stephen Soldz (), a psychoanalyst and researcher, is a faculty member at the Institute for the Study of Violence of the Boston Graduate School of Psychoanalysis. He is a member of Roslindale Neighbors for Peace and Justice and founder of Psychoanalysts for Peace and Justice, and maintains the Iraq Occupation and Resistance Report web page.