Competing accounts of why violence declined in Iraq in 2007 have shaped U.S. strategy in Afghanistan, debates about force sizing and doctrines on counterinsurgency, and academic research on the dynamics of armed conflict. Nevertheless, few scholars have attempted to test these competing accounts against one another systematically. “Testing the Surge”[1] approached this issue by combining declassified, geocoded data on violent events with information about local-level military behavior gained from an original series of seventy structured interviews with Coalition officers. This evidence allowed us to leverage the substantial variation in violence patterns across Iraq in order to evaluate causal claims. We argued that the best explanation for why violence declined in Iraq in 2007 involves a synergistic interaction between the Surge and the Sunni Awakening: both were necessary but neither was sufficient, while other explanations (including the dynamics of sectarian cleansing) cannot account for local or national violence trends.
H-Diplo | ISSF
Authors’ Response to H-Diplo | ISSF Article Review 21
H-Diplo/ISSF Article Review Editors: James McAllister and Diane Labrosse
H-Diplo/ISSF Web and Production Editor: George Fujii
Commissioned for H-Diplo by James McAllister
Stephen Biddle, Jeffrey Friedman, and Jacob Shapiro. “Testing the Surge: Why Did Violence Decline in Iraq in 2007?” International Security 37:1 (Summer 2012): 7–40. DOI: 10.1162/ISEC_a_00087. http://dx.doi.org/10.1162/ISEC_a_00087
John Hagan, Joshua Kaiser, and Anna Hanson. “Correspondence: Assessing the Synergy Thesis in Iraq” International Security 37:4 (Spring 2013): 173-198. DOI: 10.1162/ISEC_c_00118. http://dx.doi.org/10.1162/ISEC_c_00118
Reviewed by Austin Long, Columbia University
Published by H-Diplo | ISSF on 10 April 2013 and on 22 May 2013
PDF (original review)- https://issforum.org/ISSF/PDF/ISSF-AR21.pdf
PDF (this response)- https://issforum.org/ISSF/PDF/ISSF-AR21-Response.pdf
Authors’ Response by: Stephen Biddle, George Washington University
Jeffrey A. Friedman, Harvard University
Jacob N. Shapiro, Princeton University
Competing accounts of why violence declined in Iraq in 2007 have shaped U.S. strategy in Afghanistan, debates about force sizing and doctrines on counterinsurgency, and academic research on the dynamics of armed conflict. Nevertheless, few scholars have attempted to test these competing accounts against one another systematically. “Testing the Surge”[1] approached this issue by combining declassified, geocoded data on violent events with information about local-level military behavior gained from an original series of seventy structured interviews with Coalition officers. This evidence allowed us to leverage the substantial variation in violence patterns across Iraq in order to evaluate causal claims. We argued that the best explanation for why violence declined in Iraq in 2007 involves a synergistic interaction between the Surge and the Sunni Awakening: both were necessary but neither was sufficient, while other explanations (including the dynamics of sectarian cleansing) cannot account for local or national violence trends.
In his review for H-Diplo/ISSF, Austin Long criticizes both of the principal sources of evidence on which we base these findings.[2] For our dependent variable of violence trends, Long argues that the data we used on “Significant Activities” (SIGACTs) offer limited insight into the nature of Iraqi politics and have several potential biases. As for our independent variables on local-level military behavior, Long is concerned with the inferences we can draw from the interviews we conducted, arguing in particular that we should have relied more heavily on Iraqi sources. We respond to each of these critiques below, explaining why the drawbacks Long identifies with the SIGACTs data do not undermine our analysis, and why Long’s objections to our interview evidence rest largely on a misinterpretation of how we use that evidence.
First, let us consider the SIGACTs. The principal advantage of using these data is that they record individual violent events, geocoded to specific locations. There are roughly 200,000 SIGACTs on record between 2004 and 2008, allowing researchers to examine violence trends across almost any possible subset of time and space. Analyzing this local-level variation is critical to our analysis, because it allows us to assess how violence rose and fell in the specific areas of operation for which the officers we interviewed were responsible. More broadly, leveraging spatiotemporal variation is critical for examining whether violence unfolded in the manner predicted by different hypotheses about what drove the dynamics of armed conflict in Iraq. The SIGACTs data provide the best resource we know of for analyzing this variation.
Though we noted several potential drawbacks to these data,[3] Long raises a different issue, arguing that there is more to Iraqi political dynamics than violence trends, and that data on violence alone thus cannot provide a complete account of the war. He argues that “violence does not necessarily tell us much about the local political dynamics that the authors claim are important”(4); he points out that a decline in violence could conceivably indicate that insurgents have gained uncontested territorial control; he notes that if violence declines in one area this could simply mean that insurgents have shifted their activities elsewhere; and he claims that because SIGACTs data do not measure the damage caused by violent events, they cannot say whether a drop in the number of attacks has been counteracted by an increase in their severity.
This is surely true. It is also irrelevant to our article, which is neither a general history of the war nor an exploration of Iraqi politics. Our paper is a focused analysis of one aspect of the war: why the violence came down in 2007. For this purpose, the SIGACTs violence data are not only appropriate, they are essential: it is the trend in these data that our analysis is intended to explain. These data do not say everything one might want to know about Iraq, but the causes of the violence reduction they present have been very controversial and constitute the research question around which the whole paper is built.
We state this scope condition explicitly, not only in the paper’s title (“Testing the Surge: Why Did Violence Decline in Iraq in 2007?”) but in the introduction (7) and then again in the conclusion (36), where we remind readers that our purpose is to explain why violence fell in 2007 but not to address various other questions, including the country’s future stability, or the degree to which the outcome met or failed to meet U.S. war aims or, one might add, the country’s internal politics per se. Our analysis tests several claims about the role of Iraqi politics in the 2007 violence reduction, but our aim is to explain the violence, not the politics – we analyze only the logical implications of others’ claims about Iraqi politics for observable violence trends. And it is precisely these violence trends that the SIGACTs data present.
In fact, questions about which elements of Iraqi politics could plausibly have driven violence trends – and which largely lagged them – drive many ongoing debates about the war. This is especially true with respect to the Sunni Awakening, which some authors have argued was mainly the reflection of a beaten insurgency switching sides in order to mitigate the losses it was sustaining at the hands of Coalition forces.[4] Others, however, argued that Sunni realignment happened before violence fell and was a necessary cause of that decline[5] – this second position is consistent with the evidence we present in “Testing the Surge,” and it is only because we separate Iraq’s violence from its politics that we can gain any traction on the issue. Since violence trends are the feature of the war that we study, the data we use for this purpose are entirely appropriate.
Long’s related argument that low levels of SIGACTs could be bad news rather than good (i.e., it could signify uncontested insurgent control) is similarly true but irrelevant. We did not write a paper on the pros or cons of reducing violence – we explain a much-discussed reduction, but we are careful to delimit our claims to exclude judgments on this outcome’s ultimate value for U.S. (or Iraqi) interests. For what it is worth, however, insurgents certainly did not emerge from the year in command of Iraq’s key terrain, and especially not in Baghdad or Anbar province, where much of the reduction in violence occurred. In some cases it is certainly possible that violence declined in one area as insurgents moved elsewhere, but the extent of our geographic coverage (and the fact that violence declined so broadly throughout the country) indicates that such shifts did not drive the decline in violence overall. And Long’s point that a declining frequency in attacks may have simply been counteracted by an increase in their intensity is also more of a hypothetical problem than an actual one – we see no indication that this occurred systematically in a way that would undermine our analysis, nor does Long offer any. If that were generally occurring, the changes in Anbar from 2006-7 would not have been accompanied by the precipitous reduction in Coalition casualties that occurred.[6]
As for our interview evidence, Long’s principal concern is that it is “almost entirely derived from the perceptions of U.S. officers.” Since “only a tiny fraction of U.S. personnel in Iraq spoke Arabic,” (5) and because U.S. personnel rotated in and out of the theater so frequently, it is unclear how much our interviewees really understood about the people they were working with (and fighting against). Long views this as “deeply problematic” for our argument. Reliance on U.S. interviewees could indeed be a problem for some uses of interview data on Iraq – but not for ours.
Long’s critique of our interviews rests largely on a misunderstanding of the role they played in our analysis. We did not ask interviewees to provide assessments of Iraqi politics, subjective perceptions of motivation, or causal analyses of why Iraq’s violence fell. Rather, we asked for factual reporting on observable events occurring in that officer’s own area of operations (AO) and which the officers were in a position to observe directly themselves. For assessing patterns of sectarian cleansing, for example, this involved asking interviewees when and where they observed Shiite militia attacks. For assessing the role of U.S. forces, this entailed questions about how many troops were present for duty, the manner in which they were deployed, and the boundaries of their assigned battlespace. For assessing the Sunni Awakening, we asked whether Sons of Iraq groups had been formed in the interviewee’s AO, and if so, when.[7]
By combining this evidence with other data, we could demonstrate that sectarian violence did not decline once intermingled areas of Baghdad had become homogenized; that where local Sunnis attempted to realign without extensive protection from Coalition forces they were generally defeated or contained by counterattacks from al Qaeda in Iraq; and that Surge forces alone were generally unable to cause violence to decline at rapid or sustainable rates until they partnered with the Sons of Iraq. Since these claims are based on facts our interviewers directly observed (such as when the local forces started cooperating with Coalition forces and not on their ability to accurately infer what was going on between Iraqis), we do not believe Long’s critiques should reduce confidence in our claims.
Here, too, it is important to note the difference between a general history of the war and a focused analysis of violence reduction. For the former, Iraqi perspectives are important in their own right for understanding the conflict holistically. For the latter, establishing key facts on the ground is the requirement for inference, and direct observation by credible sources is sufficient whether those sources are Americans, Iraqis, or others.
Long raises some additional points that are worth mentioning briefly. For example, Long criticizes the way that we portray the 2005 Sunni realignment in al Qaim as being unable to spark the kind of broad, local collaboration with Coalition forces that characterized the Awakening movement later on. Long says that this is a “serious disconnect between U.S. and Iraqi sources” because Iraqis “almost uniformly understood al Qaim to be a success” (6). This is a critique Long has made elsewhere, arguing that Sunni forces were largely capable of tackling al Qaeda with limited Coalition support, and that the Surge was thus “a tragic waste of resources.”[8] We have already responded to this argument in print, pointing out that while one can label a realignment ‘successful’ or not depending on one’s criteria, the real issue here is whether anything about al Qaim provides evidence to show that realignment without the surge could have brought about the 2007 reduction in violence. In fact al Qaim provides no such evidence:
“The outcome we seek to explain is a theaterwide reduction in violence; our claim is that the Awakening’s rapid spread across most of threatened Iraq played a critical role in this, and that it would not have happened without the Surge. None of the pre-surge realignment attempts spread [including the 2005 attempt in al Qaim], and none even survived as an organizational entity for more than six months. How does this demonstrate that realignment without the Surge would have sufficed to stabilize Iraq?”[9]
Later on, Long raises questions about how thoroughly we vetted our interview evidence, writing that when interviews are conducted they “must be used with care and checked against other sources” (7). On this point, we are in full agreement – this is among the reasons why we conducted so many interviews, and why we strove to support claims with evidence from multiple interviewees and other corroborative sources. Long identifies no examples of points where inadequate sourcing yielded error, and we are certainly aware of none. In any event, we have made recordings of our interviews publicly available, so others are free to assess our evidence themselves.[10]
Long also spends a paragraph criticizing us for overemphasizing the role of monetary incentives in explaining Sunni willingness to collaborate with U.S. forces. Yet the passage he critiques presents others’ views, not ours. The passage he cites (6) from our article reads: “Proponents of the Awakening thesis claim that violence declined in 2007 because the Sunni insurgency abandoned its erstwhile AQI allies in exchange for U.S. payments of $300 per fighter per month as “Sons of Iraq” (SOI) and a promised ceasefire” (18, emphasis added). We identify these proponents explicitly in the accompanying note, and they certainly do not include us; we suspect few other readers of our paper would describe us as supporters of the view that the Awakening alone caused Iraq’s violence reduction.[11]
In closing, we wish to echo Long’s call for more research on the dynamics of violence in Iraq. For half a decade now, debates about Iraq’s turnaround – however durable it may or may not prove to be – have essentially been stuck in first gear, with authors articulating various explanations without systematically pitting their respective causal power head-to-head. Some authors have even given up for lost the idea of sorting through these claims, saying that since Iraq is just one case, it is fundamentally impossible to draw causal inferences.[12]
At the very least, we hope to have made clear that this sort of analytic pessimism is misplaced. The SIGACTs data identify variation in Iraq’s violence trends across almost any range of time and space that scholars wish to examine. Gathering information on the independent variables that might explain this variation is much harder, but we showed that it is possible to make headway on this by conducting specially-designed, structured interviews. By combining this evidence in “Testing the Surge,” we demonstrated that U.S. policy played an important role in helping to reduce violence in Iraq, but only with a level of assistance from local forces that cannot necessarily be expected elsewhere. We did not claim that this was the last word on why violence declined in Iraq in 2007. Despite Long’s reservations, we hope that scholars will see our evidence and methodology as something on which to build.
© Copyright 2013-2015 The Authors.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 United States License.
Notes
[1] Stephen Biddle, Jeffrey A. Friedman, and Jacob N. Shapiro, “Testing the Surge: Why Did Violence Decline in Iraq in 2007?” International Security 37:1 (Summer 2012). Subsequent references cited parenthetically.
[2] Austin Long, H-Diplo | ISSF article review 2013: 4, http://www.h-net.org/~diplo/ISSF/PDF/ISSF-AR21.pdf. In another recent critique co-authored with Jon Lindsay, Long raised several additional points to which we have responded separately. See “Correspondence: Assessing the Synergy Thesis in Iraq,” International Security 37:4 (Spring 2013).
[3] See our discussion of these data on pp. 11-13 of our article, along with broader discussions in Eli Berman, Jacob N. Shapiro, and Joseph Felter, “Can Hearts and Minds Be Bought? The Economics of Counterinsurgency in Iraq,” Journal of Political Economy 119:4 (August 2011), and in the documentation for the SIGACTs data which are available at www.esoc.princeton.edu.
[4] See the references cited in note 2 of the paper.
[5] See the references cited in notes 3 and 5 of the paper.
[6] For casualties data see iCasualties.org.
[7] The questionnaire we used to structure our interviews has been made available along with recordings of those interviews, and it is also included with our online supplementary materials.
[8] See “Correspondence,”181.
[9] Ibid., 191-192.
[10]Audio files of our interviews have been deposited at the U.S. Army Military History Institute archive in Carlisle, Pennsylvania.
[11] We cite no interviews of our own on this point, and are at a loss to understand why Long believes this sheds doubt on the general utility of our interview evidence base.
[12] See note 21 of our paper.