RJISSF
H-Diplo|RJISSF

Forum 15 on “Symposium on Counterfactual Analysis”58 min read

The essays by Niall Ferguson and Benjamin Mueller are welcome commentaries on the Security Studies “Symposium on Counterfactual Analysis.” The symposium was one of four collections published in the journal in 2015 and 2016, each of which sought to refresh scholars’ understandings of a particular approach to qualitative and multi-method inquiry. The symposia followed from workshops co-organized by Syracuse University’s Center for Qualitative and Multi-Method Inquiry, and the University of Virginia’s Miller Center of Public Affairs. The four symposia—on research transparency,[1] process tracing,[2] counterfactual analysis,[3] and multi-method research[4] —largely followed the same format: a longer essay with the author’s view of the state-of-the-art, and three shorter commentaries.

H-Diplo | ISSF Forum, No. 15 (2017)

Forum Editors:  Thomas Maddux and Diane Labrosse
H-Diplo/ISSF Web and Production Editor:  George Fujii
Commissioned by Thomas Maddux
Introduction by Colin Elman

H-Diplo/ISSF Forum on “Symposium on Counterfactual Analysis” in Security Studies 24:3 (July-September 2015): 378-430. DOI: http://dx.doi.org/10.1080/09636412.2015.1070623.

Published by H-Diplo/ISSF on 10 January 2017

Permalink:  https://issforum.org/forums/counterfactuals
Shortlink:  http://tiny.cc/ISSF-Forum-15
PDF URL:  https://issforum.org/ISSF/PDF/ISSF-Forum-15.pdf

Contents

© Copyright 2017 The Authors

Introduction by Colin Elman, Syracuse University

The essays by Niall Ferguson and Benjamin Mueller are welcome commentaries on the Security Studies “Symposium on Counterfactual Analysis.” The symposium was one of four collections published in the journal in 2015 and 2016, each of which sought to refresh scholars’ understandings of a particular approach to qualitative and multi-method inquiry. The symposia followed from workshops co-organized by Syracuse University’s Center for Qualitative and Multi-Method Inquiry, and the University of Virginia’s Miller Center of Public Affairs. The four symposia—on research transparency,[1] process tracing,[2] counterfactual analysis,[3] and multi-method research[4] —largely followed the same format: a longer essay with the author’s view of the state-of-the-art, and three shorter commentaries.

As with all methodological discussions, how scholars approach counterfactual analysis is embedded in broader beliefs about what knowledge is and how it is made. Different research communities in political science and history make commitments to comply publicly with their respective community’s rules. Scholars only earn their approach’s respective warrants if they understand and follow its rules.[5] Because communities divide along epistemological boundaries, they play distinctly different games and so follow different rules.[6] What appears to be the same technique, in this instance counterfactual analysis, may be making dissimilar types of contribution depending on the framework it is attached to and motivated by.

Jack Levy’s lead article, “Counterfactuals, Causal Inference, and Historical Analysis,” is written from a mainstream (at least, in political science) position that a central purpose of scholarship is to make valid causal inferences. To be sure, Levy’s essay includes a brief typology of different kinds of counterfactuals, and the various contributions that counterfactual analysis offers. Nevertheless, the main premise of Levy’s essay is that counterfactual analysis is helpful to the extent that it supports the fundamental mission of making causal arguments. This is most obviously the case for necessary conditions, but Levy argues that it also holds true for other types of causal claims. Levy notes that counterfactual analysis is most helpful when it is used in conjunction with other qualitative methods, notably process tracing and small-n comparison.

The core of Levy’s essay is a discussion of a set of criteria for evaluating counterfactual analysis. Levy suggests that scholars should pursue: clarity; obedience to the minimal re-write rule; contentability; consistency with well-established theoretical generalizations; historical accuracy; temporal proximity; attention to reversionary or redirecting counterfactuals; and (drawing on Frank Harvey’s earlier work)[7] the systematic use of comparative counterfactual analysis.

Ned Lebow’s commentary, “Counterfactuals and Security Studies,” begins by noting that while Levy’s essay is a thoughtful and reasonable treatment, it is exclusively concerned with ‘positivist’ counterfactual analysis aimed at evaluating causal claims. Because Lebow has different epistemological commitments, however, he sees a much broader range of uses for the technique. The core of Lebow’s argument is that the nature of the social world guides how we should study it. Causation is a convenient fiction, a human invention, not an intrinsic feature of the world. Lebow believes we should celebrate contingency, and the idiosyncrasies and uniqueness of cases, and consequently he is unconvinced that the machinery of mainstream political science can produce helpful answers. He suggests an alternative two-step approach. For Lebow, cases are best understood in terms of the reasons actors have for behaving as they did. Lebow then combines this constructivist ontology with a call to map out what he describes as “inefficient causation.” In this context, counterfactuals play a key role in laying out different causes and outcomes. Envisioning this world of possibility allows scholars to synthesize their general (albeit flawed) understandings of how the social world works with the particular details of the case at hand.

Frank Harvey’s essay, ““What If” History Matters? Comparative Counterfactual Analysis and Policy Relevance” follows Levy’s focus on mainstream understandings in political science of small-n counterfactual analysis, but pays special attention to Harvey’s particular contribution to that toolkit: the use of comparative counterfactuals. Harvey’s central point is that counterfactual analysis always involves a three-way comparison. Traditional small-n counterfactual analysis typically compares (a) the world as it was with the asserted necessary condition present and the outcome occurring; and (b) the world as it might have been with necessary condition absent and the outcome missing. Harvey argues that such analyses often pay insufficient attention to (c) the world as it might have been with the asserted necessary condition absent and the outcome still present. To put it in terms of the case Harvey covers in his well-known book, scholars pay too much attention to comparing (a) President Bush è war, and (b) President Gore è peace. Harvey argues that careful counterfactual analysis shows that (c) President Gore è war is the more plausible counterfactual. In his commentary, Harvey suggests that focusing on the first counterfactual produces a misdiagnosis of the problem, and hence guides policymakers to unhelpful course corrections. By contrast, comparative counterfactual analysis leads to more accurate and more helpful conclusions.

Frank Gavin’s commentary, “What If? The Historian and the Counterfactual” is written from an historian’s perspective, and takes a view of counterfactuals that is simultaneously sympathetic and skeptical. Again, as with the preceding three essays, Gavin is explicit in couching his discussion of counterfactuals in a broader view of the scholarly enterprise. Gavin suggests that while political scientists spend too much time talking about methods, historians are insufficiently attentive. He suggests that the three essays provide discussions that will be useful for historians. Gavin closes his essay by noting five areas where further investigation is likely to be helpful. First, he echoes Levy’s point that causes closest to the outcome are not necessarily the ones deserving attention. Second, Gavin encourages scholars using counterfactual analysis to go beyond thinking vertically, over time, and add what he calls horizontal causality, involving interconnected events occurring in different places at the same time. Third, he warns that scholars using counterfactuals should be careful not to let the method bias them towards exogenous and accidental factors that are easily manipulated in thought experiments, at the expense of harder to change complex or structural variables. Fourth, and along similar lines, Gavin worries that counterfactual analysis should not bias researchers to focus on the individual level of analysis. Finally, Gavin notes that scholars concerned with complexity, equifinality and deep structure are always going to be dubious about counterfactual analysis. Gavin concludes that, while it is better for counterfactual analysis to be guided by thoughtful rules, it is unlikely to ever amount to a method on par with statistical techniques.

The current H-Diplo/ISSF Form includes essays by Niall Ferguson and Benjamin Mueller, both scholars who have spent considerable time thinking about counterfactual analysis. While both provide helpful insights on counterfactual analysis, it is important to note that neither essay is rooted in the mainstream political science tradition on which Levy’s article (and for that matter most of the Security Studies symposia series) draws. Ferguson writes as an historian, and all but a small portion of his essay is a dialogue with skeptics in that discipline. Moreover, while Mueller is an IR scholar, his take on methods seems to be well outside the mainstream, at least in the American academy.

Ferguson’s essay is initially framed as a response to Richard J. Evan’s Altered Pasts, rather than to the articles in Security Studies. Ferguson questions Evans’ claim that counterfactual analysis is necessarily conservative; suggests that Evans fails to understand that large outcomes can be affected by small causes; and insists that, contra Evans, individual choices can change the course of history. While noting that counterfactuals are useful tools, Ferguson acknowledges that they are difficult and need to be handled with care. He praises Levy’s rules as useful guidance for doing so, but diverges with Levy on two points.

Ferguson disagrees with Levy’s rule that scholars should seek “consistency with well-established theoretical generalizations.” Since Ferguson suggests that no generalizations in history are well-established, he argues that this rule should be struck down. It is worth mentioning that Levy actually agrees with Ferguson about the scarcity of good generalizations, noting in the article that:

“…we have few well-established laws of international relations. With precious few exceptions, we have at best contested theories and probabilistic relationships of modest strength that are conditional and often contextually dependent rather than universal. However, given the aim of explaining a single case and exploring a counterfactual under a specific set of historical conditions, contingent generalizations can often substitute for more general theoretical laws” (395).

Ferguson’s second disagreement with Levy turns on what constitutes a minimal-rewrite counterfactual. As part of a longer discussion on this question, Levy notes that one way to establish that the suggested change is not drastic is to show that the alternative was part of the political leaders’ choice set. The question then becomes how do scholars make that determination? Levy juxtaposes two positions: the options decision makers really did consider versus a larger set of options that adds choices they should have considered. Levy notes that in Virtual History Ferguson argues for the former position, and that he added the stricture that there must be written evidence of that contemplation. Levy suggests in a footnote that Ferguson’s evidence rule is too constraining. In his commentary in this Forum, Ferguson dismisses Levy’s critique of the evidence rule:

“the very fact that the issue is consigned to a footnote is wrong, because it is fundamental to counterfactual history, as opposed to other disciplines that use counterfactuals. For there is practically no other way of knowing what alternatives contemporaries contemplated than by consulting historical records. To be sure, in confining ourselves to alternatives that contemporaries recorded we necessarily lose counterfactuals that nobody wrote down or taped, or recorded but failed to preserve. But what exactly is the alternative? To guess what other options they contemplated? To hold a séance?”

A few points might be mentioned here. First, some portion of the apparent disagreement between Levy and Ferguson on what constitutes evidence of “options actually considered” may be reconcilable. Ferguson, in the section of Virtual History that he and Levy are discussing, observed:

“…we can only legitimately consider those hypothetical scenarios which contemporaries not only considered, but also committed to paper (or some other form of record) which has survived—and which has been identified as a valid source by historians. Clearly, that introduces an additional element of contingency, as there is nothing inevitable about which documents survive and which do not.”[8]

In a previous essay that is cited in the footnote, Levy observes that:

“While paper records of actors’ alternative options might provide particularly compelling evidence of choices not made, we should not exclude evidence based on oral interviews with first-hand observers.”[9]

Comparing the two italicized parts of these quotations, there may be more overlap here than either author realizes.

Second, it is worth remembering that Ferguson’s response to Evans suggests that counterfactual analysis is not intended to have any built-in biases towards particular types of history or kinds of decision makers. As Levy notes, however, the insistence on archival evidence might mean that we would:

“…be precluded from exploring alternative histories in repressive political actors in which actors are afraid to commit their thoughts to paper or among non-elite groups for which there are no written records.”

Moreover, in focusing on his particular disagreement with Levy in the footnote about evidence, Ferguson elides the much larger divergence in the main text. Levy is willing to consider as plausible, counterfactuals that were never considered by the actors at all. Finally, while Ferguson attributes Levy’s position on these questions to his training as a political scientist, Levy cites in favor of counterfactuals that were not contemplated by decision makers a view that was originally offered by the historian Paul Schroeder.

The other essay in this Forum is by Benjamin Mueller, an IR scholar from the London School of Economics. As noted above, Mueller’s essay springs from a very particular view of social inquiry, and his view of counterfactuals is grounded in complexity theory. In terms of the current symposium, his essay seems to overlap the most with the approach taken by Ned Lebow, although he is also somewhat sympathetic to Frank Gavin’s take as an historian. Mueller is considerably less kind to Jack Levy and Frank Harvey. At one point he observes:

“It is the false promise of epistemological closure provided by positivists that is so irksome about Levy and Harvey’s take on counterfactual, despite the indisputable quality of their work. The behaviouralist turn in social science, I argue, has exhausted its utility and is giving way to forms of analysis more sensitive to the non-linear causation that abounds in the complex system of densely interactive social relations that we inhabit.”

Without unpacking the term “behaviouralist” it is hard to know exactly what Mueller is referring to. It would, however, be hard to find a reasonable use of the term that could make his description fit mainstream political science in the United States.[10] To be sure, there is some variation across different departments, and of course individual scholars have contrasting expertise. But it is unquestionably the case that the center of gravity remains firmly at the ‘positivist’ end of the spectrum. If anything, the last decade has seen a shift to more explicit and careful statements of methods, and to increasing rigor.

It is terrific that scholars from history and political science continue to engage with each other. I have been following these kinds of conversations for a while,[11] but it is clear from the symposium and from this subsequent review that the two groups continue to be able to learn from each other. I agree with Niall Ferguson that a great strength of the symposium (and I would add of the two commentaries) is that the authors were all acutely aware that where they are coming from has an enormous impact on where they are trying to get to.

 

Participants:

Colin Elman is professor of Political Science, co-Director of the Qualitative Data Repository (www.qdr.org), and director of the Center for Qualitative and Multi-Method Inquiry in the Maxwell School, Syracuse University. He is co-founder of both the International History and Politics and the Qualitative and Multi-Method Research organized sections of the American Political Science Association, and co-director of the annual summer Institute for Qualitative and Multi-Method Research. He is series co-editor of the Cambridge University Press Strategies for Social Inquiry book series, and the new Methods for Social Inquiry book series. Elman co-chaired the American Political Science Association’s committee on Data Access and Research Transparency (DA-RT). Elman is the co‐editor of Progress in International Relations Theory: Appraising the Field (MIT Press); and Bridges and Boundaries: Historians, Political Scientists, and the Study of International Relations (MIT Press); of Realism and the Balancing of Power: A New Debate (Prentice Hall); and of the Realism Reader (Routledge). Elman has published articles in the American Political Science Review, the Annual Review of Political Science, Comparative Political Studies, the International History Review, International Organization, International Security, International Studies Quarterly, Millennium, Political Science & Politics, Security Studies, and Sociological Methods & Research.

Niall Ferguson is a senior fellow at the Hoover Institution, a visiting professor at Tsinghua University, Beijing, and the Diller-von Furstenberg Family Foundation Distinguished Scholar at the Kissinger Center of the Nitze School of Advanced International Studies in Washington, DC. He has written fourteen books, including Empire: The Rise and Demise of the British World Order and the Lessons for Global Power, The War of the World: Twentieth-Century Conflict and the Descent of the West, The Ascent of Money: A Financial History of the World, The Great Degeneration: How Institutions Decay and Economies Die, and Kissinger, 1923-1968: The Idealist. His many prizes and awards include the Benjamin Franklin Prize for Public Service (2010), the Hayek Prize for Lifetime Achievement (2012) and the Ludwig Erhard Prize for Economic Journalism (2013).

Benjamin Mueller was the Cato Stonex Scholar in International Relations at the London School of Economics and Political Science where he received his PhD in November 2015. His research examines the role of foreign-policy decision-making on the end of the Cold War, combining elements of complexity theory with counterfactual analysis. In 2015 he was a Visiting Fellow at the Saltzman Institute of War and Peace Studies at Columbia University. He is now an Associate Research Fellow at LSE IDEAS and an external consultant at the German Development Institute. He is currently working on a research project that traces the decision-making processes behind the Gulf and the Iraq Wars, identifying parallels and differences and weighing the causal impact of choices, context and chance using complexity counterfactuals.

 

Review by Niall Ferguson, Hoover Institution

How Not to Counter Counterfactuals

I

Surprisingly, some quite eminent historians still have difficulties with counterfactuals. Sir Richard J. Evans was the Regius professor of history at Cambridge between 2010 and 2014. The author of numerous well regarded books on modern German history, Evans has in recent years sought to address broader issues of historical methodology, first in his In Defence of History (2002), a tract against post-modernism, and then in Altered Pasts: Counterfactuals in History, a critique of historians who venture to post ‘what if’ questions. Counterfactual inquiries, Evan argues, are “marginal” and “of little real use in the serious study of the past.”[12]

Evans offers two objections to books—among them, one I edited myself nearly twenty years ago[13]—that explicitly consider alternative scenarios. The first is that the authors are generally conservatives.[14] “Not surprisingly,” he writes, “most of the historians writing counterfactual history … have been both politically and methodologically conservative.”[15] Indeed, at one point, he goes so far as to claim that “counterfactuals have been more or less a monopoly of the Right.” Evans’s explanation for this alleged political skew is twofold. First, because “counterfactualism” gives importance to “agency—the actions usually of a small number of individuals,” it is “clearly more favored by right-wing historians … There are indeed few, if any, counterfactuals written from a left-wing point of view,” because left-wing historians (e.g., Evans himself) attach more importance to larger and less easily deflected social and economic forces. Evans’s second explanation is that “historians of a conservative inclination began to emphasize the role of chance and think about how things might—perhaps, should—have turned out differently in the mid-1990s, because a long period of Conservative Party dominance, beginning in 1979, was clearly coming to an end.”[16] I and others were not considering counterfactual questions seriously because when we wrote ‘what if’ we really meant ‘if only.’ Counterfactuals were just a ruse enabling us “to rewrite history according to [our] present-day political purposes and prejudices.”[17] In short, counterfactual history is mostly “mere wishful thinking.”

Evans sees further proof that counterfactualism is the domain of conservatives in the fact that it “focuses almost exclusively on traditional, old-fashioned political, military, and diplomatic history of the sort that used to be dominant in the 1950s.” This, he remarks,

is in itself is one major reason to be sceptical of the more far-reaching claims of counterfactual history: it not only assumes but also implicitly preaches a history where politics and warfare are the most important subjects to be studied; in other words, it advocates a narrow, traditional approach to the past that most historians have long since moved beyond … Nowadays, the world’s most innovative historians focus overwhelmingly on social, economic, and cultural history, on global and transnational representations of the past, not on political or diplomatic history.[18]

Evans’s pose here as the vanguard of innovative historiography is rather hard to reconcile with his sideswipes at postmodernism, which, he grumbles, “has freed up writers of all kinds to imagine what might have been and to tie their imaginings in one way or another to real historical events and real historical personages.”[19]

The relevant literature on counterfactuals in fact originates in the philosophy of history but has in recent years been added to by political scientists, game theorists, and others—none of them, to my knowledge, nostalgia-prone conservatives.

Evans’s second, methodological, argument against counterfactuals is equally unconvincing. He begins by asserting that large events must have commensurately large causes. “Many counterfactualists,” he writes, “take as their starting point familiar chance events that we already know occurred”:

Often, their speculations center on the chance survival of assassination attempts, the untimely deaths of kings, the unfortunate mortality of their offspring, the sudden reversal of fortunes in battles. Few historians would deny that such chance events had a major effect; yet for serious consequences to ensue, most historians would agree that other, larger factors had to come into play, of whatever kind. … Any of these events could easily have ended otherwise than it did, but larger changes in the historical context would probably have been necessary for this to have had the effects sometimes claimed by counterfactualists. Counterfactuals such as these can only be posited as having big effects by leaving out the historical context …[20]

Turning from the realm of chance to the realm of decision, Evans constructs the straw man that counterfactual historians present “individual decision makers … implausibly, as free-floating agents.”[21] Evans insists we should essentially ignore the “roads not taken” because:

Conditional statements of … hard-edged certainty are foreign to the historian’s way of going about explanation, which is almost invariably tentative and involves considerable use of the word “probably.” “Monocausal” explanations make historians uneasy; we prefer to pile up causes until events are overdetermined, that is, they have so many causes that if one did not operate the others would, and the event in question would still have occurred. The key of course lies in designating one cause as more operative than others. Historians usually construct hierarchies of causes—primary causes, secondary causes, main causes, subsidiary causes, and so on, which affect different parts of the explanation.[22]

We see here in all its antediluvian glory an approach to the issues of probability and causation that makes the field of diplomatic history look cutting-edge.

What Evans, like E.H. Carr before him, ignores is the fact that many decision makers really do make decisions—in the sense that they face real alternatives between which they consciously choose.[23] In studying (as they often do) such low-frequency, high-impact phenomena as wars, revolutions or epidemics, historians are in fact concentrating their attention on the “fat tails” of the distribution of historical events. The most common error they make is precisely to assume that such high-impact events must be over-determined. This approach nearly always produces what Nassim Taleb has called “retrospective distortion.” [24] Evans’s hierarchy of causes is the classic product of the “narrative fallacy”: the construction of psychologically satisfying stories on the principle of post hoc, ergo propter hoc.[25] He takes no account of the well documented fact that contemporaries retrospectively exaggerate the ex ante probabilities of events.[26] Historians, who like to read contemporaries’ ex post accounts, tend to compound this error with their deterministic stories. They fail to grasp that small decisions (or non-decisions), more random than determined, can have enormous consequences, just as—in the famous example devised by Philip Merilees to illustrate Edward Lorenz’s work on chaos theory—the flapping of a butterfly’s wings in Brazil can set off a tornado in Texas.

Meteorology plays a big part in history. For most of recorded history, most people have lived from agriculture, and nothing affects the fate of farmers more than the weather. Climatic conditions also play a significant part in warfare, dooming the Armada, blessing D-Day. If the weather (as Lorenz argued) is a chaotic system, characterized by non-linear relationships and stochastic behavior, then we should expect the same to be true of much else in the human past. Evans acknowledges the role of chance, but insists that there is simply no need to consider alternative scenarios, no matter how narrowly missed: “A simple factual narrative of German unification,” he writes, “indeed a simple factual narrative of the Battle of Sadowa, should be enough without any explicit counterfactual speculations to demonstrate the role of chance in the process.”[27] Readers of his brand of history should be extremely wary of any such “simple factual narratives” intended to (as Evans puts it) “to drive the causal explanation forward.” For, when an historian sweeps the other options under the carpet in this fashion it is precisely because he wishes to endow his chosen narrative with teleological power.[28] This is an empty rhetorical device.

II

Another regius professor at another university, Hugh Trevor-Roper, much better understood the importance of counterfactuals to the historian when he declared, in a brilliant valedictory lecture:

[A]t any given moment of history there are real alternatives, and to dismiss them as unreal because they were not realised … is to take the reality out of the situation.  How can we “explain what happened and why” if we only look at what happened and never consider the alternatives, the total pattern of forces whose pressure created the event? … History is not merely what happened: it is what happened in the context of what might have happened. Therefore it must incorporate, as a necessary element, the alternatives, the might-have-beens. …

To ignore … lost moments, to erase them impatiently from the page of history as mere non-events, is surely not only an error but a vulgar error: an error, because, even though abortive, they explain the motives of historical persons and contain a historical lesson; a vulgar error because they have a deeper reality which it is philistine and insensitive to ignore: though politically barren, they have contributed, more than any mere facts, to that art and literature which is the permanently valuable deposit of past history.

It is only if we place ourselves before the alternatives of the past, as of the present, only if we live for a moment, as the men of the time lived, in its still fluid context and among its still unresolved problems, if we see those problems coming upon us as well as look back on them after they have gone away, that we can draw useful lessons from history. To restore to the past its lost uncertainties, to reopen, if only for an instant, doors which the fait accompli has closed, this requires an effort of imagination. But surely it is a necessary effort if we are to see history as reality, not merely as a convenient scheme.[29]

A very similar argument was advanced more recently by another Oxford-educated historian, in his introduction to a history of Third Reich:

Contemporaries could not see things as clearly as we can, with the gift of hindsight: they could not know in 1930 what was to come in 1933, they could not know in 1933 what was to come in 1939 or 1942 or 1945. If they had known, doubtless the choices they made would have been different. One of the greatest problems in writing history is to imagine oneself back in the world of the past, with all the doubts and uncertainties people faced in dealing with a future that for the historian has also become the past. Developments that seem inevitable in retrospect were by no means so at the time, and … things could easily have turned out very differently to the way they did at a number of points in the history of Germany in the second half of the nineteenth century and the first half of the twentieth.[30]

Strangely, the author of those words was none other than Richard J. Evans.

“At almost every turn,” Evans writes in The Coming of the Third Reich, the first of his three volumes on Hitler’s Germany, “things might have been different. The triumph of Nazism was far from a foregone conclusion right up to the early months of 1933.”[31] The difficulty is that, while Evans the practicing historian is only too well aware that these roads not taken are a crucial part of the history of Nazi Germany, his theoretical (or, rather, political) objection to counterfactuals makes it impossible for him to explore them in an illuminating way. He admits that “we can imagine what might have happened had the generals succeeded in their tentative plot to overthrow Hitler in 1938 because they did not think Germany was ready for war, or had Neville Chamberlain not secured the Munich agreement that made them abandon their plans, or had Hitler not decided to accelerate his external and internal aggression in 1937-1938, and indeed in examining these events, we implicitly do engage in such speculations because these were contingencies that might easily have ended in different outcomes.”[32] Yet his own imaginings on the subject are among the dullest ever published. Evans offers no reflections whatever on what would have happened if Hitler had been killed by George Elser’s bomb on 8 November 1939. And what he has to say about the attempt on Hitler’s life on 20 July 1944, is almost painful to read in its banality: “The death of Hitler might well have hastened the disintegration of the regime, loosened the bonds of loyalty that tied so many Germans to it still in mid-1944, and shortened the war by some months, saving millions of lives on all sides by doing so.”[33]

In Altered Pasts, Evans ventures only slightly further, but enough to incriminate himself as a closet counterfactualist:

It helps understand the motives and purposes of the men who tried to kill Hitler on 20 July 1944 if we think about what might have happened had they succeeded. In practice, they enjoyed only very limited support among the armed forces, and the death of Hitler would probably have unleashed civil war as the SS and other fanatical Nazi troops fought to put them down and exact revenge. … the death or incapacity of Hitler would indeed … surely have unleashed a power struggle within the Nazi elite: and such was the centrality of Hitler to the whole Nazi system that it would have been unable to continue for much longer anyway.[34]

So much for the scourge of counterfactual history.

III

Of course there are reasons for historians to handle counterfactuals with care. They are difficult, unlike so much of what historians do, which is merely laborious. The key question is what should the rules of engagement be, given that abstinence is not a viable option. Hence this timely Security Studies symposium.

Building on the work of Philip Tetlock and Aaron Belkin,[35] Jack Levy offers eight rules, which can be summarized as follows.[36]

First, clarity: The analyst must clearly specify the counterfactual’s antecedent and its consequences, the causal path or paths and mechanisms linking the two, and the nature of the relationship (deterministic or probabilistic), making it clear if there is more than one path or consequence. Second, minimal rewrite: The counterfactual should involve as few changes as possible from what actually happened. In particular, scholars should focus on cases where the ex ante probability of what actually happened was lower than the ex ante probability of the counterfactual. Third, cotenability: The counterfactual must be complete, in the sense that (for example) a different president would have had different advisors and administration members. In other words, a well-constructed counterfactual cannot assume ceteris paribus, because other things would automatically have been different.

Levy’s fourth rule is “consistency with well-established theoretical generalizations”: Where these exist, general or covering laws of history should not be violated by the counterfactual. Number five is historical accuracy: Only qualified historians with at least a PhD should engage in counterfactual history. Sixth is temporal proximity: The chronological range of the counterfactual should be limited, to avoid unconstrained speculation over long timeframes (a rule that would disqualify most of Richard Ned Lebow’s 2014 book Franz Ferdinand Lives).

The penultimate Levy rule if “awareness of redirecting counterfactuals”: The possibility must be overtly contemplated that another event could redirect history back onto its “original” path. Finally there is the need for comparative counterfactual analysis: Any counterfactual needs to be “tested” by the construction of at least one opposing counterfactual—so that, in effect, at least three scenarios are considered: what actually happened, and two alternatives. If another historical case closely resembles the counterfactual scenario, it should be referenced.

These rules strike me as useful, with two important exceptions. First, I know of no well-established theoretical generalizations in history, though I know of several theoretical generalizations, so rule 4 had better be struck out. Secondly, Levy writes that “we should consider as plausible or probable only those alternatives … that contemporaries actually considered,” but adds in a footnote that my own rider—” we can only legitimately consider those hypothetical scenarios which contemporaries not only considered, but also committed to paper (or some other form of record) which has survived”[37]—is “too restrictive, because some things are not committed to paper, and some records are lost.” This is wrong. Indeed, the very fact that the issue is consigned to a footnote is wrong, because it is fundamental to counterfactual history, as opposed to other disciplines that use counterfactuals. For there is practically no other way of knowing what alternatives contemporaries contemplated than by consulting historical records. To be sure, in confining ourselves to alternatives that contemporaries recorded we necessarily lose counterfactuals that nobody wrote down or taped, or recorded but failed to preserve. But what exactly is the alternative? To guess what other options they contemplated? To hold a séance?

Here we see the fundamental contrast between the social or political scientist, who is interested in counterfactuals as a means of exploring, hypothetically, social or political mechanics, and the historian, who is interested in counterfactuals as a means of gaining insight, imaginatively, into the past. The great merit of the Security Studies symposium is that it makes this distinction clear. Levy and Lebow belong in the political science camp. By contrast, Frank Harvey was playing by the historian’s rule when he wrote Explaining the Iraq War (2011), because he used archival records to show that it was not only a neoconservative fringe that favored an attack on Iraq after 9/11 and that a Gore administration would have received similar intelligence and advice. [38] Likewise, Francis J. Gavin bases his point about the early Cold War [“one cannot understand US support for the French in Southeast Asia in the early 1950s without wrestling with the explosive politics surrounding re-arming the Federal Republic of Germany” (429)] not on some “well established theoretical generalization” but on his deep knowledge of the relevant diplomatic documents. [39]

The great Oxonian philosopher R. G. Collingwood summed up his philosophy of history in three propositions: All history is the history of thought; historical knowledge is the re-enactment in the historian’s mind of the thought whose history he is studying; historical knowledge is the re-enactment of a past thought incapsulated in a context of present thoughts which, by contradicting it, confine it to a plane different from theirs.[40]

Collingwood was very firm in arguing that past thought could be “re-enacted” only on the basis of authentic records—though, as an enthusiastic archaeologist of Roman Britain, he by no means confined himself to written or printed documents:

The historian cannot answer questions about the past unless he has evidence about it. His evidence, if he “has” it, must be something existing here and now in his present world. If there were a past event which had left no trace of any kind in the present world, it would be a past event for which now there was no evidence, and nobody—no historian; I say nothing of other, perhaps more highly gifted persons—could know anything about it.[41]

As Collingwood put it, “You are thinking historically … when you say about anything, ‘I see what the person who made this (wrote this, used this, designed this, &c.) was thinking.’ … On what conditions was it possible to know the history of a thought? First, the thought must be expressed: either in what we call language, or in one of the many other forms of expressive activity.”[42] Far from being a superfluous restriction, this insistence on the interrogation of authentic remnants of past thought seems to me the cardinal rule of all history, including counterfactual history.

In concluding my introduction to Virtual History, I made a twofold argument for counterfactual history:

Firstly, it is a logical necessity when asking questions about causation to pose “but for” questions, and to try to imagine what would have happened if our supposed cause had been absent. For this reason, we are obliged to construct plausible alternative pasts on the basis of judgments about probability; and these can only be made on the basis of historical evidence. Secondly, to do this is a historical necessity when attempting to understand how the past “actually was”—precisely in the Rankean sense, as we must attach equal importance to all the possibilities which contemporaries contemplated before the fact, and greater importance to these than to an outcome which they did not anticipate.[43]

I stand by these two imperatives. It is a little depressing to me that, nearly two decades after the publication of Virtual History, an eminent historian can dismiss counterfactual history as, at best, marginal and, at worst, a form of conservative make-believe, even while continuing to engage in it himself. It is depressing, too, that the case for counterfactual history today has to be made by political scientists, who tend to have scant regard for the central principle of historical scholarship, that our role is to re-imagine the past as it was—with all its uncertainties and possible futures—on the basis of such sources as survive.

 

Review by Benjamin Mueller, London School of Economics and Political Science

Scarcely a concept proves as persistently vexing and controversial in modern political science as counterfactual analysis. Max Weber, the towering twentieth-century social theorist, defined these ‘What Ifs’ as “the mental construction of a course of events which is altered through modification in one or more ‘conditions.’” Counterfactuals, Weber proposed, were indispensable tools to compare and evaluate the relative worth of different cause-effect chains: in order to “assess the degree to which a particular cause ‘favoured’ a given effect, we must hypothetically ‘compare’ the result that actually followed with alternate possibilities.”[44] This is a powerful way of thinking about what constitutes a cause: some X, in absence of which some outcome Y would not have come about. If I argue that President Ronald Reagan’s foreign policy strategy caused the peaceful end of the Cold War, it follows that absent said strategy – say, if Reagan had not survived John Hinckley’s failed assassination attempt in March 1981 – the Cold War would not have ended in 1990. Unless, of course, there were foreign-policy strategies other than Ronald Reagan’s that could have ended the Cold War in the same manner and timeframe (or if a President other than Reagan would have deployed the latter’s strategy). There is also the possibility that the end of the Cold War had nothing whatsoever to do with Ronald Reagan or his foreign-policy strategy. This, too, yields a counterfactual – namely, that the assassination of Ronald Reagan in 1981 would not have affected the eventual peaceful conclusion of the Cold War a decade later. From an enormously simplified causal claim – Ronald Reagan ended the Cold War – flow a myriad of counterfactual implications (which is the case for almost all causal claims).

I am confident that even the relatively modest, informal counterfactual statements above will have triggered internal objections, rebuttals, and modifications among a great many of the readers of this text. After all, a few months ago I received my doctorate for a thesis titled At Cold War’s End: Complexity, Causes and Counterfactuals, which, over four years of research and writing, was accompanied by ceaseless discussions – be they with laymen or academics – of the various counterfactual ideas proposed therein.[45] ‘What Ifs’ are irresistible conversation starters which rarely fail to tease out contrasting opinions. Therein lies the paradox: counterfactuals are intuitively accessible mental tools that can aid our thinking and clarify the implications of our arguments; but they do so in a manner – viz., contrary to fact – that is inherently speculative and connected to one’s prior assumptions and arguments about what causes things in the social world (Individuals? People? Structures? Chance? All of the above?). Given the inherent risks to clarity and logic that are associated with counterfactual thinking, two things ought to be clear to all who engage therein: first, that counterfactuals, by their very nature, will always elicit debate in partially subjective enterprises like social science and history; second, that any use of counterfactual analysis ought to be accompanied by a large dose of clearly labelled epistemological humble pie. Controversy will never cease to accompany ‘What Ifs,’ and while some critiques are and will be trivial and/or fallacious, many others stem from the kinds of fact and value-based disagreements that abound in social-scientific academe and are in many ways its lifeblood.

And so the symposium on counterfactual analysis published in Security Studies is highly welcome: it offers a timely, concise, wide-ranging, and valuable discussion of the practical concerns surrounding counterfactual analysis, as well as an introduction into the deep philosophical debates that underpin this mode of thinking. As all the contributors (and many before them) point out, counterfactuals are ubiquitous to human thought; every evaluative deliberation requires some manner of sketching out alternative chains of events and selecting between them. Ned Lebow puts this point succinctly in his important book on the topic of counterfactuals in International Relations (with the appropriate title Forbidden Fruit): the difference between factual and counterfactual thinking is one of degree, not one of kind.[46] It is important to confront head-on the challenges and controversies that surround this methodological elephant in the room.

Jack Levy begins with a thorough and earnest treatment of the links between counterfactuals, causal inference and historical analysis, providing an account rich in accumulated wisdom from his decades-long engagement with the subject.[47] The use case in Levy’s article relates to counterfactuals as what I call ‘causation detection devices’: “knowledge about what might have been can help explain what actually was” (380). He seeks to address how counterfactuals can “be used to support arguments about the causal impact of particular variables in a particular historical episode” (381). Levy surveys the logic behind different kinds of causal statements and counterfactual claims and makes the important but oft-overlooked observation that “those who emphasize contingency and those who emphasize the primacy of structures and social forces often reach diametrically opposed conclusions from their respective counterfactual analyses of the same historical cases” (387). In and of themselves, counterfactuals do not prioritise any kind of causal conclusion: they can be deployed in support of determinist arguments (when re-writing history yields no change in outcomes) as much as to illustrate the role of contingency (when tiny changes have huge repercussions).

In an effort to ‘discipline’ counterfactual analysis and “prevent it from deteriorating into a ‘literature of the imagination’,” Levy proposes best practices for case-based causal counterfactuals (387). Clarity matters – i.e. laying out the causal path pursued by a given counterfactual, and how this relates to other possible causal paths – but overly intricate, long causal chains suffer from increasingly diminishing probabilities. Counterfactuals that introduce small, plausible changes to the historical record – e.g. altering the failure/success of an assassination attempt – can approximate the ideal-type of controlled comparison experiments and are easier to pursue than large rewrites (e.g. re-imagining the political system of a country). The interdependencies that accompany re-writes of history – ripple effects – must be transparently demonstrated and baseline assumptions made explicit. These assumptions, Levy convincingly argues, need not be consistent with “well-established statistical generalizations” (as many positivists demand) but instead with the less stringent criterion of “consistency with empirical evidence” (a wise remark given the paucity [impossibility?] of fashioning reliable, meaningful statistical generalizations out of history) (396). Moreover, the analyst must be open to the idea that in a counterfactual world, history can still return to its original course. These are all solid, well-documented points that build a case for the rigorous use of counterfactuals in aid of causal inference, conceived through a positivist philosophy of science.

Frank Harvey’s contribution on how to develop “strong, policy-relevant causal explanations for major events in world politics” builds on Jack Levy’s vision (413).[48] Harvey advocates the comparison of alternate, competing counterfactual scenarios to weed out weak from strong causal claims. He bases this approach on his well-received study into the causes of the Iraq War, with its controversial conclusion that the George W. Bush Administration was coincidental to the conflict. Given the dramatically altered post-9/11 security picture, Harvey argues that a counterfactual President Al Gore would also have gone after Iraq’s leader Saddam Hussein: the intelligence picture mattered more than the neoconservative ideology of the Bush team in causing the Iraq War.[49]

Harvey explain the two basic obstacles that confront all counterfactuals: the value-add of a ‘What If’ is not always clear (what, if anything, differentiates causal counterfactuals from idle speculation?), and, most critically, the plausibility of any alternative world is contestable since it is based on conjecture rather than fact (414). Before adherents to ‘What If’ thinking get too deflated, Harvey also correctly points to the key counter-argument that critics either miss or ignore: the only way to avoid counterfactuals – be they explicit or implicit – is by restricting historical analysis to narration only, because all causal thinking is based on counterfactual thinking (415). The main point Harvey makes in his article is that too much analysis in foreign policy is opinion masquerading as fact, brought about by the use of ‘weak’ – i.e. unproven, implicit – counterfactual assertions. By making explicit the counterfactuals behind different interpretations of given historical events, we bring to the fore hidden assumptions and tease out the empirical implications of various views, allowing the analyst to see which positions are backed by the most evidence. In Explaining the Iraq War, for example, Harvey reaches the conclusion that there is more evidence for a ‘Gore-war’ counterfactual than for a ‘Gore-peace’ scenario.

By way of best practices, Harvey has little to add to Levy’s points other than advocating the juxtaposition of rival counterfactual scenarios (“comparative plausibility”) and attaching this to Andrew Bennett and Alexander George’s principle of process tracing to create a model of counterfactual causal analysis (421).[50] So far so good. When Harvey applies this methodology to the Iraq War he finds its principal cause was not agency, but rather U.S. unipolarity, intelligence failures, and the effects of 9/11. Harvey structures his analysis around the falsification of what he calls the ‘neoconism’ hypothesis (which assigns causal responsibility for the Iraq War to the Bush Administration). He styles neoconservatism as the independent variable to the dependent variable of the Iraq War. Harvey then pursues a ‘Gore-war’ and a ‘Gore-peace’ counterfactual, which yields the conclusion that a Gore victory in 2000 would still have brought about the Iraq War: Gore, too, had hawkish proclivities and would have reached similar conclusions concerning Saddam Hussein’s weapons of mass destruction programmes as the Bush Administration. Thus he concludes that George W. Bush and his foreign policy team were not the primary cause for the Iraq War. In counterfactual terms, Harvey argues that even in the absence of a Bush presidency the Iraq War would have taken place: the post-9/11 domestic and international political context, in other words, conditioned the U.S. political leadership as a whole to make the War in Iraq highly likely regardless of who held the Presidency.

What is wrong with the above analysis? In essence, it is Harvey’s insistence that his is a ‘strong’ counterfactual conclusion (as opposed to ‘counterfactually weak’ arguments that fault the Bush Administration for catastrophic decision-making failures in the run-up to and during the Iraq War). Harvey thus commits the cardinal sin of counterfactual thinking: excess confidence in the certainty of his conclusions. To be perfectly clear, his research is methodologically and empirically sound; counterfactuals are presented transparently, various alternative arguments are discussed at length, the original counterfactual is a minimal re-write (Gore beating Bush in 2000), and the analysis is anchored within wider statistical and empirical generalisations about the causes of war. And that is precisely where the (distinctly positivist) problem arises: whilst analysts of international affairs can fool themselves into thinking that rock-solid adherence to rules of causal inference gives their conclusions the quality of ‘hard’ science (a cognitive bias best made sense of as ‘physics envy’), these conclusions in fact continue to be riddled with contestable assumptions and empirical leaps of faith. An obsession with predictability and generalizability ignores that only part of international relations pertains to dynamics driven by matter (which obeys scientific laws): a significant and, as I argue in At Cold War’s End, sidelined portion of international politics is driven by people, specifically by the interrelationships of leaders, ideas, and decisions. This introduces an element of ontological uncertainty and unpredictability into IR that no amount of ‘hard science’-methodology can remove. As such I think it utter folly to declare one’s counterfactual as ‘strong’ without recognizing its inescapable subjectivity. Harvey insists that the (counterfactual) evidence he finds supports a generalizable theory of viewing wars as caused by self-reinforcing sequences of rational policy choices (420) that create institutionally path-dependent, deterministic dynamics (418) – just as happened on the road to the Iraq.

This is an entirely plausible causal narrative whose internal validity I do not even necessarily discount. But the conclusion that in a Gore counterfactual the same would have happened is thin, at best: Harvey’s counterfactuals are as weak as the ones he faults (which, of course, are the ones he disagrees with). Why do I say this with such confidence? Consider this evidence of an agent deliberately resisting the aforementioned deterministic, path-dependent march to war which Harvey postulates as his ‘strong’ conclusion. Let me cite the killer blow directly from the horse’s mouth (the 44th President of the U.S.): “There’s a playbook in Washington that presidents are supposed to follow. It’s a playbook that comes out of the foreign-policy establishment. And the playbook prescribes responses to different events, and these responses tend to be militarized responses. Where America is directly threatened, the playbook works. But the playbook can also be a trap that can lead to bad decisions. In the midst of an international challenge like Syria, you get judged harshly if you don’t follow the playbook, even if there are good reasons why it does not apply.”[51] This leads Jeffrey Goldberg to conclude, “I have come to believe that, in [President Barack] Obama’s mind, August 30, 2013, was his liberation day, the day he defied not only the foreign-policy establishment and its cruise-missile playbook, but also the demands of America’s frustrating, high-maintenance allies in the Middle East – countries, he complains privately to friends and advisers, that seek to exploit American ‘muscle’ for their own narrow and sectarian ends.”[52]  All this stands in direct opposition to Harvey’s assumption that leaders bend to the pressures of context. A sentient, idiosyncratic being – i.e. a human – is perfectly capable of resisting the deterministic pressures Harvey finds so conclusive in his analysis: Obama calls this “breaking the playbook.”[53] Analysts who work on this basis study a world where free-thinking agency is at least partially generative of reality, namely through choices and decisions. Rather than assuming that modelised rational agents are trapped in an endlessly repeating linear causal universe, we are instead dealing with a dynamic, evolving realm of social causation where regularity combines with idiosyncrasy and interaction effects to preclude predictions based on the linear extrapolation of generalized trends of the past.

It is the false promise of epistemological closure provided by positivists that is so irksome about Levy and Harvey’s take on counterfactual, despite the indisputable quality of their work. The behaviouralist turn in social science, I argue, has exhausted its utility and is giving way to forms of analysis more sensitive to the non-linear causation that abounds in the complex system of densely interactive social relations that we inhabit. Levy calls for counterfactuals to have “observable implications that can be tested somewhere in the real world,” (397) something that makes little sense for phenomena that are inherently unpredictable (like, say, Obama’s decision not to enforce militarily the Syrian red line). Levy vaguely mentions this point – “relationships [that] are nonlinear can reverberate through the system, generate large irregularities, and make it impossible to make anything other than short-term predictions” (398) – but unfortunately does not go on to explore its profound implications for the conduct of causal research in IR. Levy pays vague tribute to the idea that complex causation may necessitate new causal approaches – “today’s predictions of the future are tomorrow’s historical counterfactuals. The well-known difficulties of forecasting the future should serve as a cautionary tale for those expressing too much confidence in their counterfactual arguments” (402) – but does not pursue the point; in any case, Harvey exhibits the very counterfactual confidence Levy warns against. We can test our counterfactual-driven hypotheses all we want, but treating testability as an indicator of causal validity works best in linear worlds. In non-linear worlds, a testable prediction needs to incorporate the possibility that self-reflective agency can alter its behaviour in light of such predictions. It is analogous to the unpredictability inherent in quantum physics.[54]  Those who wish to explore the suitability of counterfactuals for the study of complex non-linear causation would do well to consult the one chapter in Philip Tetlock and Aaron Belkin’s classic Counterfactual Thought Experiments in World Politics that treats this topic: despite ample citations of most of the book’s various chapters, Robert Jervis’ contribution (“Counterfactuals, Causation, and Complexity”) is not mentioned by any of the Symposium’s participants.[55]

It falls to Ned Lebow to make the case against positivist interpretations of counterfactuals and cause: “we unreasonably assume that important cases like World War I are representative of a broader class of events. Commenting on this phenomenon, Deirdre McCloskey notes that “disdain for assigning large events small causes is not rational in a world that is partly non-linear.”[56] Moreover, “in international relations there appear to be few, if any, discernable patterns beyond a few obvious ones [which] were all common knowledge before the era of theory building” (407). All “so-called patterns” in IR are “indeterminate as their consequences depend on idiosyncratic factors, [and] most cases cannot be considered independent because the policymakers in question acted on the basis of the lessons they drew from what they considered relevant previous cases” (408). Not only does the context of cases and phenomena vary dramatically over time, but a double hermeneutic is also at play. Lebow espouses the concept of “singular causation,” which treats events as amenable to causal analysis, but as non-repetitive instances: singular causal events “cannot be explained by existing regularities, and they do not support new ones that will allow prediction” (409). Background conditions, sequences and timing, and idiosyncratic choices by self-reflective decision-makers: these combine to make events ‘singularly caused’ – a radical break from the assumption that phenomena ought to be be categorized and subsumed under empirical generalisations that bind together such classes of events (‘war’, ‘peace’, ‘democratisation’ and so forth). Deterministic factors still matter – structural variables and incentives set the backdrop to events in international affairs – but they need to be weighed against indeterminate factors. This model of ‘inefficient causation’ benefits from counterfactuals in a way that is quite different from the rigours of Levy’s and Harvey’s approach. Empirics are still essential to such counterfactuals’ quality, and adherence to certain ground-rules (e.g. argumentative transparency) is key. But counterfactuals do not need their imaginative potential constrained by the strictures of a philosophy of science that is unsuited to the subject matter it studies. We ask ‘What Ifs’ to tease out complex, multi-dimensional causal effects, to weigh up the deterministic pressures of structure against the idiosyncratic influence of agents and the precarious role played by contingency and chance. Thus, a counterfactual can go in virtually any direction, so long as the analyst knows why she is going in that direction, and explains her reasoning, taking alternative viewpoints into account. No counterfactual will ever satisfy all critics, because the nature of the analysis does not offer empirical or theoretical closure. Instead, it can shed light on previously under-appreciated causal relationships, highlight causal effects in complex webs of events, and heighten our understanding of how particular historical episodes unfolded.

At Cold War’s End, in a nutshell, concludes that the grand interplay between big structural forces (the weakening of the Soviet economic and social base), relationship dynamics (between the U.S. and Soviet leadership in the 1980s and amongst themselves), policy choices (especially in the domain of nuclear disarmament) and spontaneously emerging complex developments (like the revolutions of 1989) combined to end the Cold War rapidly, unpredictably, and peacefully. Counterfactually extricating five key actors from this web of events – U.S. President Ronald Reagan, his Secretary of State George Shultz, Soviet General Secretary Mikhail Gorbachev, his Foreign Minister Edouard Shevardnadze, and Reagan’s successor George H. W. Bush – would have altered the course of events substantially, something I argue by identifying the idiosyncratic contributions these actors made to various key developments in that decade. There is little of ‘testable implications’ among these findings other than the broad idea that while agents matter, they do not possess a free reign (e.g., any incoming General Secretary in 1985 would have had to do something about the USSR’s malaise, but options other than Gorbachev’s radical reforms could have been pursued by an alternative leadership). I know full well that plenty will disagree with my counterfactual conclusions, and so I shy away from calling them ‘strong’ or even particularly disciplined. What I also know, however, is that a counterfactual convinces by virtue of the strength of its account, its ability to convince the interlocutor that the causal chain offered is plausible and compelling. Indeed, as Geoffrey Hawthorn points out, “the force of an explanation turns on the counterfactual which it implies.”[57]  Some readers of At Cold War’s End will end up buying into the idea that at certain key moments, leaders possess the ability through deft and deliberate diplomacy to re-make the international system – and some will remain unconvinced by this and continue to argue that larger (and possibly predictable?) forces surrounding us shape outcomes in world affairs. Counterfactuals will never settle the matter, but they can encourage researchers to come up with persuasive singular causal accounts that combine theory, evidence, and argument into a forceful whole. Scholars can do this and end up constructing competing, contradictory causal accounts, which need not be troubling. The Gore-war counterfactual, in my eyes, is as weak or strong as the Gore-peace counterfactual. Constructing both and comparing them, however, sharpens the debate between different interpretations of the causal mechanisms that drive issues of war and peace – and the point about comparative counterfactual analysis is most certainly a laudable contribution that Frank Harvey’s article makes.

Francis Gavin’s helpful concluding article in the symposium offers a historian’s perspective on the advantages and problems surrounding ‘What Ifs’, strongly suggests that the outright rejection of counterfactuals is misguided and counterproductive, but also points out that the yield of counterfactual analysis is limited, at least when it comes to settling causal debates.[58] The regular practitioner of counterfactual historical analyses becomes proficient not at predicting the future, but at suggesting predictions of alternative pasts. By re-opening historically closed events and re-imagining them in combination with empirical research, we point to the fundamental forces that we – as individual thinkers – feel explain historical outcomes. In general historical terms – at the ‘meso’-level, nestled between largely predictable micro-events and utterly uncertain macro-trends – this heightens our awareness of how deterministic and open-ended causal factors interact. The most practical benefit of this is that, in Gavin’s words, such counterfactuals end up “becoming a very powerful tool to assess competing policies and identify key trends in international politics,” a methodology pioneered in commerce by Shell Oil and in politics by the National Intelligence Council. Ultimately, this is “history in reverse, where different variables are swapped out to identify what forces are truly driving things” (430). Such is the promise of disciplined counterfactuals: not greater certitude in our predictive abilities, not more rigorous conclusions of why something happened, but more nuanced, fine-grained, transparent reconstructions and explanations of complex political phenomena, a deeper awareness of the multiverse of paths mankind’s future can take, and of the contingencies and trends that will bring about future X rather than future Y. At its most practical, this case for ‘complexity counterfactuals’ aids scenario planning and contingency-based forecasting, which can make scholars and policymakers more adept at suggesting possible futures and sensing what turning points are pushing the world toward one trajectory of outcomes rather than another. Counterfactuals do not settle debates: they break them down, make them manageable, and allow us to develop competing causal narratives, the judges of which ultimately remain readers at large of such analyses. Whilst the Security Studies symposium offers both breadth and depth in its discussion of counterfactual analysis, it falls short when it comes to considering more cutting-edge applications of the tool within non-linear/complexity-driven paradigms of the social sciences.

 

Notes

[1] Andrew Bennett, Colin Elman & John M. Owen, “Security Studies, Security Studies, and Recent Developments in Qualitative and Multi-Method Research,” Security Studies 23:4 (2014): 657-662, DOI: https://doi.org/10.1080/09636412.2014.970832; Andrew Moravcsik, “Trust, but Verify: The Transparency Revolution and Qualitative International Relations,” Security Studies 23:4 (2014): 663-688, DOI: https://doi.org/10.1080/09636412.2014.970846; Elizabeth N. Saunders, “Transparency without Tears: A Pragmatic Approach to Transparent Security Studies Research, Security Studies 23:4 (2014): 689-698, DOI: https://doi.org/10.1080/09636412.2014.970405; and Diana Kapiszewski & Dessislava Kirilova, “Transparency in Qualitative Security Studies Research: Standards, Benefits, and Challenges,” Security Studies 23:4 (2014): 699-707, DOI: https://doi.org/10.1080/09636412.2014.970408.

[2] Colin Elman and John M. Owen, “Symposium on Process Tracing: Note to Readers,” Security Studies 24:2 (2015), 199, DOI: https://doi.org/10.1080/09636412.2015.1036580; James Mahoney, “Process Tracing and Historical Explanation,” Security Studies 24:2 (2015): 200-218, DOI: https://doi.org/10.1080/09636412.2015.1036610; Nina Tannenwald, “Process Tracing and Security Studies,” Security Studies 24:2 (2015): 219-227, https://doi.org/10.1080/09636412.2015.1036614; Andrew Bennett, “Using Process Tracing to Improve Policy Making: The (Negative) Case of the 2003 Intervention in Iraq,” Security Studies 24:2 (2015): 228-238, DOI: https://doi.org/10.1080/09636412.2015.1036619; and David Waldner, “Process Tracing and Qualitative Causal Inference,” Security Studies 24:2 (2015): 239-250, DOI: https://doi.org/10.1080/09636412.2015.1036624.

[3] Andrew Bennett, Colin Elman, and John M. Owen, “Symposium on Counterfactual Analysis: Note to Readers, Security Studies 24:3 (2015), 377, DOI: https://doi.org/10.1080/09636412.2015.1070623; Jack S. Levy, “Counterfactuals, Causal Inference, and Historical Analysis,” Security Studies 24:3 (2015): 378-402, DOI: https://doi.org/10.1080/09636412.2015.1070602; Richard Ned Lebow, “Counterfactuals and Security Studies,” Security Studies 24:3 (2015): 403-412, DOI: https://doi.org/10.1080/09636412.2015.1070605; Frank Harvey, “What If” History Matters? Comparative Counterfactual Analysis and Policy Relevance Security Studies 24:3 (2015): 413-424, DOI: https://doi.org/10.1080/09636412.2015.1070606; and Francis J. Gavin, “What If? The Historian and the Counterfactual,” Security Studies 24:3 (2015): 425-430, DOI: https://doi.org/10.1080/09636412.2015.1070610.

[4] Andrew Bennett, Colin Elman & John M. Owen, “Symposium on Qualitative and Multi-Method Research: Note to Readers,” Security Studies 25:1 (2016), DOI: https://doi.org/10.1080/09636412.2016.1134013; Gary Goertz, “Multi-Method Research,” Security Studies 25:1 (2016), DOI: https://doi.org/10.1080/09636412.2016.1134016; Hein Goemans and William Spaniel, “Multi-Method Research: The Case for Formal Theory, Security Studies 25:1 (2016), DOI: https://doi.org/10.1080/09636412.2016.1134176; Tanisha M. Fazal, An Occult of Irrelevance? Multi-Method Research and Engagement with the Policy World,” Security Studies 25:1 (2016), DOI: https://doi.org/10.1080/09636412.2016.1134186; Jason Seawright, “Better Multi-Method Design: The Promise of Integrative Multi-Method Research,” Security Studies 25:1 (2016), DOI: https://doi.org/10.1080/09636412.2016.1134187.

[5] Arthur Lupia and Colin Elman, “Openness in Political Science: Data Access and Research Transparency –Introduction,” PS: Political Science & Politics 47:1 (January 2014): 19-42, DOI: https://doi.org/10.1017/S1049096513001716.

[6] Colin Elman and Diana Kapiszewski, “Data Access and Research Transparency in the Qualitative Tradition,” PS: Political Science & Politics 47:1 (January 2014): 43-47, DOI: https://doi.org/10.1017/S1049096513001777.

[7] Frank Harvey, Explaining the Iraq War: Counterfactual Theory, Logic, and Evidence (Cambridge: Cambridge University Press, 2012).

[8] Niall Ferguson, “Introduction: Virtual History: Towards a ‘chaotic’ theory of the past,” in Niall Ferguson, editor, Virtual History: Alternatives and Counterfactuals (New York: Macmillan, 1997, 1-90, 87, emphasis added.

[9] Jack S. Levy, Counterfactuals and Case Studies, in Janet M. Box-Steffensmeier, Henry E. Brady, and David Collier, eds., The Oxford Handbook of Political Methodology, 627-644, DOI: https://doi.org/10.1093/oxfordhb/9780199286546.003.0027, 636-7, emphasis added.

[10] Mueller’s description may be a better fit for European research communities.

[11] See for example, Colin Elman and Miriam Fendius Elman, Bridges and Boundaries: Historians, Political Scientists, and the Study of International Relations (Boston: MIT Press, 2001); Colin Elman and Miriam Fendius Elman, “The Role of History in International Relations,” Millennium – Journal of International Studies 37:2 (December 2008): 357-364, DOI: https://doi.org/10.1177/0305829808097644; and Colin Elman and Miriam Fendius Elman, “Diplomatic History and International Relations Theory: Respecting Difference and Crossing Boundaries,” International Security 22(1) (Summer 1997): 1-16, DOI: https://doi.org/10.2307/2539324.

[12] Richard J. Evans, Altered Pasts: Counterfactuals in History (Waltham: Brandeis University Press, 2013), Kindle Location [henceforth KL] 2142-2145.

[13] Niall Ferguson (ed.), Virtual History: Alternatives and Counterfactuals (London: Macmillan, 1997).

[14] Evans, Altered Pasts, KL 602-603.

[15] Ibid., KL 645-646.

[16] Ibid., KL 649-673.

[17] Ibid., KL 1129-1135.

[18] Ibid., KL 1861-1875.

[19] Ibid., KL 1591-1594.

[20] Ibid., KL 786-800. Emphasis added.

[21] Ibid., KL 1652.

[22] Ibid., KL 1053-1058.

[23] See e.g., Ian Kershaw, Fateful Choices: Ten Decisions that Changed the World, 1940-1941 (New York: Penguin, 2007).

[24] Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable (London: Penguin/Allen Lane, 2007), 8, 13f.

[25] Ibid., 62-84.

[26] Baruch Fischoff and Ruth Beyth, “I Knew It Would Happen: Remembered Probabilities of Once-Future Things,” Organizational Behavior and Human Performance, 13, 1 (February 1975), 1-16.

[27] Evans, Altered Pasts, KL 1947-1949.

[28] Ibid., KL 2005-2019.

[29] Hugh Trevor-Roper, “History and Imagination,” in Valerie Pearl, Blair Worden and Hugh Lloyd-Jones (eds.), History and Imagination: Essays in Honour of H.R. Trevor-Roper (London, [publisher tk], 1981), 363-365. Emphasis added.

[30] Richard J. Evans, The Coming of the Third Reich (London: Penguin Publishing Group, 2005), KL 186-191.

[31] Evans, The Coming of the Third Reich, KL 343-344.

[32] Evans, Altered Pasts, KL 2063-2066.

[33] Richard J. Evans, The Third Reich at War: 1939-1945 (London: Penguin Publishing Group, 2009), KL 12189-12190.

[34] Evans, Altered Pasts, KL 2034-2036, 2049-2051.

[35] Philip E. Tetlock & Aaron Belkin (eds.), Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives (Princeton: Princeton University Press, 1996).

[36] Jack S. Levy, “Counterfactuals, Causal Inference, and Historical Analysis”, Security Studies 24:3 (2015): 378-402.

[37] “Introduction,” in Ferguson (ed.), Virtual History, 87.

[38] Frank Harvey, Explaining the Iraq War: Counterfactual Theory, Logic and Evidence (Cambridge: Cambridge University Press, 2011).

[39] Francis J. Gavin, “What If? The Historian and the Counterfactual,” Security Studies, 24:3 (2015), 429.

[40] R. G. Collingwood, My Autobiography (Oxford: Oxford University Press, 1939), 111-115.

[41] Ibid., 96.

[42] Ibid., 110.

[43] “Introduction,” in Ferguson (ed.), Virtual History, 87.

[44] Max Weber, The Methodologies of the Social Sciences (Glencoe: The Free Press, 1949). 173.

[45] Benjamin Mueller, At Cold War’s End: Complexity, Causes, and Counterfactuals. Ph.D. thesis. (London: The London School of Economics and Political Science, 2015). Available online at http://etheses.lse.ac.uk/3260/.

[46] Richard Ned Lebow, Forbidden Fruit: Counterfactuals and International Relations (Princeton, NJ: Princeton University Press, 2010). 29.

[47] Jack S. Levy, “Counterfactuals, Causal Inference, and Historical Analysis,” Security Studies 24:3 (2015): 378-402.

[48] Frank Harvey, “What If History Matters? Comparative Counterfactual Analysis and Policy Relevance,” Security Studies 24:3 (2015): 413-424.

[49] Frank Harvey, Explaining the Iraq War: Counterfactual Theory, Logic and Evidence (Cambridge: Cambridge University Press, 2011).

[50] Alexander L. George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences (Cambridge: MIT Press, 2005).

[51] Jeffrey Goldberg. “The Obama Doctrine,” The Atlantic Monthly April 2016. Available online at http://www.theatlantic.com/magazine/archive/2016/04/the-obama-doctrine/471525/ .

[52] Ibid.

[53] Ibid.

[54] See, for instance, Richard Feynman, QED: The Strange Theory of Light and Matter (Princeton, NJ: Princeton University Press, 1985).  I realize that I am opening myself here to the accusation of a whole new form of physics envy…

[55] Robert Jervis, “Counterfactuals, Causation, and Complexity,” in Counterfactual Thought Experiments in World Politics, ed. Philip Tetlock and Aaron Belkin, 309-316.

[56] Deirdre McCloskey, “History, Differential Equations, and the Problem of Narration,” History and Theory 30:1 (1991): 21-36.

[57] Geoffrey Hawthorn, Plausible Worlds: Possibility and Understanding in History and the Social Sciences (Cambridge: Cambridge University Press, 1991).

[58] Francis J. Gavin, “What If? The Historian and the Counterfactual,” Security Studies 24:3 (2015): 425-430.