Imagine a cabinet-level policymaker facing a complex strategic challenge that has drawn the attention of the president of the United States. The policymaker encounters what appear to be contradictory assessments. The Intelligence Community offers persuasive forecasts about an adversary’s likely behavior based on current evidence, but scholars present theoretical and historical insights that imply different outcomes. How should the policymaker weigh these competing forms of knowledge?
This scenario depicts a recurring challenge. Policymakers often have access to credible but conflicting assessments from intelligence analysts, social scientists, and historians, each offering a distinct form of authoritative knowledge. The challenge is not necessarily deciding which profession is right; each offers useful insights from a different epistemological vantage point. What is missing is an analytic framework that embraces these epistemological differences to generate integrated judgments that offer value beyond the capacity of any single approach used alone.
The Intelligence Community is ideally positioned to perform this integration function but lacks a framework to do so systematically and at scale. Intelligence analysts produce tailored assessments that draw on classified reporting, open-source information, and familiarity with the policy environment. They also regularly convene forums that organize intelligence and scholarly communities to discuss policy problems. Together, these roles uniquely position the Intelligence Community to lead integration, but it rarely does so due to the absence of a guiding framework. By default, intelligence analysts focus on providing knowledge for specific cases or immediate issues, often overlooking the causal patterns identified by social scientists across cases and the contextual depth offered by historians who probe particular episodes. It is unrealistic for senior policymakers, operating under severe time constraints and pressure, to integrate fragmented insights from intelligence analysts, social scientists, and historians. A more helpful approach is to have the Intelligence Community assume this responsibility and create a framework to synthesize perspectives grounded in fundamentally different epistemologies.
This proposal is not the same as the familiar call to “bridge the gap” between scholars and practitioners through greater engagement.1 A group of scholars recently defined this concept as “efforts to connect the knowledge or expertise of university-based scholars to the concerns or responsibilities of policy practitioners and the broader public.”2 Advocates of that approach often urge scholars to publish in nonacademic outlets, learn the policymaking process, serve in government, and add policy recommendations to academic articles.3 Others emphasize educating current or future practitioners on how academic knowledge can shape their work.4
These recommendations share a common underlying assumption: that the gap between scholars and practitioners is a problem and needs to be narrowed through mutual exposure. Those who hold this assumption often treat greater interaction as the goal itself, rather than ensuring that such interactions are analytically productive. The result is more conversations but not necessarily better analysis. For example, Daniel Byman and Matthew Kroenig contend that scholarship becomes policy relevant by becoming “part of the conversation” within government.5 While this is a critical threshold, there are limits to what more conversations can achieve without deeper reflection on what each profession uniquely contributes to understanding national security problems.
Framing the goal as “embracing the gap” between scholars and practitioners redirects efforts toward a more productive foundation for integration. Scholars and practitioners studying hard national security problems should preserve epistemological differences as sources of comparative advantage, rather than treating them as obstacles to overcome. These differences are precisely what make integration valuable. When scholars try to think more like practitioners—or vice versa—they sacrifice the very thing that makes their contributions unique.
Two examples illustrate the problem. The first is the call for scholars to formulate policy recommendations when publishing academic articles. This forces scholars to venture beyond their comparative advantage and adopt a policymaker’s perspective, while relying solely on theoretical or historical knowledge.6 Making policy recommendations requires an understanding of how US statecraft works, which scholars may or may not have developed. It also requires awareness of the latest intelligence on the issue at hand. The second example is the call for scholars to pursue government service. This risks distracting scholars from their core competencies and is an inefficient way to inject academic insights into government.7 The true challenge lies not in narrowing the gap between scholars and practitioners, but in systematically harnessing their differences to produce integrated analysis.
Additionally, much of the bridging literature treats the policymaking community as the primary audience for academic research, overlooking other types of practitioners. Advocates of policy-relevant research typically envision influencing senior policymakers, such as cabinet officials and National Security Council staff. Practitioners, however, serve different functions: Some make policy decisions, others provide analytic support to inform those decisions, and still others implement policy through military operations or diplomatic engagement. Intelligence analysts fall into the second category. They synthesize diverse forms of knowledge, refrain from policy advocacy, and provide decision support, rather than make policy decisions themselves. Delivering scholarship directly to policymakers, while bypassing the Intelligence Community, risks overwhelming policymakers with fragmented inputs and burdening them with the task of synthesizing these inputs. For many types of national security scholarship, integration through intelligence analysis offers a more productive pathway than direct engagement with senior policymakers.
This article offers two core insights. First, effective integration of intelligence analysis, social science, and history for national security purposes requires leveraging the comparative advantages of each profession. This can only occur through an analytic framework that embraces the unique epistemological foundations of how these professions produce knowledge. Second, the Intelligence Community should perform this integration function, as an extension of its existing integrative role. Intelligence analysts already synthesize classified and open-source information, technical and human intelligence, tactical and strategic perspectives, and interagency inputs. Social science and history represent additional sources of knowledge requiring integration, albeit ones that produce knowledge quite differently. While the Intelligence Community already conducts regular academic outreach, integration requires an analytic framework that accounts productively for epistemological differences.8
Epistemological Differences
What caused World War II in Europe? A seemingly simple historical question reveals profound differences among the epistemologies—the theories of knowledge—that guide intelligence analysts, historians, and social scientists. Understanding these differences is essential for any framework that seeks to integrate the work of the three professions.
To an Allied intelligence analyst in 1939, the answer was straightforward: Adolf Hitler. He consolidated power, personally directed German foreign policy, ordered rearmament in violation of the Treaty of Versailles, and made his intentions clear to military leaders. The causal chain from Hitler’s decisions to the outbreak of war would have appeared direct and observable. An intelligence analyst at the time would likely conclude that removing Hitler from power could prevent war in Europe, a judgment grounded in extensive evidence of the leader’s intentions and actions.
Historians and social scientists, however, apply different standards when making claims—explicit or implied—about causation. To determine whether Hitler caused the war, one must ask the counterfactual question: Would war have occurred without him? Although historians debate whether it is appropriate for them to engage in counterfactual analysis, their work can nonetheless yield insights that inform such analysis.9 For example, historian Francis J. Gavin recounts that his PhD advisor highlighted the release of the private papers of Gustav Stresemann, Weimar Germany’s foreign minister from 1923 to 1929. Stresemann won the Nobel Peace Prize for his reconciliation efforts toward France, but his papers revealed that he harbored territorial ambitions similar in some respects to Hitler’s, particularly the reincorporation of German-speaking territories into Germany. Achieving this goal would likely have required war. This, Gavin concludes, challenges the “widely accepted counterfactual—that if Hitler had not risen to power, war in Europe could have been avoided.”10
Dale Copeland reinforces this conclusion through systematic comparison across other cases of war among major powers, illustrating the comparative advantage of a social science approach. He offers a theory, tested across thirteen major wars and crises, that identifies a structural factor driving preventive war: dominant military powers anticipating a “deep and inevitable” decline.11 Copeland demonstrates that Nazi Germany fit the broad pattern observed in these thirteen cases. German military power peaked by 1939–40 but faced inevitable decline relative to an industrializing Soviet Union. German military elites shared Hitler’s fear of a rising Russia, a fear that predated the Nazi regime and drove their predecessors to push for war in 1914. Copeland concludes that these structural conditions, which recur in conflicts from ancient Greece to the nineteenth century, would have pressured any German leader toward preventive war. Hitler’s leadership and Nazi ideology explain the manner in which the war was conducted, in particular the atrocities, but they were not the main causes of war in Europe.12
“Intelligence analysts, social scientists, and historians diverge sharply in how they think about contemporary issues, not just historical ones.”
This example from World War II illustrates the challenges and opportunities involved in integrating distinct forms of knowledge. Intelligence analysts, social scientists, and historians diverge sharply in how they think about contemporary issues, not just historical ones. Intelligence analysts typically seek a better understanding of the actors, institutions, and circumstances involved in the immediate issue at hand. In contrast, social scientists examine similar cases throughout history to find general patterns in the relationships between key variables, or run experiments that seek clear causal identification of a given phenomenon. Historians emphasize the contextual factors that make past episodes unique, while carefully drawing out parallels to contemporary circumstances. These approaches reflect varied standards for what constitutes valid knowledge. As a result, intelligence analysts, social scientists, and historians may investigate the same phenomenon but produce assessments that appear to be in conflict.13
Intelligence analysts occupy a distinct epistemological position relative to social scientists and historians.14 Analysts provide assessments for government officials (treated as clients) under strict deadlines, even when uncertainty is significant and evidence is insufficient. This mission limits what questions analysts pursue. Policy relevance is the main criterion for useful knowledge, which often results in prioritizing case-specific expertise and future-oriented judgments. When asked whether a foreign leader will order an invasion, the analyst cannot respond with a history lesson or discussion of analogous cases. Policymakers need to know what this specific foreign leader will likely do, not what other actors have typically done in similar circumstances.15 Historical insights and theories can inform intelligence assessments, but they are not sufficient grounds to reach a conclusion. As Richards Heuer Jr. wrote in 1978: “The intelligence analyst is almost invariably concerned with the explanation and prediction of what he perceives to be unique events, not with searching for general patterns in events.”16 This observation largely remains true today.
Intelligence analysts do engage with history and social science, but their engagement reflects this case-specific orientation.17 Some draw on biographies, institutional histories, past behavior, and theoretical frameworks, but always to inform judgments about current conditions or future developments. Scholars often pursue knowledge as an end in itself, whereas intelligence analysts apply what scholars produce to inform the design and implementation of national security policies.
This shapes how the intelligence profession develops knowledge. Many analysts filter scholarly insights to meet the evidentiary requirements of making case-specific judgments for government officials. Others bypass academic work entirely, relying on reporting from sensors and human sources about the current situation to reach a final conclusion. Too often, however, intelligence analysts do not use a systematic approach when deciding which insights to include and which to screen out. Carmen Medina, a former CIA senior executive, calls this “analysis by anecdote.”18 The nature of the Intelligence Community’s mission incentivizes a certain epistemological orientation, one that leaves analysts vulnerable to seeing the world through a narrow perspective.
Social science is unique in privileging causal inference as the standard for what constitutes valid knowledge.19 Intelligence analysts and historians engage with questions of causality in their own ways, but social scientists are typically expected to make rigorous causal inference a core part of their work. Counterfactual reasoning is central to causal inference. Such reasoning involves establishing that X causes Y by comparing situations where X is present to situations where X is absent, while holding other factors constant. This is the logic underpinning the growth of experiments in political science,20 but not all issues lend themselves to experimental controls. When such controls are not feasible, social scientists can still try to approximate the counterfactual ideal by comparing cases as similar as possible except for the variable of interest. Knowledge emerges from patterns across cases, not from any single case. This points to a counterintuitive insight: Deeper understanding of a contemporary issue comes not from closer scrutiny of it but from looking elsewhere.21
Social science’s distinctive epistemology can be easily misunderstood by those outside the profession.22 Social scientists have a higher threshold than practitioners for determining whether a variable “matters” in international affairs. A variable may appear obviously causal to someone immersed in a single case, but it will remain theoretically insignificant until validated across other cases. Qualitative social scientists will seek to identify causal mechanisms within cases or conduct structured comparisons across a few cases; quantitative scholars will conduct so-called “large-N” studies to identify causal patterns across many cases; and experimentalists will look for opportunities where conditions for causal inference are strongest, such as when randomization isolates the effects of a variable of interest.
Practitioners cannot rely on the same threshold as social scientists and must account for variables that have not met the standards of causal inference. For example, when the effects of multiple variables constantly offset one another, any given variable will show no net effect, leading social scientists to register it as insignificant. But practitioners cannot ignore that variable because of its role in maintaining an equilibrium in the international system. A variable that looks insignificant under social science standards may be a top priority for policymakers, military officers, and intelligence analysts. The stability that such a variable underwrites is something practitioners cannot take for granted.
Historians occupy an epistemological position oriented toward contextual depth and an appreciation for the uniqueness of past events. They often share with intelligence analysts a skepticism about excessive generalization and theorizing across cases. At the same time, historians differ from analysts in their temporal orientation. As Jill Lepore has noted: “The American historical profession defines itself by its dedication to the proposition that looking to the past to explain the present falls outside the realm of serious historical study.”23 This reflects a commitment to understanding past events on their own terms, rather than using them to guide contemporary statecraft or make broader generalizations. This is not to say that historians reject all engagement with current issues, but there is a limit to what history alone can provide government officials. Michael Scriven reminds us that “history teaches us . . . about possibilities rather than regularities.”24 Similarly, Philip Zelikow cautions that history is “only useful for suggesting what is possible, not what is probable.”25
When applied to contemporary issues, historical reasoning takes different forms. Zelikow distinguishes between direct historical reasoning—examining the history of the actors at hand—and indirect reasoning based on analogies to other cases.26 Each form of historical reasoning draws on a different source of knowledge and requires a specific type of inference. Direct reasoning emphasizes learning from the decisions and past behavior of a specific actor to understand that actor’s current circumstances. Sometimes, “windows of opportunity” for current policy problems become apparent only through such reasoning, as Hal Brands and Jeremi Suri argue.27
Analogical reasoning, by contrast, rests on the principle that insights from past cases can inform the present, even when the actors are different. The challenge is to establish that the circumstances of historical cases are sufficiently comparable to the present situation to justify transferring lessons. Brands and Suri contend that analogies can generate initial insights but also caution that such insights require careful inquiry into “how the present is both similar and different” from the past.28
Framework To Integrate Intelligence, Social Science, and History
Effective integration requires embracing the gaps among intelligence analysts, social scientists, and historians. These gaps represent epistemological differences that, if properly leveraged, become sources of comparative advantage. The goal should be to harness the productive friction that emerges when fundamentally different approaches to knowledge production collide. This collision of paradigms can generate questions that no single profession would ask otherwise, resulting in new forms of analytic reasoning not widely practiced. Done well, integration expands the epistemic frontier, producing an outcome that is more than the sum of its parts.
Existing Intelligence Community directives already encourage integration, but their interpretation and implementation have been too narrow. Intelligence Community Directive (ICD) 203 sets the standard of using all sources of information,29 and ICD 205 directs engagement with outside experts to solicit unique insights unavailable internally.30 These directives, however, do not specify how to integrate distinct forms of knowledge productively. Extending current practices across epistemological boundaries—and systematically generating new questions and triggering new forms of analytic reasoning—would better realize the intent of existing directives.
The Intelligence Community can promote integration by inviting scholarly input to strengthen four analytic functions: idea generation, assumption checks, probabilistic judgments, and exploratory analysis. Scholars can contribute more productively through these functions than by trying to provide advice to policymakers or answers to intelligence questions.31 Providing academic work directly to policymakers risks overwhelming them with fragmented inputs and burdening them with synthesis tasks better suited to intelligence analysts. Furthermore, scholarship cannot be inserted into the main judgments in intelligence products. Instead, academic work should focus on shaping how intelligence analysts think about issues by informing their questions, assumptions, probability estimates, and organizing logics. This approach will ensure that policymakers receive assessments that integrate, rather than merely aggregate, the distinct insights of intelligence analysts, social scientists, and historians. Each of the four analytic functions presents opportunities to leverage these professions’ comparative advantages and develop integrated intelligence judgments that would not otherwise exist at scale.
Analytic Function 1: Idea Generation
Idea generation seeks to initiate new analytic inquiries that expand what the Intelligence Community monitors, both by generating new hypotheses for existing intelligence requirements and by identifying entirely new requirements. Scholars ask questions that practitioners may never consider, not because practitioners lack imagination, but because their epistemological orientation directs attention in specific ways. Intelligence analysts derive their questions from the needs of policymakers, expressed through formal channels or informal requests. Analysts and policymakers share an epistemological foundation: Both privilege knowledge that is case-specific, focused on immediate issues, and relevant to pending decisions. The similarities between analyst and policymaker epistemologies far exceed the differences, especially when contrasted with how social scientists and historians approach issues.
The Intelligence Community has a clear opportunity to leverage the epistemologies of scholars to reframe conversations and pursue new lines of inquiry that highlight emerging or overlooked challenges. One way is through social science puzzles, which can help reorient the Intelligence Community and motivate new hypotheses or requirements. A puzzle emerges when empirical patterns contradict conventional wisdom or theoretical expectations. Identifying the puzzle itself—even if scholars do not reach consensus on how to explain it—has intelligence value, as it alerts analysts to the possibility of overlooked factors.
“The Intelligence Community has a clear opportunity to leverage the epistemologies of scholars to reframe conversations and pursue new lines of inquiry that highlight emerging or overlooked challenges.”
For example, conventional wisdom once asserted that democratic countries enjoyed advantages in international crisis bargaining because their leaders faced electoral accountability—so-called “audience costs”—which made threats against other countries more credible.32 This implied that authoritarian leaders faced no comparable domestic accountability and could therefore bluff without consequences, making their threats less credible. Jessica Weeks challenged this by showing that certain authoritarian rulers do, in fact, face accountability from domestic coalitions of elites.33 This insight could help intelligence analysts update existing assessments of particular regimes by examining their internal accountability mechanisms, a line of inquiry that may not emerge without prompting from social scientists. Domestic accountability in authoritarian regimes remains an underused lens for assessing the credibility of threats from these regimes and, ultimately, their true intentions.
Inquiries like this leverage one of the Intelligence Community’s comparative advantages: the ability to gain direct insight into phenomena that scholars cannot study with post hoc data. Weeks uses proxy variables to operationalize the concepts of audience costs and the credibility of threats, reflecting the inability to measure them directly in a research context.34 In contrast, intelligence analysts can examine both concepts by drawing on classified reporting regarding the perceptions of regime elites. By obtaining insights into these perceptions, analysts are able to make judgments about audience costs and the credibility of threats that such costs generate. Productive synergy occurs when puzzle-driven theories drive the Intelligence Community to pursue inquiries with fewer measurement limitations than scholars face.
Historians contribute differently to idea generation by providing granular accounts that bring theories to life. While intelligence analysts can gain insight into present-day elite dynamics in authoritarian regimes, historians can illuminate how such dynamics operated in past cases. Social science theories may therefore trigger historical investigations that provide additional benefits to intelligence analysts. Historians could build on Weeks’s insights about authoritarian audience costs, such as her example of Soviet politics under Nikita Khrushchev,35 to more fully probe how Politburo dynamics operated in the Soviet Union: what factions existed, how information circulated, and how elites signaled dissatisfaction. Deeper historical contributions would therefore allow intelligence analysts to concretely visualize causal mechanisms and how they operated in the past.
Alternatively, historians can challenge theories or offer competing interpretations of past events. There is a natural tension between theorizing, which seeks to generalize across cases, and historical inquiry, which treats past events as unique. In some instances, the development of a theory may reflect what Williamson Murray calls “historically undisciplined theorizing.”36 A critical historian provides value by showing that a theory’s causal variable functions differently across cases, and that unique contextual factors may be what actually produce outcomes. This gives intelligence analysts a concrete question to consider: Will a causal variable function the same way in the present context? Analysts can then identify alternative hypotheses and associated indicators by synthesizing generalized explanations with historically grounded critiques. A theory’s weakness, once a historian engages with it, can become a source of analytic creativity.
Historians can also trigger new inquiries by the Intelligence Community without relying on prompts from social scientists. They can challenge prevailing analogies in the public discourse about national security issues, drawing on a deep understanding of the past to show where such analogies are wrong or misleading.37 For example, Melvyn Leffler argues that the Cold War is a “profoundly wrong” and “dangerous” analogy for current US-China competition.38 He proposes instead that China resembles a rising United States at the turn of the twentieth century, when the world’s dominant power at the time, the United Kingdom, chose to accommodate Washington.39 Hal Brands and John Lewis Gaddis provide a different perspective from Leffler’s, arguing that the Cold War remains a useful guide despite differences between then and today.40 Debates among historians expose the unstated assumptions in analogies and reveal factors that would otherwise go unexamined. For intelligence analysts only familiar with the Cold War analogy, Leffler’s challenge offers a new mental model to explore US-China relations.
Competing historical analogies do more than offer new frames that expand the range of factors to consider; they establish reference points for monitoring how a strategic situation could evolve. Intelligence analysts do not need to adjudicate which analogy is correct. Instead, they should track whether emerging developments are moving closer to the circumstances associated with any particular analogy. Historians define the factors and dynamics that make past cases unique, while intelligence analysts make judgments about whether current conditions are shifting toward the defining features of one analogy over another.
The interaction could unfold in a different sequence. If intelligence analysts detect shifts in the present situation that existing analogies cannot adequately explain, historians may be able to identify new frames that more closely resemble the conditions analysts are observing. The value of history lies not in forcing a choice between analogies, but in using competing or new analogies as tools to monitor emerging developments.
Analytic Function 2: Assumption Checks
Intelligence analysts can share assumptions with scholars in unclassified settings, enabling integration in a unique way by opening these assumptions to outside scrutiny. Without such sharing, academic outreach by the Intelligence Community risks being one-sided: Scholars impart knowledge, while analysts ask questions but remain limited in what they can disclose. The result is conversations that lack reciprocity and productive synergy.
Using assumptions to guide the agenda has the potential to change this dynamic. Assumptions are typically shareable with outside experts, as long as analysts do not reveal the questions they are officially tasked to answer, the conclusions they have reached, and the sources and methods underpinning those conclusions. Furthermore, assumptions often have subordinate components that serve as potential agenda items for discussion. For example, the assumption that a ruling coalition in an authoritarian regime will remain intact may depend on factors such as elite power-sharing agreements, succession plans, or the strength of institutions. Table 1 contains examples of assumptions that could be further disaggregated and used as the basis for academic outreach sessions.
Table 1. Example assumptions
| Assumption type | Example |
|---|---|
| Political continuity | A political party will remain in power |
| Trend persistence | Previous trends or rates of change will continue to hold |
| Decision-making priorities | A leader’s calculations will prioritize certain considerations (economic growth, domestic politics, or operational realities) |
| Conflict duration | A ceasefire will not occur; the war will continue for a specified period |
| Bureaucratic consensus | The defense ministry and foreign ministry will continue to agree on an issue |
| Stated intentions | Publicized goals and timelines for a project reflect actual goals and timelines |
Such purposeful interactions can help intelligence analysts check the validity of linchpin assumptions that underpin existing analytic conclusions. Assumption checking is foundational to analysis and a skill the Intelligence Community trains its analysts to apply regularly. An assumption that seems self-evident to an analyst specializing in a contemporary issue, however, may become debatable under scholarly scrutiny.
Consider a common assumption in ongoing discussions that link the war in Ukraine to a potential Taiwan contingency: the claim that displays of resolve in the past will lead other countries to infer resolve in the future.41 Social scientist Daryl Press challenges this perspective, arguing that present-day calculations about another country’s resolve are more likely to be based on capabilities and interests, rather than on that country’s past behavior on an unrelated issue.42 Whether or not intelligence analysts take this assumption to be true could shape how they assess the decision-making calculus of adversaries.
There is a unique synergy when intelligence analysts and historians jointly scrutinize social science claims that challenge existing assumptions. Many social science theories hold only under specific scope conditions. For example, a theory might apply only to declining powers, only to certain regime types, or only when vital rather than peripheral interests are at stake. The Intelligence Community can establish formal requirements to identify and compare current conditions to those specified by any given theory. The role of historians in this context is to surface nuances that both intelligence analysts and social scientists have overlooked, perhaps even factual errors. As Gavin notes, “Historians well understand what many social scientists often forget: getting the facts straight is essential and often close to impossible.”43
Assumptions require rigorous testing through a collaborative process because they are particularly vulnerable to error. Practitioners often rely on intuition, which functions like theory by guiding interpretation of evidence but has never been systematically critiqued and tested.44 While valuable, especially from experienced professionals, intuition can lead to flawed assumptions. Gavin warns that decision-makers routinely act on “unspoken assumptions”: beliefs so widely held that they remain unidentified and unchallenged.45 Former diplomat Stephen Del Rosso recounts how common this was: “There were theoretical underpinnings to policymaking, but the very practitioners who were engaged in that work could not articulate them.”46 In such situations, where intuition substitutes for theory, unexamined assumptions are common among intelligence analysts and policymakers. This is where social science and history are particularly useful. Even contradictory academic work can help reveal and test assumptions that would otherwise remain implicit.
Analytic Function 3: Probabilistic Judgments
Bayesian reasoning offers a novel way to organize the interaction of intelligence and scholarship to support probabilistic judgments. The form of reasoning itself is not new; it involves updating prior probabilities—the initial baseline estimates of likelihood—in light of new evidence, a process formalized mathematically in Bayes’ theorem. Previous applications in intelligence have focused on improving the internal reasoning of analysts and training them to think probabilistically.47 The potential for cross-disciplinary application of Bayesian reasoning, however, remains largely untapped.
“A division of labor will help realize that potential and improve probabilistic judgments in intelligence products.”
A division of labor will help realize that potential and improve probabilistic judgments in intelligence products. Social science is well suited to two core tasks associated with Bayesian reasoning: establishing prior probabilities and assessing the diagnostic value of evidence. Both tasks draw on aggregate data and patterns across cases, precisely the kind of work many social scientists specialize in. Historical knowledge, in turn, provides an important check on both tasks. This approach emphasizes qualitative logic that crosses epistemological boundaries, rather than strict mathematical calculations using Bayes’ theorem. This aligns with the views of intelligence practitioners and scholars who argue that Bayesian reasoning is valuable even without precise numerical calculations.48
Bayesian reasoning rests on two core concepts: prior probability and the diagnostic value of evidence. Prior probability is the baseline likelihood of a hypothesis before analysts consider case-specific evidence. It answers the question: How common is the outcome historically? When determined rigorously, the answer is based on the historical base rate of the phenomenon in question, something that social scientists and historians can help establish. When done less systematically, the answer relies on the intuition of intelligence analysts.
The diagnostic value of evidence refers to how well evidence distinguishes among possible outcomes. Evidence consistent with many outcomes is less informative than evidence uniquely tied to only one outcome. For example, troop movements near a border are consistent with invasion preparation, but similar movements occur during exercises, defensive posturing, or operations to suppress rebellions. As a result, troop deployments alone are not highly diagnostic. By comparison, other indicators are more diagnostic of offensive preparations: large deliveries of specialized medical supplies, evacuation of border communities, and prepositioned fuel in amounts that exceed exercise requirements.
Prior probability presents a challenge for intelligence analysts seeking to make future-oriented judgments. Analysts are often vulnerable to the base rate fallacy: an error in which individuals neglect the general prevalence of outcomes (base rate or prior probability) and overemphasize case-specific evidence. When the base rate of an outcome is low, even highly diagnostic or predictive evidence may not indicate that the outcome is likely. Without anchoring to proper base rates, analysts risk being misled by diagnostic evidence.
A scenario illustrates how a highly diagnostic sensor can be misleading. Suppose an adversary has installed a specialized component in 1 percent of a certain class of military vehicles (base rate). An intelligence sensor detects indicators associated with that component. Analysts are confident this component is in the area based on the sensor’s highly diagnostic 90 percent success rate, established through testing against vehicles containing the component. However, the sensor has a 10 percent false-positive rate because other vehicle configurations trigger similar readings. The actual probability of the component’s presence is 8.3 percent using Bayes’ theorem. An analyst who prioritizes the sensor’s results will significantly overestimate the probability that a vehicle with the specialized component has entered the area.
Prior probabilities, or base rates, also exist for more complex outcomes. Social scientists have compiled datasets on a wide range of outcomes: coups, civil wars, interstate wars, and war outcomes disaggregated by regime types, among many others. Such data allows analysts to anchor judgments in historical base rates before incorporating case-specific reporting from traditional sensors and human sources. Other data can establish base rates not only for the primary outcome of interest but for prerequisite or component events. For example, an analyst assessing the likelihood of regime collapse may need to consider base rates for events that frequently precede or correlate with collapse, such as elite defection, military mutiny, or mass protests. Because existing scholarship may not always provide base rates that are directly applicable to a particular intelligence question, the Intelligence Community may need to sponsor tailored research projects that address such questions.
The second concept underpinning Bayesian reasoning—diagnosticity—also benefits from scholarly input and mitigates an epistemological vulnerability of intelligence analysts. Social scientists can help assess whether an indicator is, in fact, diagnostic of a specific outcome, by examining how often the indicator appeared when the outcome occurred in history, compared to when it did not occur. As with base rates, the necessary social science research may not exist, in which case the Intelligence Community can sponsor studies tailored to its requirements. A more rigorous approach to evaluating diagnosticity will help analysts who might otherwise rely on untested intuition and become overconfident in any particular set of indicators. This avoids what Daniel Kahneman and Dan Lovallo call the “inside view,” which involves focusing on the case at hand while ignoring the “statistics of a class of cases chosen to be similar.”49
Historians can contribute to Bayesian reasoning by scrutinizing the historical parallels that intelligence analysts draw and assessing the reference classes that social scientists use. Intelligence analysts may assume that a current crisis resembles a particular set of historical cases used to build and test a theory. Historians can examine the plausibility of this assumption by identifying the parallels that contemporary practitioners can realistically draw from past episodes, as well as key differences.
In addition, historians can carefully assess whether the past cases themselves share enough relevant attributes to be grouped together as part of the same reference class. Social scientists use reference classes to group similar cases together to allow for an “apples-to-apples” comparison, which is important for effective counterfactual reasoning. Historians are ideally positioned to identify when proposed reference classes obscure critical differences or overlook cases that should be included.
Such historical scrutiny can surface a question that intelligence analysts rarely ask: whether a contemporary actor belongs in the reference class a theory was built on. For example, a theory’s reference class—the types of cases the theory covers—might consist of countries where the governing elite’s interests diverge from those of the country as a whole. This contrasts with theories that treat countries as having a unified set of national interests. An intelligence analyst facing a contemporary problem would need to determine whether an adversarial country fits the reference class established by the theory. This inquiry may warrant a formal intelligence requirement, one that analysts might overlook without interaction with social science and history.
Analytic Function 4: Exploratory Analysis
Social science and history can facilitate what the Intelligence Community calls exploratory analysis. This type of analysis examines a range of outcomes without making probabilistic judgments. It is useful when several conditions make determining probability infeasible: Evidence is too scarce, an issue is new or still developing, or the outcome of interest is conditional on an unlikely event. By surveying what could happen without requiring probability estimates, exploratory analysis promotes creativity and mitigates strategic surprise.
There are two ways to conduct exploratory analysis. One is to assume that an unlikely event has already occurred and then trace plausible pathways that could have produced it. Sometimes, analysts will also note unique implications associated with each pathway. Another approach is to describe several scenarios for how current circumstances could evolve over time, without assigning probabilities or making a determination about which scenario is most likely. This approach is forward-looking from current conditions, rather than focused on pathways leading from the past to a hypothetical, unlikely event. What sets this apart from conventional forecasting is that it addresses ambiguous issues where uncertainty is too high to assign meaningful probabilities. Social science and history can provide the underlying logic for these creative endeavors, compensating for limited evidence or illuminating contingencies worth preparing for.
Dan Reiter’s work illustrates how social science theory could guide exploratory analysis.50 Reiter asks why some countries at war refuse to sue for peace even when losing on the battlefield, and proposes a theoretical framework that highlights two key dynamics. First, battlefield outcomes reveal information about relative strength, pushing the weaker side to make concessions. Second, countries weigh whether their opponent can be trusted to honor peace settlements. When an opponent is seen as untrustworthy, countries prefer to keep fighting despite military setbacks. Reiter’s framework focuses on the interaction of these dynamics, identifying the conditions under which one overshadows the other.
Intelligence analysts asked to explore the future trajectory of a war could find this theory useful in illuminating different possibilities, especially when a probabilistic judgment is not feasible. What happens when the war begins producing convergent beliefs about relative strength, but distrust is high? What happens under other combinations of these factors? Answers to these questions can shape Intelligence Community assessments of whether particular assurances or enforcement mechanisms could shift the calculations of wartime leaders.
Historians can deepen the logic that drives exploratory analysis, whether that involves identifying pathways that produce an unlikely event or constructing future scenarios. Historical insights reveal how past episodes aligned with, or diverged from, the causal logics specified in social science theories. Historians also identify factors those theories leave out, such as details regarding the sequencing of political decisions, political maneuvers that preceded major turning points, and the influence of third parties external to a particular conflict. These insights will keep exploratory analysis grounded in documented accounts of how other actors have behaved in the past, rather than left to pure speculation.
History is a particularly valuable source of vicarious experience for exploratory analysis on wartime issues. As Williamson Murray asserts: “The military is the only profession that does not get to practice its profession on a regular basis.”51 Unlike physicians or engineers, military professionals cannot always accumulate experience through repetition in their craft, as many militaries rarely participate in war. History therefore compensates for the lack of direct experience by offering insights from past wars. Intelligence analysts face a similar situation; they develop exploratory scenarios and pathways without always possessing experience in analyzing wartime topics. Under such circumstances, history becomes the principal substitute for that missing experience.
Evaluating Scholarship for Integration Purposes
Scholarship must undergo evaluation before it can inform the four analytic functions discussed above. Yet evaluation of scholarly work for integrative purposes requires a unique form of analytic tradecraft that has not been institutionalized in the Intelligence Community. Academic literature on any given topic is often incomplete, fragmented, and contradictory, and each individual study has limitations. A core challenge for the Intelligence Community is making sense of competing scholarly arguments and synthesizing findings that were never designed to speak with one voice.
Research on the impact of targeting terrorist leaders (so-called “decapitation strikes”) illustrates this problem. Social science studies differ along several dimensions, making them difficult to reconcile. Existing research examines the impact of decapitation strikes by measuring different outcomes: organizational collapse or decline,52 alliance termination,53 and tactical shifts.54 Some studies condition their findings on group attributes, with some attributes mitigating the consequences of decapitation strikes and others exacerbating them.55 Moreover, studies rely on different methods and samples to make generalizations about the impact of killing terrorist leaders. Historians may further complicate the picture by arguing that unique, case-specific factors ultimately determine outcomes, not the generalizations about cause and effect that social scientists derive from comparing cases. The Intelligence Community must evaluate how scholars arrive at these findings, rather than merely engaging the findings themselves. The goal is not to adjudicate scholarly debates, but to identify their implications for specific national security problems where decision support is needed.
Evaluating scholarly studies for intelligence purposes requires analytic tradecraft built around epistemological and methodological awareness. Current tradecraft for evaluating traditional intelligence sources lacks this awareness and is ill-suited for scrutinizing academic work. In addition to having a particular epistemological foundation, every scholarly study makes certain methodological choices about what to examine, what to exclude, and how to gather and interpret evidence. Scholars need to master a particular methodology to produce findings, but intelligence analysts must also understand it enough to evaluate whether—and how—those findings can inform intelligence assessments.
Before applying a theory, for example, intelligence analysts must determine whether their specific case resembles the sample used to test the theory. If the intelligence problem is significantly different from the sample, the theory may not apply. This difference means that the conditions making the theoretical argument valid may be absent in a specific, present-day crisis. The same applies to indirect historical reasoning involving analogies: The past case must resemble the current one to justify transferring lessons.
Another methodological challenge involves understanding the outcomes studied by scholars, compared to those examined by intelligence analysts. Social science theories often explain narrower outcomes than what practitioners face, in part because doing so allows researchers to isolate the effects of causal variables across cases. In contrast, practitioners are accountable for safeguarding national security in full, so they must address every dimension of a problem. Moreover, they cannot isolate variables or choose which conditions to confront; every factor has potential consequences, even those treated by academics as theoretically insignificant in the aggregate.
The work of Todd Sechser and Matthew Fuhrmann illustrates how even rigorous theories can only provide partial insights into multifaceted policy problems. Sechser and Fuhrmann argue that nuclear weapons are ineffective in compelling adversaries to make concessions, while recognizing that such weapons retain value for deterrence.56 The precision with which they bounded the outcome of interest is necessary for rigorous causal inference. However, policymakers facing a decision must weigh other potential outcomes that shape the cost-benefit calculus: escalation, proliferation, and damage to alliance cohesion, among others. No single study will address the full range of factors that intelligence analysts must integrate before presenting assessments to decision-makers. The Intelligence Community must therefore accumulate and reconcile partial insights across a fragmented scholarly literature, while synthesizing those insights with classified intelligence reporting.
A final evaluative challenge lies in translating numerical findings from social science to intelligence settings. A social science finding that X causes Y in 80 percent of examined cases is a conclusion about the sample, not a probability that X will cause Y in a contemporary case. Intelligence analysts may face a specific case that falls outside the study’s scope conditions, differs significantly from the sample, or involves variables the study did not account for. Even for a study with strong internal validity, the 80 percent finding is still logically different from the probability of a causal relationship existing in any particular case. The Bayesian approach described earlier—where social science establishes base rates and helps assess the diagnostic value of evidence—offers one way to properly translate such findings.
Conclusion
This article has presented a framework to integrate intelligence analysis, social science, and history in support of policymakers. Such a framework rests on a fundamentally different conception of what is required for integration than the predominant emphasis in scholarly and policy literature. The prevailing approach of “bridging the gap” treats differences between scholars and practitioners as a problem requiring mitigation through greater mutual exposure. In contrast, the framework outlined here embraces differences among professions—specifically, epistemological differences—as sources of comparative advantage to leverage, rather than obstacles to overcome.
“Maximizing this potential requires reframing the challenge from 'bridging' to 'embracing' the scholar-practitioner gap.”
Leveraging these comparative advantages, however, requires more than increasing engagement; simply bringing scholars and practitioners together more frequently is insufficient. Greater engagement is certainly an improvement over passively assuming that scholarly insights will naturally “trickle down” from universities to analysts and policymakers.57 But meaningful collaboration demands attention to the epistemological foundations that shape how each profession produces and evaluates knowledge. It also requires designing scholar-practitioner interactions deliberately, so that they yield analytic value rather than mere conversation. Without such attention and design, scholar-practitioner engagements will fail to realize their full potential.
Maximizing this potential requires reframing the challenge from “bridging” to “embracing” the scholar-practitioner gap. While “bridging” is a worthy goal, it is too often operationalized through an emphasis on increased engagement and mutual exposure, an approach that ignores or downplays how differing epistemologies can serve as complements, rather than sources of conflict. This article’s reframing is therefore substantive, not merely rhetorical. It signals that productive integration requires leveraging differences and even encouraging them, instead of seeking to erode them.
The Intelligence Community is ideally positioned to harness epistemological differences to produce useful knowledge for policymakers. As noted above, delivering scholarship directly to policymakers is rarely ideal. Sending fragmented insights that have not been vetted against case-specific evidence or coordinated with other sources of knowledge burdens policymakers with integration tasks better suited to intelligence analysts. Expecting prudent policy decisions based on partial, uncoordinated inputs is also unreasonable. Moreover, even when routed through the Intelligence Community, social science and history do not provide ready-made answers to intelligence questions. Academic work requires further evaluation and synthesis before it becomes useful to intelligence analysts. In most cases, scholarship should focus on shaping how intelligence analysts think about issues and should serve as one of many inputs informing assessments of policy problems, instead of bypassing the analytic process entirely.
An analogy may help to clarify this point. A detective investigating a crime cannot rely solely on aggregate patterns in how similar crimes have occurred. Such patterns are analogous to the causal averages identified by social scientists and can inform the investigation but must be integrated with evidence specific to the case at hand. Detectives can also learn from history, drawing on knowledge of similar crimes in past decades or other jurisdictions, but must recognize the limits of such comparisons and treat them only as partial inputs. Patterns and past cases illuminate possibilities but are rarely enough to reach an investigative conclusion and solve a crime. Similarly, causal patterns and historical insights on national security issues are not enough to answer an intelligence question; answering such questions requires specialized analytic skills distinct from scholarly work.
Integration at scale will likely require a dedicated cadre of analysts within the Intelligence Community who possess both academic and analytic expertise. It is unrealistic to delegate the task of evaluating scholarship—navigating fragmented literatures, reconciling contradictory findings, and assessing methodological limitations—to the analytic workforce writ large. Instead, an Intelligence Community cadre should take on the burden of evaluating scholarly works and managing their integration, allowing the majority of analysts to focus on applying vetted scholarship through the four analytic functions described above. This protects against what Paul Musgrave calls a “political science lab leak”: the misapplication and oversimplification of social science by nonspecialists, who remove scope conditions and nuances from theories and therefore apply them incorrectly to current challenges.58
In addition to evaluating scholarship, the proposed cadre would directly inject scholarly insights into the workflows of intelligence analysts and provide guidance on the best use of such insights. Cadre members can scan a fragmented literature, evaluate research designs, match academic work to intelligence priorities, and sponsor new studies when needed. This requires specialized analytic tradecraft that will be difficult for many analysts to develop independently, and will therefore require some focused effort to establish.
The proposed integrative framework is not a guarantee of analytic accuracy, but it promotes awareness and transparency, and offers a new way to incorporate scholarship into intelligence analysis to productively inform policy. All academic work privileges certain variables or perspectives, adopts a particular causal logic (implicitly or explicitly), and makes methodological choices that shape findings. The goal is for intelligence analysts to apply theories and historical accounts in an informed manner. The four analytic functions explained here represent concrete opportunities for intelligence-scholarly integration—an approach that seeks not to narrow or bridge the scholar-practitioner gap, but to put that gap to work.
James Kwoun, PhD, is an active-duty lieutenant colonel in the US Army and a professor of strategic intelligence at the National Intelligence College, National Defense University. He is currently the director of the college’s core course in intelligence analysis. He previously served at the Defense Intelligence Agency, where he led a branch of analysts in support of national policymakers and taught analytic tradecraft as a senior instructor. His other assignments include the Joint Staff, the headquarters of two combatant commands, and US Army units at the brigade level and below. He has served overseas in the Republic of Korea, Iraq, and Afghanistan, and has traveled to three African countries to work with partner militaries. He earned his doctorate in political science from the University of Virginia.
National Intelligence College, National Defense University, Bethesda, MD, USA, email: james.s.kwoun.mil@ndu.ed
Opening image credit: lensnmatter, CC BY 2.0 via Wikimedia Commons.59