Abstract
For some years, US defense officials have admonished the modeling and simulation (M&S) community to go beyond modeling military “kinetics” to address complex challenges with labels such as stabilization, irregular warfare, counterterrorism, counterinsurgency, hybrid warfare, and the many shades of deterrence. Modeling such challenges must draw on the social and behavioral sciences, which are famously “harder” than the hard sciences (a point famously made by the late Nobel Laureate Herbert Simon). It is fair to say that the M&S community has not yet successfully met the challenge, despite significant efforts and bright spots. One obstacle has been the community’s relative unfamiliarity with methods suitable for modeling social-science phenomena.
This special issue of the JDMS includes seven papers that suggest some of the breadth of additional methods that can be brought to bear in security-related modeling, simulation, and analysis (MS&A), but that are not yet widely familiar. In our call for papers, we cast a wide net and received numerous submissions. Most of the seven papers that appear here are not classic journal papers describing the rigorous application of mature methods to a problem, with succinct listing of crucial assumptions. Most of the papers describe new and emerging methods, “illustrative” applications that are insightful but not definitive, speculations about how newer methods can help, and exploratory analyses to enlighten but not to predict with precision. Much work lies ahead, but the papers suggest great promise and also that much can be done now.
The research described in the papers was accomplished within government organizations, universities, and think tanks. Some was applied; some was more nearly basic research.
The paper by Yuna Wong and co-authors describes a unique effort by the Marine Corp Combat Development Center (MCCDC) to build an analytic baseline of a campaign involving irregular warfare. The authors began the study by fully embracing the social, cultural, and behavioral issues, as well as the more strictly military issues. From the outset, they pulled in active participants from the State Department and the larger interagency community, and from the “soft” academic discipline. The result was to shape the approach in a distinctly different way than is common. It was in contrast to a traditional modeler organizing around military tasks and, at the end, asking subject matter experts to estimate values of a few parameters that the modelers naively believe adequate to cover the “soft” issues. The team experimented with multiple social-science methods, while deliberately avoiding classic quantitative M&S. Much was learned.
Aaron Frank provides a history of how Department of Defense’s (DoD’s) Office of Net Assessment has long deplored traditional modeling and analysis for its failure to reflect issues of culture, organizational behavior, and heterogeneity, all of which can be crucial in net assessments. He argues that computational modeling, particularly agent-based modeling, has much to offer in representing such matters. Some such modeling can even be done at the individual level (i.e., “at scale”) when sufficient data is available. Frank acknowledges that the current state of the art has many problems, but sees great opportunities.
Carley, Morgan, and Lanham describe work at Carnegie Mellon University that demonstrates the analytic power of agent-based modeling, social network analysis, and belief-diffusion modeling to help understand nuclear weapons proliferation in the Pacific Rim. They demonstrate a flexible multi-level model in which the abstract level uses relatively austere causal relationships from some theories of deterrence. The more detailed layer can be instantiated for particular cases. Information for the particular case illustrated was obtained from expert elicitation and open-source data. Significantly, the model’s flexibility enables re-use across research teams or scenarios. The authors also illustrate how computational modeling can help one to assess the potential impact of alternative courses of action, at least for the limited set of causal pathways that they considered.
Davis and O’Mahony describe a body of recent work at RAND to improve integrative causal social-science modeling (as distinct from correlational approaches). The stream of work began with a large study pulling together the literature’s rich social-science knowledge on terrorism and representing it in structured but qualitative causal models (factor trees). Factor trees synthesize the factors identified in numerous more fragmentary studies. The goal is a holistic understanding of the factors and—roughly and with uncertainties—of how the factors combine to create effects. This theory-driven structuring also suggests more meaningful specifications for future quantitative data analysis: it suggests using a multifaceted and nonlinear model as the hypothesis, rather than a set of single-factor hypotheses. That is, it encourages empirical work to test and enrich theory, rather than empirical work to identify correlations. The authors illustrate the approach with a factor-tree model of public support for insurgency and terrorism, which was also tested subsequently with qualitative case studies. The authors then go on to demonstrate that factor trees can be extended to a new kind of uncertainty sensitive computational modeling that allows what ifs, broader exploratory reasoning, and cautious analysis. It provides a “system view” that can be helpful even though precise and reliable quantitative trade-off analysis is not feasible. The factor-tree approach is a deliberate simplification to promote reasoning, communication, and debate. Factor trees were inspired by influence diagrams and the causal loop diagrams of system dynamics, but they suppress dynamics and many network aspects to view matters at a snapshot in time. The work also illustrates the modularization of social-science knowledge in a high-level model that can be understood, debated, shared, and used for composition (typically with reprogramming).
Osoba and Kosko apply research methods developed at the University of Southern California to add dynamics to the Davis–O’Mahony model. They bring to bear a very different modeling approach, that of fuzzy cognitive maps (FCMs), which developed from the concepts of fuzzy logic. These FCMs include feedback loops that can help in exploring short-term and longer term implications of actions taken. Although more complex than factor trees, the methods are still relatively transparent and comprehensible. Another significant feature is that FCMs can be improved and tuned with machine-learning algorithms and time-series data. The authors have demonstrated that FCMs from multiple experts can be combined.
The final two papers describe dynamic models of behavior. Karimov and Matthews at RAND build on Matthews’ earlier research at Harvard to address diffusion of beliefs, behaviors, and technologies within dark networks. Their paper is the first systematic evaluation of the statistical performance of published methods for studying such diffusion effects, notably autoregression, dyadic regression with permutations, and dyadic regression with random effects. Their experiments made extensive use of synthetic datasets generated on two historical networks. The authors Bruce Keith (West Point) and David Ford (Texas A&M) apply the system dynamics method as part of informing policymaking in a specific rancorous international dispute about water management in an era of uncertainty about the effects of climate change. The disputes involve the Ethiopian Renaissance Dam Reservoir and water issues between Egypt and Ethiopia, issues that create regional tensions but that also relate significantly to regional economic development. The paper stems from work done when Keith was a Fulbright Scholar in Ethiopia.
Overall, the articles demonstrate that social and human behavioral considerations can be included in M&S, and that M&S can help identify relationships that may not otherwise be readily apparent in complex environments. The articles also offer suggestions on how to use both theories and empirical data well—sometimes to test and improve models and sometimes to generate empirical models directly when no satisfactory theory is available. That said, the challenges are formidable. There is no generally accepted set of theories that explain social behavior. Further, the fragmentary theories that do exist are often contradictory or incommensurate. Modelers cannot pull well-developed theories of causal relationships “off-the-shelf” and just “plug them in” (although claims on the matter are sometimes made). As a result, analysts need to think deeply about how to deal with model uncertainty, not just uncertainties about model inputs. Collecting data is also challenging, as most of the concepts of interest do not have easily observable empirical proxies. Analysts also need techniques for fusing heterogeneous data of variable quality. The articles in this special issue provide a different take on handling some of these challenges.
While we are impressed with the diversity of modeling approaches available for exploring national security questions requiring social-science insights, we note some additional problems. We have already mentioned that the approaches depend on confusingly different theories and assumptions. We are struck by how difficult it is to understand and compare the underlying conceptual models used in these methods. One major negative in modern-day computer-based M&S is that concepts and driving assumptions are often buried, inaccessible to readers. In data-driven work there may be little or no unifying theory, even when theory is needed for going beyond past data. As a specific contrast, there is no analog in much of the current work to a classic approach of describing a model in words and then in equations, following with a tight summary of input data and discussion of sensitivities, and then exposing the work to in-depth peer review. We see profound challenges of transparency, comprehensibility, and treatment of uncertainty. As the special issue indicates, however, progress is being made and we can hope for more convergence in the years to come.