Baseline Analysis: The Epistemology of Scenario Support

Introduction

The core principle of strategic foresight is that the future is multiple, not singular. Futurists describe the future as a set of plausible alternatives (scenarios) rather than as a single-valued prediction. Policy makers and the public at large prefer predictions because they are simple and easy to grasp. Predictions are singular just like the present or at least the way people experience the present. If there is to be one present in the future, then it seems logical that there is one future that will become the present. According to this logic, the task of forecasting is then to figure out which one it is. Second, they like predictions because they are often supported by evidence, like history and science. Predicting the future is just applying modern scientific techniques to human affairs. And, finally, predictions carry an air of certainty about them. They are simple, specific, and uncomplicated because they omit the inherent uncertainties about the future.

The major problem with predictions, of course, is that they are often wrong. In fact, predictions in social science are so often wrong that people rarely believe them even though they prefer them to multiple scenarios. Everyone says, “You can’t predict the future,” even though they do it all the time! People hold this paradoxical position because they do not know what else to do. Predictions are based on evidence, which can be examined, discussed, critiqued, and improved, the same process that other professions use. Traditional forecasters state their evidence and make their prediction in typical scientific fashion, which the consumer can accept or not.

The problem with evidence, however, is that it takes more than evidence to support predictions. Any inference, including a prediction, requires assumptions. Assumptions are beliefs about the world and the people in it. They are not nearly as solid as evidence. Nevertheless, historians and scientists use assumptions all the time, and they are usually pretty good ones because they are hard to critique or challenge. One can almost certainly assume that a person writing a diary is trying to record their experience as best they can. They are not trying to fool future generations! In the same way, one can assume that a careful scientist is going to calibrate her instruments before taking measurements. People make mistakes, and a few even fudge the data, but the assumption that the instruments measure what they are intended to measure is hard to challenge.

Social science uses the same process to support its inference, but the assumptions required to make inferences in social science and particularly about the future are quite different. Assuming that a trend will continue, that a goal will be achieved, or that an influential stakeholder is telling the truth can be challenged, often quite successfully. Despite that fact, forecasters blithely use those assumptions anyway; but in doing so, they hide the uncertainties inherent in the forecast. “State your assumptions”—another truism. But stating them does not make them true. The world does not know that we have made an assumption. It is going to do what it is going to do, regardless of the assumptions we make. Stating them without challenging them only confers a false sense of certainty on the forecast.

The RAND Corporation has developed and published an extensive method for identifying and challenging assumptions called Assumption-Based Planning (DeWar 2002). While RAND’s technique focuses on planning assumptions, their procedures work equally well for the assumptions involved in scenarios.

So predictions are still preferred because they look modern and scientific, they use logic, they are based on evidence, and are often supported with mathematics and computers. Is it possible to develop scenarios with those same characteristics? Even asking the question might seem odd or even traitorous to some. Foresight is not modern; it is postmodern. It is not scientific or logical; it is imaginative and even speculative. And, most of all, scenarios are not based on evidence. They are the products of insight and imagination. Scenarios do not use evidence; they are self-evidently plausible. Really?

But would not an evidence-based approach to scenario development provide more credibility in this scientifically dominated culture? Many futurists do not care what the modernists think. They prefer to be different, outside the mainstream, the perennial “other.” Society needs “others,” to be sure, but it also needs a credible field that promotes holistic, creative, contingent, alternative thinking about the future in a way that appeals to the majority. If futurists want to change the way people think about the future, they need to consider an approach that appeals to those same people.

An evidence-based approach to scenario development would also satisfy the perennial calls for a more transparent approach to foresight (Van der Steen and Van der Duin 2012). As it stands, anyone’s Global Business Network (GBN) matrix ¹ is just about as good as anyone else’s. Anyone’s fixed scenario archetypes (e.g., best case, worst case, muddling through, transformation) are about as useful as the next one. The evaluation of the product is often purely subjective—how the client feels about the outcome rather than the methodological rigor of the process (Piirainen et al. 2012). Client satisfaction is clearly one of the most important outcomes of good foresight work, but is it the only one? How are one’s peers to judge the quality of a product without the methodological transparency required to do the evaluation?

Professional historians and scientists rest their case on evidence more than on the utility of their conclusions. Decision-makers do not understand the science behind scientific conclusions, but they believe that the scientist’s peers do. And if the support for the conclusions is insufficient, the peers will tell. Can futurists do the same? The answer is yes, but they must use a transparent process based on evidence and assumptions the way others can understand, discuss, and thereby critique the process, not just the outcome. Clients might even be more willing to engage in a substantive discussion about provocative scenarios if they believed that the foundation for the scenarios is more rigorous and transparent. Without that, the decades-long effort to increase self-criticism and reflection in the field and thereby the credibility of the field as a whole will go nowhere.

At the same time, the relative lack of rigor and support for scenarios is only one of the many reasons that clients do not like to think of the future in terms of scenarios. We are a certainty-seeking species. We prefer simple answers to simple questions and simple solutions to simple problems. That is even what we are taught in school. Giving the right answer earns an A even when there are multiple legitimate answers available.

Other reasons that scenarios have a hard time are that they might recommend a change, but people usually like the familiar over the novel. Scenarios are hypothetical what-ifs. Why spend time and money on issues that might never occur when we have so many issues that we know are urgent right now? Finally, one version of the future might support one person’s or group’s interests in maintaining or increasing their economic or social position. Indeed, we see today how easy it is for some to ignore even well-supported scientific conclusions about the future.

So Baseline Analysis is far from a complete solution to scenarios being the preferred way of describing the future. However, it might help those who are looking for a more rigorous approach to their justification. If anything, foresight professionals at least should embrace this approach as a way to increase the legitimacy and credibility of the field.

The Expected Future

The approach outlined here, titled Baseline Analysis, begins with a region of the future that few futurists pay attention to—Roy Amara’s (1981) “probable future.” The probable future is usually not probable in an absolute sense—that is, it is not more than 50 percent likely. Herman Kahn is supposed to have said, “The likely future isn’t.” With so many alternatives, it is hard to imagine any interesting future that is more than 50 percent probable. But the alliteration of “probable, possible and preferable” made Amara’s categories memorable.

A better term for the probable future is the expected future, that future that will emerge if nothing surprising happens—Kahn’s surprise-free future (Kahn and Bruce-Biggs 1972; Micic 2010). Others call it the official future (Randall and Ertell 2005) or the most likely future or the best estimate—that is, it is more likely than any other future. The expected future is what traditional forecasters use as their predictions. As a result, most futurists do not even refer to the expected future because they want to be different. They do not want their clients or audiences to be so taken by the expected future that they reject the alternative futures.

But that is a risk that futurists must take because the expected future is already in everyone’s image of the future so it is better to deal with it head-on. And starting with the expected future also has some advantage. It is a scenario after all, and the most likely one at that. Clients and audiences are familiar with it so it is good to start there as they explore the other futures available. Begin with the more familiar and move to the less familiar.

Support for the Expected Future

The expected future is also called the Baseline Future in this approach because it is the foundation for scenario development. A baseline is a starting point, “a usually initial set of critical observations or data used for comparison or a control.” ² It is where traditional forecasters end up, but it is where futurists should begin. The Baseline Future is an inference, “a conclusion or opinion that is formed because of known facts or evidence.” ³ An inference requires support in the form of evidence before one accepts it. Almost all statements in history and science are inferences because they cannot be directly observed. They are supported by facts or evidence just like the expected future is.

Stephen Toulmin (1958) described this process most clearly in The Uses of Argument. The basics of the process include the Claim (in this case, the prediction), the Ground (Fact, Evidence, Data that support the Claim), and the Warrant (the assumption required to use the Ground to support the Claim; Figure 1).

OPEN IN VIEWER

Figure 1.

The Toulmin model of argument.

But the support for inferences is open to a form of critical thinking called rhetorical analysis. Rhetoric has received a bad reputation in recent times (as in the phrase “just rhetoric”). Traditionally, however, rhetoric was the art of discourse and the skill of persuasion, which was included in the core curriculum of every school from ancient Greece to the nineteenth century. Its classical meaning was the analysis of argumentation and persuasion, the art and science of providing support for one’s claims. Rhetoric is still studied in schools of communication, but hardly anywhere else.

That is too bad because rhetoric is also the basis for one version of critical thinking, a skill held dear by most teachers. Critical thinking has taken on a whole host of meanings as well, including creativity, problem solving, out-of-box thinking, and so on. In its pure form, it is thinking critically rather than accepting the stated conclusion or interpretation. It is not criticizing or finding fault. It is more like quality assurance (QA), testing the validity of the conclusion by looking beneath the obvious to the underlying reasons and rationale. Software developers turn their software over to quality professionals who evaluate whether the software does what it is supposed to do without error. The QA professionals want the software to work, but their job is to test it to be sure. So in rhetorical analysis, critical thinking is evaluating the quality of the support for inferences.

Figure 1 paints a picture of that support where knowledge is divided into two regions—observable and unobservable. Unobservable knowledge includes inferences, knowledge that requires some degree of support to be accepted because it is not directly observable, such as forecasts. Observable knowledge includes evidence, knowledge that is observable or at least not challenged as untrue. Evidence is used to support inferences. ⁴

Evidence supports inferences, but only if one accepts one or more assumptions, called warrants, required to use the evidence in support of the inference. Just like a search warrant or an arrest warrant, which allows police to take actions that they would normally not be allowed to do, the warrant in support of an inference gives one permission to use the evidence in support of the inference. Even evidence that is incontrovertibly true cannot be used to support an inference unless one accepts the assumption(s) required. DeWar (2002) makes the same point with respect to plans. One cannot accept the claim that a plan will work unless one accepts the assumptions required by the plan.

For example, political scientists make an assumption when they use the data from a sample of likely voters to forecast the outcome of an election even though the sample is only a tiny fraction of the voting population. They are allowed to make that inference if we assume that the sample is representative of the population—that is, the sample is like the whole population in every relevant way and that the people interviewed will vote and that they will vote in the way they said they would. Notice that the pollster cannot measure how representative the sample is or whether people are telling the truth. These are assumptions, reasonable ones perhaps, but assumptions nonetheless.

So every piece of evidence in support of an inference requires at least one assumption to allow that piece of evidence to be used to support the inference (Figure 2). Assumptions are required because no piece of evidence directly and univocally supports an inference. Data do not interpret themselves. Humans interpret data when they use them to support inferences, and the interpretation requires assumptions.

OPEN IN VIEWER

Figure 2.

Inference model.

Support for Alternative Futures

Historians, scientists, and futurists all make inferences that they support with evidence. Historians use evidence to make inferences about the past—artifacts, writing, pictures, and other items that previous generations have left behind. Scientists use observations, measurements, and so on to support their inference. Foresight professionals also make inferences, such as statements about an unobservable future, and they should support those inferences with evidence.

The difference between history, science, and futures studies, however, is not in the use of evidence. Futurists have evidence, such as empirical trends, stated plans, and general expectations for the future. The difference is the quality of the assumptions. History and science use strong assumptions (warrants). There is usually little doubt they are true. Not so in social science and, particularly, in futures studies. The result is that the futurist’s inferences are correspondingly less well supported. The weaker support for their statements about the future is not because they are doing anything wrong. It is the phenomena they are dealing with. Their assumptions are inherently easier to challenge. Futurists manage this greater uncertainty by proposing not one, but many futures in the form of scenarios. If traditional forecasters cannot support their predictions with unchallengeable assumptions, then something else could plausibly happen instead. Those “something else’s” are the scenarios of strategic foresight.

All of this maps directly onto scenario development. The original inference is the expected future. The expected future is supported by evidence—trends, plans, and so on. Each of those pieces of evidence requires one or more assumptions to use it to support the forecast. Each assumption has an alternative, which might plausibly be true instead of the original assumption. Every plausible alternative assumption weakens the forecast of the expected future. Something else might plausibly happen instead, and that something else is an alternative future or a scenario. This logic grounds the justification for scenarios on the critical analysis of the support for the expected future.

As with Toulmin, Assumption-Based Planning’s application to scenarios rests on its first few steps (Figure 3). Every plan requires that certain assumptions turn out to be true for the plan to be successful. DeWar narrows the field of all assumptions to those that are important (those that would harm the plan if they turned out to be false) and vulnerable (those that could plausibly turn out to be false). An assumption’s vulnerability rests on two factors—plausible events that could make the assumption false and signposts, data that suggest that the assumption is, in fact, turning out to be false. Those are both captured in foresight work as weak signals, events, or new information that could become strong signals that would alter the expected future if they were to grow.

OPEN IN VIEWER

Figure 3.

Assumption-Based Planning steps and logical dependence.

The key to the analysis lies in the concept of plausibility or what DeWar calls vulnerability. An alternative assumption and the alternative future that results is plausible when there is reason to believe that it might be (come) true. It is more than just theoretically possible. Any future is possible, even those that violate the physical laws that we believe today because those laws may change in the future. Only a relatively small subset, however, is plausible. The distinction is the one that American lawyers make in their instructions to a jury in criminal trials. They distinguish all doubt from reasonable doubt. The accused might be innocent. That is always possible. But to acquit, the jury has to have a reason to believe he or she did not commit the crime. If the jury has reasonable doubt (i.e., they have an explicit reason to believe that the accused might be innocent), they are required by law to acquit, but not otherwise. They may not even believe that he is innocent, but they must acquit if there is a reason to believe he is innocent.

In the same way, critically analyzing the support for the expected future leads to one or more plausible alternative assumptions, which lead to one or more plausible alternative futures. The reasons or evidence for the alternative assumptions also act as reasons and foundation for the corresponding alternative future. So the best alternative futures are not just possible. They are plausible because there are reasons (evidence) to suggest that they might happen. Those reasons and evidence are not definitive. The alternative is still an alternative. It is, by definition, less probable than the original assumption and, hence, expected future, but more probable than mere possibility for which there is no reason or evidence. DeWar (2002, 71–89) contains an extensive discussion of techniques for assessing the plausibility (vulnerability) that assumptions could turn out to be false.

Nuclear war is possible; pandemics are possible; meteors are possible. Does the futurist have a plausible foundation for that scenario, a causal chain based on some evidence, a plausible story about how that future could arise at this moment in history? If so, then other futurists can weigh the evidence for the expected future, evaluate the assumptions proposed, examine the reasons for their plausible alternatives, and thereby judge whether the scenarios are plausible or not. The whole process is out in the open. No magic here. It does require a healthy dose of human judgment, but that judgment is bounded by the evidence and the logical analysis of the assumptions, all of which are open for discussion, critique, and improvement by other professionals. That lays out the path, not only to a credible foundation for scenarios but also to the kind of scrutiny that other professions require as a matter of course.

So, for example, one might use a trend to support the expected future. The required assumption, of course, is that the trend will continue until the time horizon; the alternative is that it will not, that it will stop or reverse. That is always possible, but do we have any evidence to suggest that it is plausible, that it could actually stop or reverse? The same can be said about the plans of influential stakeholders. Will the stakeholder be able to successfully execute the plan and achieve the goal? Always possible that it will not, but is there any evidence to believe that it will not. And so on through all the assumptions required by the expected future.

And here is where environmental scanning and weak signals come in. A weak signal is a fact, a recent event, or a new piece of information that could signal the approach of more substantial change to come. It is weak because the big change has not occurred yet, but weak signals are crucial because they act as the early warning signs of a plausible future—something different from the expected future. Weak signals then can be the evidence that an alternative assumption might actually be true and that an alternative future might actually occur. They distinguish the plausible scenario from the merely possible. They are the evidence for alternative futures in evidence-based scenario development.

DeWar (2002) distinguishes two types of weak signals, one during the planning process and one during the implementation of the plan. He encourages planners to imagine possible events that could alter an assumption. Baseline Analysis, however, goes beyond purely imagining possible events, but it also requires that there be some reason to believe that event could happen—that it is plausible, not just possible. The second set of weak signals occurs during the implementation of the plan, what some call monitoring, which is paying attention to a predetermined set of indicators that signal whether an assumption is still valid and that the plan is progressing as it should. The more that weak signals challenge the validity of an assumption, the more vulnerable the original assumption and the more plausible the alternative. Plausible alternative assumptions and the weak signals that support them are then the empirical basis for alternative scenarios.

The question, of course, is whether one or more detectable weak signals precede every alternative future. If not, if the Black Swan phenomenon actually occurs in which an alternative future gives no warning at all before it occurs, then no evidence-based process can support such a scenario. Pure surprises do occur; they are wildcards. But how often does that occur versus how often do we say, “We should have known. We should have seen that coming?” The collapse of the oil price in 1981 and again in 1986, the fall of the Berlin Wall in 1989, the attacks on 9/11 in 2001, the collapse of the banking system in 2007, all were known and expected by a small group even though the population at large was surprised. Foresight professionals should be among that group that finds the weak signals and uses them to support alternative scenarios.

And a final word about the use of Baseline Analysis in actual foresight work before moving on to the process itself. Baseline Analysis is a way to support the plausibility of scenarios more than a way to discover the scenarios in the first place. Scientists present their findings in a well-known format, roughly called the scientific method—problem statement, literature review, hypothesis, methodology, data gathering, results, and conclusion. Doctoral dissertations follow this format as do most articles in scholarly journals. But scientists do not actually work that way. They have a hunch, they try a few things, some work, some do not, they refine the hunch, and so on. It is only after they believe that they have something to report do they bring out the canonical format to present it to their peers.

In the same way, futurists can use any technique they wish in developing their scenarios. Bishop et al. (2007) enumerated twenty-two of them. In fact, any forecasting technique can be used to develop scenarios because every technique requires assumptions that may be plausibly challenged, resulting in alternative futures. Baseline Analysis then is a format for communicating and justifying the plausibility of scenarios using reasons and evidence, not a method for developing them in the first place. The value of the format, however, is that other professionals can judge the plausibility of the scenarios based on the evidence presented.

Process

The process proposed here is quite simple:

A Toy Example

Here is a simple example on the possibility of military conflict between the United States and the People’s Republic of China (PRC). ⁵

There is some concern that the United States and the PRC might engage in military conflict sometime in the next twenty years. One can take that as a baseline future. Notice that it does not have to be a probable future (i.e., more than 50% likely) to act as the baseline future in this analysis. Some believe that this conflict is almost inevitable in the future; others believe it only a remote possibility. No matter. It is a prediction that requires evidence.

Some of the evidence for this prediction is,

The first step in critically analyzing this argument is to challenge the evidence. Even though reasonable people might disagree with one or more of these pieces of evidence, let us accept them now for the sake of this demonstration.

The next step is to identify the assumptions required to use the evidence to support the inference. Table 1 contains some of those assumptions, their alternatives, and reasons that the alternatives might be true.

OPEN IN VIEWER

Table 1.

Example of Baseline Analysis.

Evidence	Assumption	Alternative assumption	Reasons for the alternative
1. Historical wars among major powers	The present is like the past.	The present is not like the past.	Economies are more integrated now than in the past.
2. U.S. conflict with many world powers	The United States will engage any peer competitor in military conflict.	The United States only engages peer competitors when it believes it is physically threatened.	PRC seeks hegemony in East Asia, but so far has not projected its military power elsewhere.
3. Recent buildup of PRC military	PRC believes that it has strength to challenge the U.S. military sometime in next twenty years.	PRC does not believe that.	China has many problems today—food, water, energy, environment. Those problems could take resources away from continued military buildup.
4. PRC’s stated intention to reintegrate Taiwan	Integration is seen as the best or the only way to benefit from Taiwan.	Strong trade relations might seem better than integration.	Wars destroyed the economics of Tokyo, Berlin, Rome, and Iraq in wars over the last twenty years. The cost of a military action would destroy the economic value that China currently gains from Taiwan.

Note. PRC = People’s Republic of China.

The following scenarios result from these plausible alternative assumptions:

While one could arrive at these simple scenarios directly, Baseline Analysis reveals the actual support for the baseline and the foundation for the alternative scenarios. The scenarios are not just possible; there are reasons to believe that they could plausibly come true. Others can then debate the plausibility of those reasons and the overall quality of support for the baseline, subjecting the conclusions to the kind of scrutiny that a professional product requires. The evidentiary support for the baseline and for other scenarios also provides more credibility for the process in the eyes of clients and audiences.

Conclusion

The premise of this article is that the practice of foresight needs more legitimacy in modern society if it is to succeed. While the mission remains the same—to establish scenario thinking as the de facto approach in all long-term forecasting (and to many short-term ones as well)—the means that futurists have used have not gained the credibility they deserve. Scenarios are the product of imagination, and futurists are right to insist that the imagination is the only way to envision a truly novel world in the future. In fact, all great endeavors in history, science, business, and politics have made use of a healthy dose of imagination, despite the fact that the training for those fields emphasizes its algorithmic and logical techniques. But developing scenarios is not the same as justifying them. The process of discovery in science is messy, even chaotic, and often serendipitous, but it does not look that way when the discovery is written up in a scientific journal. Futurists need to make that same distinction between discovery and justification.

Foresight techniques help people expand their horizons and see more futures than they otherwise would. But seeing those futures is not the same as justifying which ones are plausible. Justification requires support for one’s judgment of plausibility. Baseline Analysis supports the plausibility of scenario statements indirectly by seeing them as alternatives to the Baseline itself. That support requires identifying the baseline, something many futurists are reluctant to do, critically analyzing its support, and generating alternatives when that support is less than ironclad.