Abstract
This brief note contains some doubts about the predominant kind of statistical election forecasting that is discussed in this issue. It is not meant to be a full-scale critique of the approach, the methods and the models that have been reported in the literature. Rather, it is intended to be an attempt to explicate some of the recurring feelings of disenchantment that can be experienced every time we come across these forecasts.
It is, of course, worth emphasising that we should be positively inclined to any attempt to specify statistical models of election outcomes, and to derive forecasts from such models. Forecasting is a legitimate scientific enterprise that does not deserve to be looked upon as a mere parlour game. In his excellent review of the field, Michael Lewis-Beck (this issue) states that forecasting contributes to our understanding of what causes election outcomes. But, this is exactly the cause of so much disappointment. Forecasting is indeed a potentially powerful way to develop and evaluate substantive theories, yet we usually learn so little (if anything at all) from actual forecasting exercises. To elaborate this, this note briefly discusses the theoretical core of statistical forecasting models, and argues that its theoretical foundations are unsatisfactory. It then discusses the implausibility of the functional specification of the core specifications of forecasting models. It then concludes with some comments on the theoretical scope of the forecasting tradition. It focuses particularly on Lewis-Beck's overview of principles and practice, elsewhere in this issue.
The Theoretical Core of Forecasting Models
Lewis-Beck pictures forecasting as a ‘theory-driven process’. More specifically, he states ‘what counts is that the variables in the equation are based on strong electoral theory’. He describes the Abramowitz model as built on ‘strong theory’ because it closely follows the core specification that Lewis-Beck earlier formulated in his equation 2. This equation states that the vote share of the incumbent is a linear function of the incumbent's (lagged) popularity and of (lagged) economic conditions. Is this a ‘strong theory’? It is doubtful, for a number of reasons.
First, theories of election results cannot escape the fact that an election outcome is an aggregate of individual choices. The fact that the specific aggregation rule is not an individual, but an institutional characteristic is irrelevant in this respect. Election outcomes are aggregates of individual behaviours. In order not to reify electoral outcomes, theories explaining them must therefore be ‘in sync’ with theories of the individual choices that underlie the outcome (cf. Lazarsfeld and Menzel 1969 [1961]). This does not require them to be identical. Indeed, to the extent that factors explaining individual choice have no overall effect on the predicted proportion of individuals voting for the incumbent, they may be omitted from the aggregate-level model at no cost. From this perspective it is not surprising that aggregate level models can be frugal, leaving out many independent variables that could not be omitted when explaining individual choice. But, are we really to believe that all other explanatory variables in the accumulated literature on individual electoral choice amount to no more than random noise when investigating aggregates? Maybe we should believe so, but not merely on the basis of their absence in forecasting models. It is the responsibility of the proponents of forecasting models to substantiate this implicit assumption, and so far they have made little effort in this respect. Were they to try this, however, they would not have an easy task. In all kinds of other analyses of aggregate-level phenomena—such as intended vote choices in weekly or monthly polls—it is commonly found that sundry other factors cannot be omitted after taking into account the popularity of the incumbent and the state of the economy.
A second reason for discounting the claim that the core specification of forecasting models represents ‘strong theory’ is the nature of the two independent variables that are always present in one form or another. Incumbent popularity—or its functional equivalents, such as government approval—is more a container that encapsulates all kinds of other phenomena than a theoretically explicated independent variable in its own right. That does not diminish its contribution to forecasting, but it does not represent ‘strong theory’ in any other sense of the word. The other ubiquitous independent variable in statistical forecasting models—the state of the economy in one form or another—does not fare much better. Certainly, a lively field exists of economic voting studies that all purport to demonstrate the relevance of economic conditions on individual vote choice, and thus their potential importance in forecasting aggregate-level outcomes. But, this field itself is far from unproblematic. Two edited volumes have recently appeared which provide an overview of the state of the art in studies of economic voting. The first was a special issue of Electoral Studies, edited by Lewis-Beck and Paldam (2000), and the other was a book edited by Dorussen and Taylor (2002). In both volumes the editors mention as one of the biggest puzzles in the economic voting literature the ‘instability’ of results across countries and over time (Lewis-Beck and Paldam 2000, 114; Dorussen and Palmer 2002, 1). In view of these instabilities Lewis-Beck and Paldam (2000, 114) exclaim: ‘We all prefer to think that the instability is apparent only. That is, it is due to something we are missing or doing wrongly—if we could just find the “trick”, everything would be well’ (original emphasis). Stated differently, the plausible hunch that ‘the economy matters’ does not yet constitute by itself ‘strong theory’ when investigating individual vote choice, and thus—in view of the aggregate nature of election outcomes—neither does it do so in forecasting models.
The Functional Specification of Forecasting Models
The ‘core specification’ of forecasting models is of a straightforward linear kind. I will not discuss the ‘incumbent popularity’ component in this respect because of its slightly tautological and theoretically unexplicated character. Neither will I rehearse the implication of my critique that important factors may unjustifiably have been omitted. Instead, I will focus exclusively on the economic component of the specification and its functional relationship with the dependent variable: incumbent vote share. The linear specification of the model implies that the vote share of the incumbent is boosted by good economic conditions, and diminished by bad ones. And, the linear nature of this relationship implies symmetry: the positive effect of 1 per cent economic growth is of the same magnitude as the negative effect of 1 per cent negative growth. This specification mirrors that in most individual-level models of economic voting which, as we saw above, are riddled by anomalies and instabilities.
There are good reasons to reject any specification that imposes such symmetrical effects. To explain this, we must theorise a little about the structure of the electoral competition between parties or candidates. It is probably not too far-fetched to propose that the actual field of electoral competition in established democracies does not encompass the entire electorate. Some people are set in their ways. They will only support one of the parties or candidates and it is virtually inconceivable that they will vote for another one. The group of voters that will vote for the incumbent party no matter what can be thought of as that party's minimum vote share. Other voters are less stuck in a rut. They do not exclude voting for the incumbent, but they may also vote for other parties. A third group will never vote for the incumbent, no matter what. Only the second group is subject to electoral competition; the first and the third are not. How large these respective segments are is a matter of empirical assessment, and does not concern us here. As a consequence, the share of the vote the incumbent can conceivably obtain is constrained by a maximum that is smaller than the size of the entire electorate. This maximum consists of the group that will not consider any other option than the incumbent, and the group where the incumbent has to compete with other parties. Let us refer to this maximum as the potential vote share of the incumbent. In situations where, for whatever reason, incumbents are so popular that they approach this potential vote share, any positive developments of the economy cannot benefit them anymore, but negative economic developments may very well hurt them badly. Conversely, incumbents (or, for that matter, any party or candidate) who are close to their minimum vote share, will not lose any support from poor economic conditions—after all, they are already at rock bottom—while they stand to make substantial gains from good economic times. In other words, it is very unlikely that economic conditions will have a linear and symmetrical effect, and any model that imposes such functional form cannot be thought to represent ‘strong theory’, and has to be viewed with suspicion.
Such a critique on the imposed symmetrical effect of economic conditions does not only pertain to aggregate-level forecasting models, but equally to individual-level models of (economic) voting. Many of the anomalies and instabilities found there can indeed be attributed to this, as has been demonstrated in an extensive analysis of economic voting in the countries of the EU in the period 1989–1999 (van der Brug, van der Eijk and Franklin (forthcoming)).
In order to integrate the implications of this line of argument in forecasting models, it is necessary to distinguish segments of the electorate in terms of the potential electorates of the various parties (cf. van der Eijk and Oppenhuis 1991). And that, in turn, requires an empirical approach that explicitly distinguishes between vote choices on the one hand, and the electoral attractiveness (utility) of each of the parties or candidates on offer, a necessity that has been recognised in formal approaches to voting and elections since Downs (1957). This can be achieved in various ways. The recent popularity of discrete-choice models (such as multinomial logit, conditional logit and so on) can to some extent be understood from the fact that they are based on this distinction. For reasons that do not have to be elaborated here, however, the discrete-choice approach has serious drawbacks that can be avoided by direct empirical measurement of electoral utilities (van der Eijk et al. (forthcoming)).
The Scope of Forecasting Models
It is little wonder that forecasting models originate in the context of US presidential elections. Not only are these of obvious political importance, but they are also simple. The choice is usually between just two candidates. Most analysts experienced great difficulties in the 1992 three-candidate race, but the occurrence of semi-successful third candidates is sufficiently rare to avoid American scholars spending much thought on the development of more general models. Moreover, US presidential elections are exclusively about the allocation of executive power. But, these conditions do not apply in most of the rest of the world. In the UK, a general election is about more than just the allocation of executive power, as supporters of Liberal Democrats, nationalist parties and various smaller parties would easily admit—and as it would behove academic analysts to do as well. Modelling elections as merely a referendum on the incumbent is myopic, not only in academic terms, but in political ones as well.
In yet other countries—in fact in the majority of representative democracies—the link between election outcomes and the allocation of government power is an indirect one, as processes of coalition formation intervene. In those cases it would be absolutely ludicrous to model elections as referenda on the incumbent.
The overall sense of disappointment with forecasting comes from a doubt that its proponents have taught us anything that helps in developing and evaluating theories about elections and election outcomes. They almost universally fail to justify their self-imposed limitation of independent variables to popularity/approval and economic conditions. They include implausible model specifications that wilfully neglect the implications of almost half a century of theorising and research on electoral competition, and they argue—unreasonably—that elections are no more than referenda on incumbents. In this context it is difficult to see forecasts as more than parlour games. They can (and should) be more than that. But, they have yet to live up to their potential.