Abstract
This article contributes to the literature on comparative methods in the social sciences by assessing the strengths and weaknesses of regression analysis and fuzzy-set qualitative comparative analysis (fsQCA) for studies with a moderately large-n (between approximately 50 and 100). Moderately large-n studies are interesting in this respect since they allow for regression analysis as well as fsQCA analysis. These two approaches have a different epistemological foundation and thereby answer different, yet related, research questions. To illustrate the comparison of fsQCA and regression analysis empirically, I use a recent data set (n = 53) that includes data on the conditions under which governments in Western democracies increase their spending on active labor market policies (ALMPs). This comparison demonstrates that while each approach has merits and demerits, fsQCA leads to a fuller understanding of the conditions under which the outcome occurs.
Introduction
In his 1970 article, the eminent comparativist Giovanni Sartori discussed what he saw as the dismal state of political science at the time, with the profession oscillating between two unsound extremes: unconscious thinking—which an overwhelming majority does—and overconscious thinking—done by a small minority. 1 Against this backdrop, Sartori called upon scholars to acquire training in (elementary) logic “to steer a middle course between crude logical mishandling on the one hand, and logical perfectionism (and paralysis) on the other hand” (Sartori 1970:1033). This would help scholars to get out of the “sea of naïveté” they were swimming in (Sartori 1970:1033). Although to my knowledge Sartori has not taken part in the debate on configurational comparative methods—namely, qualitative comparative analysis in its original crisp-set variant (csQCA), its fuzzy-set variant (fsQCA), or its multi-value variant (mvQCA) (see Rihoux and Ragin 2009)—Sartori might well like what he sees when he would examine the development of these methods in the social sciences over the past decade. Researchers applying these techniques typically try to acknowledge their own “naïveté,” aim to limit this as much as possible, and strive to be a conscious thinker walking the logical middle path—although it remains an empirical question whether they also succeed in this regard.
In any case, the number of such “walkers” has risen exponentially in the last couple of years. 2 It was in the late 1980s when Charles Ragin brought Boolean algebra and set-theory to the social sciences with his innovative book The Comparative Method (Ragin 1987). Still, the real spur in attention started some years after Ragin’s next book on fuzzy-sets (Ragin 2000). By now, there are many scholars who use configurational comparative methods, many of whom are no direct students of Ragin.
Configurational comparative methods can formalize case-oriented analysis and thereby offer tools to improve comparative research. These methods are particularly apt for identifying the minimally necessary and/or minimally sufficient (combinations of) conditions that bring about an outcome (i.e., assess causes-of-effects). The goal of many QCA analyses is to account for a particular outcome, like welfare state development in Germany or Finland. Regression analyses, conversely, typically aim to explain the effects of particular causes (see Wagemann and Schneider 2010). In recent discussions on configurational comparative methods, scholars argue that these approaches are best applied next to another one (Ragin 2008; Schneider and Wagemann 2010; see Rihoux 2006). Scholars should not become “QCA monomaniac” (Ragin and Rihoux 2004:6).
This study adds to this discussion by assessing the strengths and weaknesses of a configurational approach, fsQCA, vis-à-vis ordinary least squares (OLS) regression analysis for moderately large-n studies (n between approximately 50 and 100). Moderately large-n studies are particularly insightful in this respect since both fsQCA analysis and regression analysis are plausible approaches for studies with such a number of cases. The n is high enough to estimate (simple) regression models. Still, the number of cases is not thus high that regression analysis seems the only—or perhaps most logical—option. 3 FsQCA analysis could be a good alternative and/or complementary approach when the number of cases is moderately large. Many topics are excellent candidates for a moderately large-n analysis. Examples include comparisons of the states of the United States, political parties in parliamentary democracies, protests in a substantial number of countries, and so on. Note that this is not to suggest that the number of cases is or should be the overriding concern when selecting an approach; the aspects that I discuss in the following, such as what is a researcher’s approach to explanation or concept of causality, are more important for making this decision. Still, oftentimes, the number of cases excludes the possibility of using an approach—in-depth case studies of, say, a 1,000 cases is no option, as is regression analysis of five cases. In moderately large-n analyses, there is no such exclusion beforehand based on the number of cases.
To examine the comparative advantages of regression analysis and fsQCA analysis for moderately large-n studies, I use a recent data set with a moderately large number of cases (n = 53) with which Vis (2011) examines the conditions under which governments in Western democracies increase their spending on active labor market policies. Instead of using these data to arrive at substantive conclusions, I employ them in this study to compare the two methodological approaches and reveal each one’s merits and demerits. This comparison will show that for studies with a moderately large-n, fsQCA has typically most to offer. However, the relatively large number of cases in a moderately large-n analysis also comes with the price that not all cases can be accounted for. The comparison furthermore suggests that combining a regression analysis and fsQCA in one study may help us to become even more conscious thinkers—without becoming overconscious ones.
The article has the following structure. The next section discusses some of the main similarities and differences between regression analysis and fsQCA. The following two sections form the empirical core of the study by presenting the regression analysis and the fsQCA analysis of governments’ increased spending on active labor market policies. The final section discusses the findings of the analyses and draws some conclusions.
Regression Analysis Versus fsQCA: Similarities and Differences
To what extent are regression analysis and fsQCA analysis—and configurational comparative methods more generally—similar or different? Recent work indicates that epistemologically as well as ontologically, traditional quantitative approaches like regression analysis differ from configurational comparative methods (e.g., Rihoux and Ragin 2009; Mahoney 2010; but see Rohwer 2010). In many respects, the basis of configurational approaches is what is typically called “qualitative” (see Mahoney 2010; Wagemann and Schneider 2010:) while that of regression analysis is “quantitative.” Two topics are illustrative in this respect: the approaches to explanation and concepts of causality.
Regarding approaches to explanation, qualitative scholars adopt a so-called causes-of-effects approach. The goal of this approach is to account for individual outcomes (effects), such as revolutions, to explain meaningfully the patterns in the cases under study (Wagemann and Schneider 2010). Quantitative approaches, conversely, typically follow an effects-of-causes approach, in which the goal is to estimate what is the average effect of one (or more) variables in a population of cases. FsQCA fits the causes-of-effects approach most because this approach aims to reveal the minimal (combinations of) conditions bringing about a particular outcome in specific cases. However, as we shall see in the following, the larger the number of cases under study in an fsQCA analysis, the more fsQCA moves in the direction of identifying the effects-of-(multiple)-conditions-of-causes rather than the causes-of-effects. Since such an fsQCA analysis still aims to understand how outcomes come about, namely, wants to account for the effect, the approach remains more “qualitative” than “quantitative.”
In terms of concept of causality, fsQCA also has the closest fit with qualitative research. Configurational approaches’ concept of causality is rooted in Boolean operations. These operations can be expressed either in set-theoretic terms (e.g., subset, superset, intersection) or in logical terms (e.g., necessary and sufficient conditions, conjunction), with set-theoretical and logical terms being equivalent terminologies (see Ragin 2008; Wagemann and Schneider 2010). When the fuzzy-set membership scores of all cases on the (combination of) condition(s) are equal to or smaller than the fuzzy-set membership scores on the outcome, this means that the condition is a fuzzy subset of the outcome. Such a set-relationship indicates that the condition is sufficient for the outcome. For a condition to be necessary for the outcome, the fuzzy-set membership scores on the outcome need to be a perfect subset of the fuzzy-set membership scores on the (combination of) condition(s). That is to say, for all cases, the fuzzy-set scores of the outcome are equal to or smaller than the fuzzy-set scores on the condition. 4
In line with its set-theoretical foundation, scholars applying configurational methods often adopt the so-called INUS approach to causation. An INUS cause is an “insufficient but non-redundant part of an unnecessary but sufficient condition” (Mackie [1974] 1980:62; italics in original). The INUS approach to causality is typically connected to qualitative, case-oriented research. However, there are also scholars positing that quantitative, population-oriented research allows for identifying INUS conditions. For example, Mahoney (2008) argues that partial causal effects in well-specified quantitative models suggest that these factors are important INUS conditions—namely, conditions that are “probabilistically sufficient” or “probabilistically necessary” (Mahoney 2008:426) on their own, namely, not in explicit combinations. Moreover, Braumoeller and Goertz (2000, 2003) develop a procedure to test for necessary conditions in statistical analyses. Their two-step procedure first assesses whether a condition is necessary or not and, if so, establishes whether the necessary condition is a nontrivial one. This procedure is more cumbersome than testing for necessary conditions directly using the fsQCA software, where the measure of coverage provides information of the (non)trivialness of the necessary condition(s). The requirements on the quality of the data when using the Braumoeller/Goertz approach are also (very) high. Nonetheless, I consider this approach a welcome addition to the repertoire of methodologies for the social sciences. This is a view that not all scholars share. Dul et al. (2010), 5 for example, hold that conclusions regarding necessary conditions can never be drawn from covariational analyses like regression analysis.
Another difference in terms of causality is that fsQCA is especially attuned to multiple conjunctural causation. Scholars, typically qualitatively oriented ones, speak of multiple conjunctural causation when at least one of the following situations arises. First, a combination of conditions produces the outcome. For example, governments increase spending on active labor market policies when they are of leftist composition and when there is no corporatist system in place. Second, the situation when there is more than one condition that generates the same outcome, an issue also known as equifinality. For instance, the combination of leftist partisanship and the absence of corporatism as well as the absence of leftist partisanship and decreasing unemployment bring about activation. Third, when depending on the context, an outcome results from the presence of a condition or from its absence. In the previous (fictitious) example, either the presence of leftist partisanship or its absence leads to higher expenditure on active labor market policies, depending on the context (the absence of corporatism or decreasing unemployment). Because of these three characteristics, configurational approaches relax a series of conventional quantitative assumptions: permanent causality, uniform causal effects, unit homogeneity, additivity, and causal symmetry (Berg-Schlosser et al. 2009).
Regression analysis can also accommodate multiple conjunctural causation, but doing so comes less natural than in fsQCA. With regard to the first type of such causation, a combination of causes producing an outcome, interaction-effects immediately come to mind. Interaction-effects are increasingly used in the social sciences and offer a means to assess the joint effect of usually two conditions. Since many theories pose conjunctural relationships, this is a welcome development. However, interpreting an interaction consisting of more than two variables is challenging—to say the least (Braumoeller 2004). Also other developments, such as Clark, Gilligan, and Golder’s (2006) statistical approach to test asymmetric hypotheses, namely, hypotheses stating necessary or sufficient causes, run into problems when the interactions are of higher order. On a more practical note, when the number of cases is moderately large, the degrees of freedom will typically be only high enough to include one interaction of two variables.
There have also been attempts in quantitative research to address the multiple paths to an outcome (equifinality). Specifically, Braumoeller (2003) proposes econometric techniques, Boolean probit and Boolean logit, to test theories that posit such paths. Braumoeller’s approach requires modeling the probabilities of the occurrence of each of the causal factors. These probabilities are then multiplied to get the overall probability that the event in question will occur (Braumoeller 2003:215). Although Braumoeller’s techniques are also welcome additions to the methodological repertoire, they suffer from two drawbacks. Both relate to the shape of the likelihood function that is maximized (Braumoeller 2003). First, the multidimensional analyses that Braumoeller’s approach requires are very complex, possibly leading to convoluted likelihood functions with multiple maxima. Second, the multidimensional analyses entail complex and quite demanding data requirements. If not enough information is available, the coefficient estimates become meaningless. It is therefore not surprising that Braumoeller (2003:210) states himself that “Ragin’s solution to the problem of dealing with causal complexity, qualitative comparative analysis, is of necessity more concrete” (italics added). Still, the advantages of Braumoeller’s econometric techniques are that they offer statistical modelers a way to test theories positing multiple paths and that they allow for modeling the relevant causal processes correctly—thus reporting findings that may well be missed by other techniques (Braumoeller 2003).
FsQCA and regression differ in how they tackle limited diversity, namely, the situation that not all possible combinations of conditions and outcome occur in reality (Ragin 2008; see also Simon 1952). In fsQCA, the researcher can use the so-called intermediate solution when theoretical guidance is available (Ragin 2008). The intermediate solution makes use of “easy counterfactuals,” namely, counterfactuals that are “easy” in that the researcher has a strong expectation about how a condition contributes to the outcome (e.g., existing work that shows that the presence of a condition—and not its absence—leads to a particular outcome). Also when such guidance is not available, fsQCA is still applicable. 6 In this case, the researcher has to decide whether to use the parsimonious solution or the complex one. The former employs all possible simplifying assumptions (statements about the logical remainders), whereas the latter uses none. Since the complex solution means that no statements are being made about the situations that did not occur empirically, this is the most conservative approach. When using the parsimonious solution, conversely, solutions typically involve difficult assumptions, which the researcher should evaluate.
In regression analysis, there are techniques to estimate unknown data and to deal with limited diversity. In their review article, Winship and Morgan (1999) discuss several such techniques. One is a statistical model that controls for underlying selection processes, with the goal to replicate experimental procedures for observational data. A key difference between how quantitative approaches like regression analysis and qualitative approaches like fsQCA deal with counterfactuals is that the former is inductively driven and focuses on counterfactual estimation while the latter employs theory-informed and knowledge-informed thought experiments. 7 Both fsQCA and regression analysis are thus able to address limited diversity to some extent, but the way they do so differs across the approaches. Table 1 sums up some of the differences between regression analysis and fsQCA.
Regression Analysis Versus Configurational Comparative Methods
Source: Inspired by Mahoney and Goertz (2006) and Kenworthy and Hicks (2008:7, Table 1.1).
If fsQCA and regression analysis differ on such core issues like approaches to explanation and concepts of causality, are they applicable in one study? A methodological purist, or skeptic for that matter, might argue they are not. As I discussed previously, the underlying epistemology of regression analysis and fsQCA differ, which is one of the reasons why they test different hypotheses. A regression analysis tests if an individual variable (or interaction of variables) has a positive or negative significant effect on the dependent variable, net of the other variables. A configurational approach, conversely, tests if a condition or combination of conditions is minimally necessary and/or sufficient for the outcome. However, a more pragmatic scholar might argue that these differences are a strength rather than a weakness since the different yet related hypotheses tested shed a distinct but hopefully complementary light on the research topic at hand. I take the latter, pragmatic approach.
Given their differences, it is unsurprising that the conclusions drawn from a regression analysis and a configurational comparative analysis often diverge, whereby they complement rather than invalidate one another. A sequential approach to using the two approaches may therefore be the most useful (cf. Rihoux et al. 2009). Sequentially conducting a regression analysis and a configurational analysis (crisp-set QCA) is exactly what Amenta and Poulsen (1996) do in their moderately large-n study into the conditions affecting the public social provision in 48 American states in the 1930s. For Amenta and Poulsen, the strength of csQCA over regression analysis is mainly theoretical: Their theory suggests that spending outcomes are the result of complex interactions that cannot be tested well with interaction terms in multiple regression because of too low degrees of freedom and/or possibly multicollinearity. The interaction terms in the multiple regression analysis offer some first support for the importance of these combinations but are indeed hindered by multicollinearity. The subsequent csQCA analysis provides strong evidence for Amenta and Poulsen’s theoretical argument. Still, given the centrality of complex causality in their theory and regression analysis’s problem to unveil this, one cannot help but wonder whether the authors needed the multiple regression analysis in the first place.
Another example of a moderately large-n study that conducts regression analysis in addition to csQCA is Ford, Duncan, and Ginter’s (2005) assessment of the relationship between health agencies’ adherence to the recommendations of a specific report (of the U.S. Institute of Medicine) and changes in their populations’ health. Different from Amenta and Poulsen (1996) and other studies employing regression analysis and configurational analysis, Ford and his colleagues test different hypotheses with regression analysis and with csQCA. Specifically, they first assess if assessment, assurance, and policy development (individually) are significantly and positively related to improvements in health by a regression analysis of 50 American states. Subsequently, they test if and which of these three functions are individually sufficient for explaining health improvement and, finally, if these three together were necessary for such improvements. With this three-step design, Ford et al. provide a good example of how to combine the strengths of regression analysis and the strengths of csQCA.
A key difference between this study and the work of Amenta and Poulsen (1996) and Ford and colleagues (2005) is that their research questions were not methodological. To the best of my knowledge, this study is the first to bring out the differences between regression analysis and (fs)QCA from a purely methodological perspective by applying the methods to concrete data.
Regression Analysis Versus fsQCA: Increased Spending on ALMPs
In this section, I present the regression analysis and the fsQCA analysis of governments’ increased spending on active labor market policies (ALMPs). 8 Walking through each analysis will help to reveal the comparative advantages or disadvantages of each approach for moderately large-n studies.
Data
For the comparison of regression analysis and the fsQCA analysis, this study uses a recent data set on the conditions under which governments increase spending on active labor market policies, such as job training and subsidized employment (Vis 2011). The data set contains data for 53 governments from 18 developed democracies between 1985 and 2003. 9 Hereby, these data add to the growing body of literature that examines the politics of active labor market policies. By now, the idea that benefits should be “active” is widely supported among developed democracies. On the individual level, most people also prefer active programs to passive ones (OECD 2006). Political parties from different sides of the political spectrum agree on the value of activation too. For example, in the party manifestos for the 2010 Dutch national parliamentary elections, parties as ideologically diverse as the social democrats, Christian democrats, and the conservative liberals underlined the importance of labor market integration and activation. Given the typical support for ALMPs, how does one account for the variation in spending on these policies across countries and—perhaps even more interestingly—across governments?
Table A1 in the online appendix (which can be found at http://smr.sagepub.com/supplemental/) displays the data on the dependent variable (in the regression analysis) and outcome (in the fsQCA analysis): the percentage point change per cabinet period in active spending per unemployed (cf. OECD 2003; Armingeon 2007). Active spending per unemployed is the share of gross domestic product (GDP) that is spent on ALMPs per 1 percent standardized unemployment, 10 or in formula:
Active spending per unemployed is a better measure for activation than the typically used active spending as a share of GDP because it is less sensitive to the state of the economy, like a recession. Specifically, a recession results—ceteris paribus—in higher spending on ALMPs as a share of GDP (GDP drops and thus ALMP/GDP increases, even when the government does not do anything). This is where the problem of the typical studies lies. However, by dividing ALMP/GDP by the level of standardized unemployment, one corrects—at least partly—for this problem. The larger number of unemployed in a recession means that the (higher) ALMP spending as a share of GDP needs to be divided among a larger number of unemployed. For activation to appear using this study’s measure, the government would thus need to boost ALMP expenditure substantially. The data set excludes governments displaying less than one percentage point change because although such a small change could result from a political decision, it is more likely that it results from measurement error. 11 The changes range from minus 23.5 percentage points under Carlsson 3 (Sweden, 1990-1991) to plus 29.0 percentage points under Carlsson 2 and 1 (1986-1990). The average change per cabinet (either plus or minus) is 4.5 percentage points. Interestingly, both increases and reductions are not limited to one period; both occur in the 1980s, 1990s, and early 2000s alike. Moreover, spending on ALMPs did not automatically rise after the OECD’s and EU’s recommendations in the mid-1990s, with some governments displaying even reductions in this period (e.g., Kohl 4 in Germany and Bolger 3 and Shipley 1 in New Zealand). This suggests that the cross-government variation is puzzling indeed.
The data set also contains five causal conditions or independent variables: the change in unemployment and economic growth during the cabinet period (both capturing the socioeconomic situation), the political color of the cabinet, the degree of corporatism, and the extent of trade openness. Theoretically, Vis (2011) proposes that the socioeconomic situation is (almost always) necessary for activation (Hypothesis 1). She argues that because of the high costs of ALMPs and their low electoral reward, if any, governments increase ALMP spending only under an improving socioeconomic situation. While such an improving socioeconomic situation is usually necessary for activation, it is by itself not sufficient. Leftist partisanship, corporatism, and openness are all expected to be INUS conditions for activation (Hypothesis 2). Moreover, Vis tests Bonoli’s (2008) hypothesis that the combination of leftist partisanship and openness spurs activation (Hypothesis 3). Similarly, Vis also assesses whether leftist partisanship combined with the absence of corporatism is sufficient for activation (Hypothesis 4) or if the combination of leftist partisanship and increasing unemployment is (Hypothesis 5) (cf. Rueda 2007).
As indicated earlier, there have been some attempts to test for necessary and/or sufficient conditions using statistical techniques (Braumoeller 2003; Braumoeller and Goertz 2000, 2003; Clark et al. 2006). However, in regression analysis, it is still much more common to test hypotheses of the following structure (see Table 2): Reductions in unemployment and/or increases in economic growth (i.e., an improving socioeconomic situation) increase(s) activation (Hypothesis 1); leftist partisanship, corporatism, and openness have no direct significant bearing on activation (Hypothesis 2); leftist partisanship conditioned on high openness increases activation (Hypothesis 3); leftist partisanship conditioned on the absence of corporatism increases activation (Hypothesis 4); and leftist partisanship conditioned on increasing unemployment increases activation (Hypothesis 5). The different approaches to explanation are apparent in the formulation of the hypotheses. While the fsQCA hypotheses test for necessary conditions, INUS conditions, and sufficient combinations of conditions, the regression hypotheses focus on the average effect of individual variables or interactions. To facilitate the comparability, the hypotheses referring to the same (combination of) conditions have the same label (see Table 2).
Hypotheses
Note: act is increased spending on active labor market policies; left is leftist partisanship; corp is corporatism; open is economic openness. For the fuzzy-set qualitative comparative analysis (fsQCA), a ~ indicates the absence of a condition. INUS = insufficient but nonredundant part of an unnecessary but sufficient condition.
Table 3 presents the measurement of the dependent variable and the five independent variables; I return to the fuzzy-set calibration column when discussing the fsQCA analysis.
Measurement and Fuzzy-set Calibration of the Outcome and Conditions
Regression Analysis
An important assumption in regression analysis is that observations are independent from one another. Because the unit of analysis in this study is a government, this assumption might be violated. This proves not to be the case. The Durbin-Watson test statistic for autocorrelation is 2.278, indicating that the errors of the 53 cabinets are uncorrelated and can therefore be treated as independent observations. 12 There is no multicollinearity between the five variables (VIF scores between 1.0 and 1.6). The full regression equation is:
To test the effect of the three interaction variables (gov_left*openness, gov_left*corporatism, and gov_left*unemployment), I run four models: a baseline model without the interaction variables and three regression analyses that include one interaction at the time. With 53 cases, there are not enough cases to warrant the inclusion of more than one interaction per model. Also, the more interactions per model, the higher multicollinearity becomes. Table 4 presents the results of the regression analyses.
Ordinary Least Squares (OLS) Regression of the Change in Spending on Active Labor Market Policies (ALMPs) per unemployed
Note: Standard errors between parentheses.
Figure A1 in the online appendix graphs this interaction effect, showing the conditional effect at the minimum score of the variable, one standard deviation below the mean, the mean, one standard deviation above the mean, and the maximum (cf. Brambor, Clark, and Golder 2006). This figure demonstrates that the interaction fails to reach significance.
Figure A2 in the online appendix shows that the conditional effect of the interaction does not reach significance (see note a).
Figure A3 in the online appendix reveals that the conditional effect of the interaction is not significant (see note a).
p < .01.
The results of the analyses are straightforward. In all four models, unemployment is the only factor reaching statistical significance in the expected, negative direction. The lower the level of unemployment, the more governments activate. Specifically, if unemployment increases by one unit (i.e., by 1 percent), active spending per unemployed falls around 1.6 units. This finding supports Vis’s (2011) hypothesis of the relevance of an improving socioeconomic situation (Hypothesis 1). Also as hypothesized (Hypothesis 2), leftist partisanship, corporatism, and openness have no significant influence on activation. Looking only at the signs of the variables, leftist partisanship and openness do have a positive effect on spending on ALMPs and corporatism a negative effect. None of the interactions are significant, neither when we look at the unconditional effects (presented in Table 3), nor when we examine the conditional effects (cf. Brambor, Clark, and Golder 2006; see online appendix Figures A1-A3). This means that we have to reject Hypothesis 3, Hypothesis 4, and Hypothesis 5 based on these findings. Summing up, the regression analyses find that the higher the change in the level of unemployment during a cabinet period, the lower the change in spending on ALMPs per unemployed. The other variables (economic growth, partisanship, corporatism, and openness) have no overall influence.
FsQCA Analysis
To what extent do the findings of the fsQCA analysis corroborate those of the regression analysis? Before we can answer this question, we first need to transform the raw data—as used in the regression analysis—into fuzzy-sets, the so-called calibration process (Ragin 2008; see Vis 2010). A fuzzy-set is “a fine-grained, [pseudo] continuous measure that has been carefully calibrated using substantive and theoretical knowledge relevant to set membership” (Ragin 2000:7). While it is still uncommon in the social sciences to use calibrated measures, the use of such measures is more common in fields such as chemistry, astronomy, and physics (Ragin 2008). Because of the practice of calibrating in fuzzy-set logic, this approach’s measurement practice fits both qualitative researchers’ interest in interpreting variation (i.e., identifying relevant and irrelevant variation) and quantitative researchers’ interest in precisely placing cases relative to one another (Ragin 2008). It allows for combining the best of both worlds. Still, scholars favoring an empiricist way of doing research—the majority of quantitative scholars—will probably disagree on this statement and would prefer using uncalibrated data instead.
For calibrating fuzzy-sets, the researcher establishes when a case is “fully in” a set (1), “fully out” of it (0), and when it is “neither in nor out” of the set (the so-called crossover point at .5) using external criteria, in particular theoretical and substantive knowledge (Ragin 2000, 2008). The third column in Table 3 sums up the fuzzy-set calibration for the outcome and the conditions. Let me briefly elaborate on the calibration (for more information, see Vis 2011). For the outcome, the change in active spending per unemployed, the qualitative breakpoints 0 and 1 are placed at −15 and +15, a decision based on substantive knowledge of the cases that derives from, among others, Huber and Stephens (2001), Clasen (2005), and the OECD Employment Outlooks (various years). Such a reduction (increase) implies a change of .15 percent of GDP per percent standardized unemployment. If the unemployment rate is 6 percent, the share of GDP spent on activation would then reduce (increase) by .9 percent during the cabinet period—a lot given that total social expenditure generally hardly exceeds 30 percent. The qualitative breakpoint .5 is placed at 0. To calibrate the in-between scores, I use the so-called direct method of calibration (Ragin 2008). This method is the most appropriate way to transform an interval-scale variable into a fuzzy-set. The direct method of calibration uses estimates of the log odds of full membership and is thus no linear transformation of the raw, interval data. The calibrate command in the fsQCA 2.5 software gives the resulting fuzzy-set.
Also the five causal conditions are calibrated into fuzzy-sets. The change in unemployment and economic growth during the cabinet period capture the socioeconomic situation a government faces. The first qualitative breakpoint 0 (fully out the set of growth or the set of unemployment) is put at minus 5. Substantive knowledge about developed democracies indicates that a reduction of economic growth or unemployment by 5 percent is both rare and has a substantial influence on the possibilities for and necessity of socioeconomic policymaking. For similar reasons, the second qualitative breakpoint 1 (fully in the set of growth or the set of unemployment) is placed at plus 5. The in-between scores are established based on the coding scheme in Table 3; Table A2 in the online appendix displays the resulting fuzzy-set scores.
The fuzzy-set for partisanship is based on leftist cabinet composition, calculated as social democratic and other leftist parties as a percentage of total cabinet posts or seats (Armingeon et al. 2008). The qualitative breakpoints 0 (fully out of the set of leftist partisanship) and 1 (fully in the set) are placed at 0 and 100, as these scores correspond with the hegemony of rightist parties and the hegemony of social democratic and other leftist parties in cabinet, respectively. The often used party families’ categorization by Budge and colleagues (2001) underlies the differentiation between leftist and rightist parties. The crossover point is placed at .50, which is where leftist and rightist parties hold the same percentage of cabinet posts and the cabinet. To differentiate between more left-wing and more right-wing cabinets, these cabinets are coded as either .55 (when the leftist party or parties receive most of the votes) or .45 (when the rightist party or parties receive most of the votes). Table 3 displays the procedure for the scores in between the qualitative breakpoints.
For the fuzzy-set corporatism, the first qualitative breakpoint 0 (fully out of the set) is placed at 1 on the Kenworthy (2001) index. Countries scoring 1 on this index have fragmented wage coordination, which is confined largely to individual firms or plants, and have no corporatist system. The second qualitative breakpoint 1 (fully in the set corporatism) is put at 5. Countries with a score of 5 on the index have centralized coordinated wage bargaining by peak confederation(s) or government imposition of a wage schedule/freeze, with a peace obligation, which is typically corporatist. The in-between scores are calibrated using the same procedure as for the outcome, with 3 on the Kenworthy index being the crossover value.
Finally, the fuzzy-set openness is calibrated as follows. The first qualitative breakpoint 0 (fully out of the set) is placed at 0 percent. An economy scoring 0 percent on openness is completely closed and has no import or export relations with other countries. The second qualitative breakpoint 1 (fully in the set) is placed at 100 percent, which means that a country’s trade relations with other countries are so extensive that they (more than) match that country’s GDP. The in-between scores are calculated using the same procedure as for corporatism and the outcome. 13
For the fsQCA analysis, I use the fsQCA 2.5 software. 14 In a first step, I test if the conditions and their absence are necessary for the outcome. The results, available on request, show that none of the conditions reach the—conservative—benchmark of .90 consistency but also demonstrate that the absence of unemployment and openness come close to being necessary (consistency of .84 and .88, respectively). In the second step, I conduct the sufficiency analysis. I use the so-called truth table algorithm, which transforms the fuzzy-set membership scores into a truth table. A truth table lists all logically possible combinations of causal conditions and each configuration’s empirical outcome (Ragin 2008). The algorithm uses the direct link between the truth table rows and the corners of the property space, namely, the multidimensional space including all logically possible combinations of causal conditions (configurations). This study’s property space has 25 (the five conditions) = 32 corners (the configurations). The truth table, available on request, reveals nine logical remainders, that is, configurations with no empirical observations. This finding means that the variation in the data is limited since not all possible configurations are observed empirically. However, the degree of limitedness is fairly low since over 70 percent of the configurations are observed empirically.
After having reviewed the truth table, we logically minimize the table using Boolean algebra to reveal the combinations of causal conditions that are minimally sufficient for producing the outcome (Ragin 2008). The researcher needs to decide what to do with the logical remainders (see previous). I employ the most complex solution, which is the most conservative, and report the results of the parsimonious solution and the intermediate one in a note.
In set-theoretical logic, logical AND (*) refers to the intersection of sets and logical OR (+) to the union of sets. Moreover, ~ indicates the absence of a condition. The fsQCA analysis finds that there are three routes, or causal recipes, toward increased spending on ALMPs (act): (1) decreasing unemployment combined with openness OR (2) decreasing unemployment combined with leftist government and the absence of corporatism OR (3) decreasing unemployment combined with the absence of leftist government and the presence of corporatism. In fuzzy-set notation, the analysis’s result is
Consistency indicates the degree to which cases sharing a given combinations of conditions agree in displaying the outcome (Ragin 2008). Coverage indicates the proportion of the sum of the membership scores in the outcome. This result thus in 89 percent suffices to bring about increased spending on ALMPs, covering 83 percent of the membership scores in the outcome. Table 5 presents the results in fuzzy-set notation, whereby the table also lists which cases correspond to which of the three causal paths. The visibility of cases signifies the qualitative, case-oriented character of fsQCA—even when the number of cases is moderately large.
Results From the Fuzzy-set Qualitative Comparative Analysis (fsQCA)
Note: Raw coverage indicates the proportion of the sum of membership scores covered by the particular path; unique coverage is the proportion of the membership scores covered by that path only; consistency indicates the degree to which cases sharing a given combination of conditions agree in displaying the outcome. The cases listed below each path are the ones with membership >.5 of that path.
In general, the fsQCA findings indicate that without an improving socioeconomic situation, in the form of a reduction in unemployment and/or improving economic growth, governments will not increase active spending. Consequently, an improving socioeconomic situation is almost always a necessary condition, or more precisely an essential ingredient in each of the causal combinations derived. This result is in line with Hypothesis 1 and the regression analysis findings. Decreasing unemployment is only sufficient for activation in combination with other conditions (the presence of openness or the absence of corporatism and leftist government or the presence of corporatism and the absence of leftist government). Depending on the path, partisanship does not matter (~unem*open), leftist partisanship does (~ unem*~corp*left), or the absence of leftist partisanship does (~unem*corp*~left). As expected (Hypothesis 2), leftist partisanship, corporatism, and openness are thus INUS conditions. Note that the finding that it is either the absence of corporatism combined with the presence of leftist partisanship (and decreasing unemployment) or the presence of corporatism combined with the absence of leftist partisanship (and decreasing unemployment) is particularly interesting because it indicates an “exclusive or” relation. This means that one of the other conditions must be present, but not both. Although it goes beyond the scope of this article to discuss this, such a result offers interesting opportunities for theory development and case-level analysis. 16
Like the OLS regression findings, the results of the fsQCA analysis fail to support Bonoli’s (2008) hypothesis that the combination of leftist partisanship and openness fosters activation (Hypothesis 3); this combination of conditions does not feature in any of the three paths. Different from the regression analysis, the results of the fsQCA analysis do offer partial support for Rueda’s (2007) expectation of a positive effect of leftist partisanship and the absence of corporatism (Hypothesis 4). Specifically, one of the paths displays this combination although another condition is needed to have a combination of conditions that is sufficient for bringing about activation. Finally, the fsQCA analysis contradicts Rueda’s other hypothesis of an effect of increasing unemployment and leftist partisanship (Hypothesis 5). Instead, the analysis shows that it is the reduction of unemployment that matters (combined with leftist partisanship and/or other conditions). Interestingly, this is the only point where we see a clear difference between the findings of the regression analysis and the fsQCA analysis. While the regression model including the interaction between leftist partisanship and unemployment revealed a positive, yet not significant, effect for unemployment, the fsQCA analysis finds that the reduction of unemployment is relevant.
Overall, the results of the two approaches prove more complementary than conflicting, with the absence of unemployment entering in each of the causal paths of the fsQCA analysis. This suggests that decreasing unemployment is an important condition that comes close to being necessary for activation, as the necessary condition analysis conducted earlier also suggests. The fsQCA analysis shows that the effect of corporatism could go either way, since both the presence and the absence of this condition enter in a (different) causal path. In a regression analysis, such a pattern would amount to a nonsignificant finding, as it also did. However, a low correlation between variables does not preclude the existence of relationships of necessity and/or sufficiency (see Mahoney 2004).
Comparative Advantages?
What can we conclude from the comparison of the regression analysis and the fsQCA analysis of the conditions under which government activate? What do the analyses teach us with respect to these approaches’ strengths and weaknesses for moderately large-n studies? 17 A first advantage of configurational methods for studies with a moderately large-n is the possibility of addressing multiple conjunctural causation straightforwardly. This is particularly important when it is likely, based on theory, that there are more ways than one to bring about the outcome or that the causal conditions combine in complex ways. Braumoeller’s (2003) Boolean logit and probit, conversely, likely run into problems when the n is only moderately large.
A second advantage of configurational comparative methods for studies of all kinds of n’s that also became apparent in this study is the possibility of identifying the combinations of multiple causes. In regression analysis, there is a limit to the number of interaction effects that can be included in one analysis, which, in a typical moderately large-n study, lies around one. In this article’s empirical example, the five hypotheses, including three with interactions, could not be tested simultaneously in an OLS regression analysis. Moreover, the interpretation on an interaction consisting of more than two variables is challenging—to say the least. In regression analysis, if an outcome (dependent variable) occurs and the given cause (independent variable) does not, it counts as negative evidence for the strength of that causal relationship (Epstein et al. 2008). This means that a factor that influences the outcome in only a subset of cases—but some cases nonetheless—becomes invisible in a regression analysis; in fact, it only inflates the variance and deflates the coefficients. Configurational comparative methods, contrarily, can identify the patterns that differ across subsets of cases easily and with less severe data requirements than the statistical advances. FsQCA can also reveal the situation where the presence of a condition (a high score) leads to the presence of the outcome, while its absence (a low score) produces the absence of the outcome.
An important strength of regression analysis over fsQCA for analyses of all kind of n’s is that it allows for assessing the average effect of a variable. This possibility is particularly relevant if the scholar’s theory emphasizes a particular factor and he or she want to estimate how large the net impact of this variable on the dependent variable is. With the recently developed coverage and consistency measures (Ragin 2008), configurational approaches have an option to assess the empirical relevance of results. However, these measures focus on the empirical relevance and set-theoretical importance of the separate paths to the outcome and the overall solution and cannot tell what an individual condition’s contribution is.
Another advantage of regression analysis over fsQCA, again holding for all kind of n’s, is that the former is less demanding regarding prior causal knowledge. Regression analysis has a clear empiricist foundation and does not require the calibration of data. The merit hereof is that regression analysis is less influenced by the researcher’s prior knowledge. This merit also comes with a demerit. It can be considered an advantage that fsQCA requires the calibration of data precisely because this means that prior knowledge can be included; the study consequently does not start out from the—often incorrect—assumption that nothing is known yet. In fact, by including prior knowledge, fsQCA resembles Bayesian’s use of prior probabilities.
Overall, the results of the fsQCA analysis were more detailed; fsQCA allows for more horizontal complexity than does regression analysis. Three causal paths emerging from the analysis, each consisting of at least two conditions, and the cases with membership to these paths were identifiable. Hereby, fsQCA allowed for accounting for the individual cases under study. Note also that the fsQCA analysis could not account for all 53 cases. For one, there were eight cases that did activate yet had no membership to one of the paths (Rocard 1 et al., Lubbers 3, Guterres 2, Felber, Bildt 1, Kok 2, Reagan 2, and Bolger 2). Inspection of these deviating cases suggests that there is no (single) factor that these cases share, such as period in office or type of government, that explains this variation. The cases also do not group together in any meaningful way; there is simply no pattern to detect. Theoretically, it is possible to delve into the details of these cases using primary or secondary literature to identify what are the conditions under which each specific case pursued activation. Practically, however, with eight cases, such an endeavour would go beyond the scope of this article. Moreover, the fsQCA analysis reveals that two governments should have pursued activation because of their membership to at least one of the paths, but did not (Dehaene 2 and Lipponen 2). To understand fully the conditions under which governments pursue activation, these two cases need to be studied too, making the number of cases to examine in more detail 10. The larger the number of cases in an fsQCA analysis, the larger the number of such cases may be. Consequently, providing an account of (all) these cases might not be possible. This is a price to pay for conducting an fsQCA analysis with a moderately large-n.
Discussion
This article has shown that both fsQCA and regression analysis have something to offer for moderately large-n studies. Regarding the former, one has to accept that with such an n, being able to account for all cases is likely difficult. While most scholars using regression analysis would probably be quite or even very happy when they can explain almost 80 percent of the variation in their dependent variable (depending on the type of regression analysis pursued of course), for someone interested in the actual cases, a failure to account for the outcome of 20 percent of them is disappointing. With a higher number of cases, the fsQCA analysis paints more of the broad picture—it helps to identify the effects-of-(multiple)-conditions-of-causes rather than the typical qualitative mode of analyzing causes-of-effects. From a qualitative perspective, this is a price to pay indeed. However, it may be one worth paying since an advantage of a moderately large-n is that complementing the results with a regression analysis becomes an option. This allows different yet complementary hypotheses to be tested within one study and generates more insights into the research topic at hand.
Summing up, this article has tried to contribute to the recent literature of comparative methods by examining the strengths and weaknesses of regression analysis and fsQCA analysis for studies with a moderately large-n. Despite the increased attention for the combination of configurational approaches with more traditional statistical techniques or case-oriented ones, most focus has so far been on studies with a large-n or an intermediate-n. Since a moderately large-n study has some specific features, making it a perfect candidate neither for a statistical analysis nor for a configurational comparative method, this new angle is a contribution to the literature. Adding a configurational approach to a regression analysis helps to uncover patterns in the empirical data that otherwise would have remained hidden. Making use of the set-theoretical logic underlying fsQCA and a traditional quantitative approach as regression analysis thus seems an excellent way (to stay) out of the “sea of naïveté” Sartori warned us of four decades ago.
Footnotes
Acknowledgements
Previous versions of this article have been presented at a master class on QCA applications, Leuven, Belgium, at a seminar on Comparative Methodology at the University of Southern Denmark, Odense and at the ESPAnet annual conference 2010. Thanks to all participants for their constructive comments, especially to Patrick Emmenegger, Alexander Goerne, Olli Kangas, Jon Kvist, Mattias Larsen, Svend-Erik Skaaning, Peter Starke, and Maarten Vink. Additionally, thanks to Gijs Schumacher for his useful input and to the three anonymous reviewers of Sociological Method & Research for their constructive comments.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is supported by a Veni grant from the Netherlands Organization for Scientific Research (NWO, grant nr. 451-08-012).
Notes
Bio
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
