Comparative mainstreaming? Mapping the uses of the comparative method in social policy,sociology and political science since the 1970s

Abstract

This article maps the development and uses of the comparative method in academic research since the 1970s. It is based on an original database that we constructed for our review of 12,483 articles extracted from leading journals representing the disciplines of Social Policy, Political Science and Sociology. We proceed to a quantitative and qualitative analysis of the reported comparative research effort. We find that the comparative method became mainstream in the 1990s – following the publication of the Three Worlds of Welfare Capitalism and that JESP is the most comparative journal of all. In 2020, 66% of articles published in JESP are comparative. The comparative turn has been stronger in Social Policy than Sociology and Political Science over the last three decades. We witness a rise in the use of formal techniques (case studies and comparative historical analysis, SEM/factorial techniques, cluster analysis, QCA/Fuzzy-set) and mixed-methods in comparison to descriptive analysis, and this is particularly pronounced in Sociology. Regression analysis is dominant, however the most cited comparative articles are based on case studies and descriptive statistics. Overall, we argue that the comparative method is, in essence, ‘a way of thinking’ and not simply the application of a set of disparate techniques.

Keywords

Comparative method the three worlds of welfare capitalism,social policy sociology political science case studies factorial techniques cluster analysis QCA/fuzzy-set regression analysis

Introduction

As some topics across the social sciences, like the study of gender, became increasingly ‘mainstream’ (Daly, 2005), important methodological developments are occurring as well. By methodological developments here we do not simply refer to the innovations stemming from the use of new techniques (an important factor of course!), but also to a ‘new’ way of looking at the world largely inspired by the nineteenth-century classics (for example, Tocqueville, 1960; Weber, 1930) and the formalization of the comparative method. As Giovanni Sartori (1970: 1033) acutely observed more than 50 years ago:

‘In a very crucial sense there is no methodology without logos, without thinking about thinking. And if a firm distinction is drawn – as it should be – between methodology and technique, the latter is no substitute for the former. One may be a wonderful researcher and manipulator of data, and yet remain an unconscious thinker.’

From this point of view, when investigating the uses of methodology in the main journals in Social Policy, Sociology and Political Science, it appears that the formalization of the comparative method has assumed the status of ‘a new way of thinking’ in order to better understand social and institutional change. The ‘comparative turn’ in the 1990s has been more radical in Social Policy than in Sociology and Political Science. Works like the Three Worlds of Welfare Capitalism and the arrival of the Journal of European Social Policy (JESP) (also Social Politics (SP) and the European Sociological Review (ESR) to a degree) have deeply contributed to it. Accordingly, three research questions drive our enquiry:

To what degree are we seeing a growth in comparative research over time and are there specific disciplinary trends at play? Did comparative methods really ‘come of age’ in 1990s and become ‘mainstream’?

Are scholars using methods in a more formalized way today and are there discernible trends by discipline? What are the key trends in techniques and approaches over the past 50 years?

What are the methodological and substantive characteristics of the most cited comparative research articles?

To address these questions, we created an original database – ‘The Comparative Journals Database’ – that allows us to quantitatively and qualitatively map the uses of the comparative method in research articles published in leading Social Policy, Sociology and Political Science journals over the past five decades.

The next section discusses the attributes of the newly created database. We then report our findings based on a quantitative analysis of the database. Finally, we undertake a qualitative review of the most cited comparative articles – the ‘greatest hits’.

The comparative journals database

The database contains 12,483 articles appearing in ten leading social science journals for the period 1970–2015, with an update for JESP in 2016–2020:

For Social Policy, we include JESP and SP, the Journal of Social Policy (JSP), and Social Policy & Administration (SPA).

For Sociology, we include the American Journal of Sociology (AJS), American Sociological Review (ASR) and the ESR.

For Political Sciences, we include the American Political Science Review (APSR), Comparative Political Studies (CPS) and World Politics (WP).

With our sample,¹ we cannot of course claim to represent the entire field, across the disciplines. First, by definition our database does not include books, volumes or research reports in the ‘grey’ literature. Second, our review is restricted to research articles in the English language. Parsimony is our guiding principle, since we cannot hope to review all of the scholarly literature. So we decided to focus on a selected sample of leading journals, striking a balance between the disciplines and research output from well-established European and American journals. Arguably, this approach allows us to accurately trace and assess the most important methodological developments and trends in the comparative field.

Each article in the database was sorted, reviewed manually and cross-checked to identify and separate the comparative articles from the non-comparative contributions. Despite the inherently comparative nature of scientific inquiry, Ragin (2014) pointed to the fact that, while all research methods are comparative in a broad sense, in the social sciences the idea of comparative inquiry is mostly used to refer to research involving the use of large macro-social units of analysis.

This definition of comparative research is not universally accepted. Other scholars have proposed different boundaries to delimit the domain of comparative inquiry. On the one hand, those more geared toward the use of quantitative and multivariate techniques have defined the comparative method simply by considering studies that include comparative data from different societies (Andreski, 1965; Armer, 1973) or works based on multilevel analysis (Rokkan, 1966; Przeworski and Teune, 1970). On the other hand, scholars more versed in qualitative/historical analysis such as Moore (1966) and Skocpol (1979) tend to distinguish between case-based and variable-orientated comparative methods (the lineage is of course traced to the founding fathers of sociology and political science, that is, Tocqueville, Durkheim and Weber). These views are perhaps too restrictive, and for this reason, we follow Ragin, defining the comparative method on the basis of its goals – attempting to understand causal complexity and relations between macro and micro units of analysis for example – rather than specific methodological orientations. The analysis of macrosocial unit is a ‘meta-theoretical category’, which basically distinguishes comparative social scientists from the others, because they use ‘macrosocial units in explanatory statements’ (Ragin, 2014: 5). Indeed, the vast majority of scholars working in the field (including the authors of this article!), often do not define the nature and the role of the macrosocial units, but rather use them implicitly as ‘observation’ and/or ‘explanatory’ units of analysis (Ragin, 2014: 8).

In light of this definition, we found that 14.7% (n = 1834) of research articles in our sample were comparative in nature and the rest (85.3%) we classified as non-comparative or theoretical (n = 10,649). Relevant details and basic bibliographic information were extracted from the comparative articles, including the DOI, the year of publication, the authors, the journal and the discipline, we coded methods into seven categories: (1) descriptive statistics only (that is, no use of formal methods beside simple descriptive statistics), (2) case studies and comparative historical analysis (CHA), (3) qualitative comparative analysis (QCA)/fuzzy-sets, (4) regression techniques, (5) structural equation modelling (SEM) and factorial analyses, (6) cluster analysis, and (7) other techniques. The ‘other techniques’ category includes methodologies that are used infrequently, such as diagonal reference models, sequence analysis, scale construction, thematic analysis, propensity score matching (PSM), optimal matching, Krippendorff’s alpha (KA) and event history analysis. Moreover, we included the total number of methods used.

The coding process followed four steps, over a period of 36 months:

We calibrated the measurement process by coding a random sample of 50 articles from each journal. Each article was coded by the two authors and a team of research assistants.

We then discussed the results, checking for consistency according to our definition of comparative method.

The research assistants then coded all of the articles in the entire database.

Finally, a sample comprising 50 articles was then independently coded and checked by the two authors to ensure reliability.

We are also interested in the relative importance of top cited articles in the database. For this reason, we employed as a proxy measure the number of citation counts extracted from Google Scholar (19 July 2020). We developed a search command written in R to capture the citation counts associated with each record contained in the database. Google Scholar is less than perfect for determining the relative importance of research articles in the literature, but it does at least provide a reliable metric on highly cited articles (Jacsó, 2012; Martin-Martin et al., 2017). The R programme included a validation step that enabled us to detect errors. It imported bibliographic information as well as the citation details, which enabled us to run cross-checks against our bibliographic records in the database. Thanks to this device, all the database records were carefully checked, missing citation entries were entered manually (n = 549, 4.4% of the total) and any errors were corrected (n = 265, 2.1% of the total). The top cited comparative articles included in the qualitative literature review are extracted using this citation count.

Quantitative trends in comparative analysis

A growing number of comparative articles

The scholarly production of comparative articles starts at relatively low levels in the 1970s, and then progressively increases during the 1990s. The rise in Social Policy is more marked than in Sociology and Political Science (Figure 1). In 2015, comparative articles constitute more than a third of all articles published in leading Social Policy journals, while only around one-fifth are published in the other two disciplines. In addition, there are considerable differences across journals within the same discipline.

Between the early 1970s and the mid-1980s, less than 10% of all articles published in Sociology, Political Science and Social Policy were comparative. Some difference already exists in this early period between Sociology and Political Science on the one hand and Social Policy on the other. While in Sociology and Political Science, the trend is rather flat, in Social Policy there is considerable variation. A special issue of SPA – focused on sub-national units of analysis in England (that is, counties and regions) – accounts for the spike observed in 1971.

The year 1990 is a ‘watershed moment’, that is, a point in time that marks an historical change for comparative inquiry. Just 11% of Social Policy articles published during the period 1970–1990 were comparative studies, compared to 26% in the later period. We call this development ‘the Esping-Andersen effect’ suggesting The Three Worlds of Welfare Capitalism has helped to transform comparative inquiry in Social Policy (Esping-Andersen, 1990; see also the subsequent debate: Arts and Gelissen, 2002; Ferragina and Seeleib-Kaiser 2011; Emmenegger et al., 2015; Deeming, 2017; Ferragina and Deeming, 2022; Ferragina and Filetti 2022). While we talk of an ‘effect’, we recognize Esping-Andersen’s (1990) work represents the zeitgeist of the time more than determining an effect in itself; also evident in Graham Room’s decision to launch JESP, the first issue appearing in 1991.² This rise in the use of the comparative method is also observable – although less pronounced than in Social Policy – in Sociology (7% before 1990, 14% thereafter) and Political Science (from 9% to 16%) (Figure 2).

The consideration of trends within disciplines shows an interesting variation across journals. In Social Policy, we witness a strong rise in the number of comparative articles in all four journals but JESP is the most comparative of all (Figure 3). In 2015, one in two articles published in this journal is comparative. Next comes SP and SPA, with about a third of comparative articles. Finally, comes the JSP, with only around 20% of comparative articles. While our database allows us to analyse the trends up to 2015, we extended our coverage of JESP to 2020 (as it is clearly the most comparative journal in our sample). We find the trend in the use of the comparative method has steadily continued (Figure 4). In 2020, two thirds (66%) of all articles published in JESP report comparative research.

For some journals, the rise after 1990 is sharp, for example, WP, SPA and JSP (Figure 5). WP trebles comparative research output, 7% before 1990, 21% after. The picture is similar for SPA and JSP, 13%–21% and 9%–15% respectively. However, a 15% comparative share is still relatively low compared to most other journals as well as the total average of 18% (Figure 5).

For other journals, the increases in comparative outputs are smaller and only marginal at times. This is particularly true for flagship American journals in Sociology and Political Science (ASR and APSR). The low number and increment of comparative production seem to signal a smaller interest in comparison to European journals. AJS constitutes a partial exception to this pattern, and although only 11% of articles published are comparative since the 1990, the percentage has almost doubled in comparison to the pre-1990 period. CPS is broadly constant over the two periods, the comparative share of research articles increased from 23% to 25%. In other words, ‘the Esping-Andersen effect’ is less evident here.

It is not possible to do before- and after-1990 comparisons for the newer journals like JESP, SP, and the ESR. Together, these journals are leading the comparative research development and one can reasonably argue that the establishment of these journals constitutes an effort in itself to spread the comparative approach in Social Policy and Sociology (Figure 5). From the data it is clear that the comparative effort in leading journals has increased in all disciplines and this increase had been particularly marked in Social Policy, and within the pages of JESP in particular.

The changing landscape in the use of comparative methods

The sharp rise in the use of formal comparative methods stands in contrast to the steady but lower rise in the use of descriptive analysis. We witness a rise in the use of formal methods after 1990 and more decisively since 2004. In 1990, for example, there were eight comparative descriptive articles published and 13 articles employing a formal comparative method. By 2015, 67 articles employed a formal comparative method, while only 23 were comparative and descriptive (Figure 6).

Overall, 65% of published comparative articles report the use of a formal method. The preference for the use of a formal method is more marked in Sociology (86% of the total number of comparative articles) compared to Political Science (57%) and Social Policy (56%) (Figure 7).

In Figure 8, we witness the dominant position of regression techniques in the comparative research literature, as scholars exploit the growing number of comparative datasets such as the Comparative Welfare State Dataset (CWD) and the Comparative Welfare Entitlements Dataset (CWED/CWED2) as well as the array of datasets from the OECD. The regressions trend line in Figure 8 shows a steep gradient, rising sharply from the early 1990s onwards.

Case study and CHA are also increasingly employed in comparative research, although to a lower extent than regression techniques. Only two or three studies were published each year using these approaches during the 1970s and 1980s, while the annual average throughout the 1990s was ten. A similar rise – although less steady – is also observable for SEM and factorial techniques. The use of clustering techniques and QCA/fuzzy-sets has also been increasing, and notably picking up during the 2000s and 2010s but remains lower than the other techniques previously described.

Most comparative scholars employ one method of analysis, but we note an increasing preference for the use of multi-methods designs during the 2000s. Overall, there are 635 descriptive only comparative studies, 929 comparative studies are based on one method only and 270 comparative studies employ multi-method designs (Figure 9).

Qualititative analysis of the ‘greatest hits’

For our qualitative analysis of comparative research trends, we identified the most cited comparative articles in our database and labelled them as the ‘greatest hits’. Table 1 shows the top 20 ‘greatest hits’ overall, and Table 2 the most cited ‘greatest hits’ in the Social Policy Journals (Table A2 in the online Appendix shows the most cited ‘greatest hits’ in the Political Science Journals and Table A3 shows the most cited comparative articles in the Sociology Journals).

Table 1.

‘Greatest hits’ (most cited comparative articles). Measured using Google Scholar 19 July 2020.

Rank	Journal	Authors	Year	Citations	Unit of Analysis	N of units	Methods	EA
1	WP	Pierson	1996	5097	Nations	4	Case study, descriptive statistics	Yes
2	CPS	Laakso and taagepera	1979	4126	Nations	15	Correlation techniques, descriptive statistics
3	JESP	Ferrera	1996	3738	Nations	4	Case study, descriptive statistics	Yes
4	JESP	Lewis	1992	3098	Nations	4	Case study, descriptive statistics	Yes
5	APSR	Cameron	1978	2851	Nations	18	Regression techniques
6	APSR	Inglehart	1971	2365	Nations	6	Multivariate models
7	ASR	Korpi and Palme	1998	2239	Nations	18	Regression techniques, descriptive statistics	Yes
8	WP	Hellman	1998	1974	Nations	19	Regression techniques, descriptive statistics
9	WP	Krasner	1976	1720	Nations	10	Descriptive statistics
10	CPS	Mishler and Rose	2001	1513	Nations	10	Regression techniques, factor analysis
11	APSR	Laver et al	2003	1405	Nations	3	Other techniques, word scoring technique
12	AJS	Huber et al	1993	1390	Nations	17	Regression techniques, PTS	Yes
13	APSR	Korpi and Palme	2003	1342	Nations	18	Regression techniques	Yes
14	AJS	Hamilton and Biggart	1988	1298	Nations	3	Case study, descriptive statistics
15	APSR	Bingham Powell	1986	1236	Nations	9	Regression techniques
16	JSP	Macintyre et al	1993	1228	Local areas	2	Descriptive statistics
17	ESR	Allmendinger	1989	1214	Nations	3	Regression techniques
18	SP	Korpi	2000	1140	Nations	18	Descriptive statistics	Yes
19	ASR	Phillips	1974	1153	Nations	2	Regression techniques
20	APSR	Anderson and Guillory	1997	1112	Nations	11	Regression techniques

Abbreviations: EA - Esping-Andersen, 1990, is cited.

Journals abbreviations: AJS - American Journal of Sociology; APSR - American Political Science Review; ASR - American Sociological Review; CPS - Comparative Political Studies; ESR - European Sociological Review; JESP - Journal of European Social Policy; JSP - Journal of Social Policy; SP - Social Politics; WP - World Politics.

Table 2.

‘Greatest hits’ in Social Policy (most cited comparative articles). Measured using Google Scholar 19 July 2020.

Rank	Journal	Authors	Year	Citations	Unit of Analysis	N of units	Methods	EA
1	JESP	Ferrera	1996	3738	Nations	4	Case study, descriptive statistics	Yes
2	JESP	Lewis	1992	3098	Nations	4	Case study, descriptive statistics	Yes
3	JSP	Macintyre et al	1993	1228	Local areas	2	Descriptive statistics
4	SP	Korpi	2000	1140	Nations	18	Descriptive statistics	Yes
5	JSP	Bonoli	1997	1074	Nations	22	Descriptive statistics	Yes
6	SP	Lewis	2001	1001	Nations	2	Case study, descriptive statistics [very little]	Yes
7	JESP	Anttonen & Sipilä	1996	945	Nations	14	Descriptive statistics	Yes
8	JESP	Gornick et al	1997	816	Nations	14	Descriptive statistics	Yes
9	JESP	Van Oorschot	2006	745	Nations	23	Regression techniques	Yes [1996]
10	JESP	Bettio et al	2006	728	Nations	4	Case study, descriptive statistics	Yes
11	SP	Knijn & Kremer	1997	706	Nations	3	Case study, descriptive statistics	Yes
12	JESP	Albertini et al	2007	593	Nations	10	Regression techniques	Yes
13	JESP	Scruggs & Allan	2006	545	Nations	18	Descriptive statistics	Yes
14	JESP	Castles	2003	543	Nations	21	Regression techniques	Yes [1996, 1999]
15	JESP	Deacon	2000	540	Nations	8	Descriptive statistics
16	JESP	Van Oorschot & Arts	2005	503	Nations	23	Regression techniques, factor analysis	Yes [1999]
17	SP	Walby	2004	482	Nations	8	Descriptive statistics [very little]	Yes
18	SP	Daly	2005	475	Nations	8	Descriptive statistics [very little]
19	JESP	Pavolini & Ranci	2008	426	Nations	6	Descriptive statistics	Yes [1999]
20	JESP	Alber	1995	395	Nations	3/12	Case study, descriptive statistics	Yes

Abbreviations: EA: Esping-Andersen, 1990 is cited. We also include Esping-Andersen, 1996 and 1999 where cited.

Journals abbreviations: JESP - Journal of European Social Policy; JSP - Journal of Social Policy; SP - Social Politics.

Among the 20 ‘greatest hits’ (Table 1) we find works from every decade, the oldest in the early 1970s (Inglehart, 1971) and the most recent in the early 2000s (Laver et al., 2003; Korpi and Palme, 2003). Five articles were published in the 1970s (Inglehart, 1971; Phillips, 1974; Krasner, 1976; Cameron, 1978; Laakso and Taagepera, 1979), three in the 1980s (Bingham Powell, 1986; Hamilton and Biggart, 1998; Allmendinger, 1989), eight in the 1990s (Lewis, 1992; Huber et al., 1993; Macintyre et al., 1993; Pierson, 1996; Ferrera, 1996; Anderson and Guillory, 1997; Korpi and Palme, 1998; Hellman, 1998), and four in the 2000s (Korpi, 2000; Mishler and Rose, 2001; Laver et al., 2003; Korpi and Palme, 2003). The pre-eminence of greatest hits in the 1990s when compared to the earlier period confirms the findings from our quantitative review, The Three Worlds and the establishment of journals geared to comparison like JESP, SP and ESR constitutes a watershed for the use of comparative analysis. Moreover, three of the five most cited articles are directly related to The Three Worlds of Welfare Capitalism. These works employ Esping-Andersen (1990) as a theoretical framework to discuss welfare state retrenchment and case selection (Pierson, 1996), to include Mediterranean countries within welfare regime typology (Ferrera, 1996), and to criticize his male breadwinner orientation from a feminist perspective (Lewis, 1992) (Table 1). Esping-Andersen’s influence is clearly evident across the highly cited ‘greatest hits’ in Social Policy seen in Table 2 – where 17 of the top 20 articles³ actively engage with his work on welfare regimes (Esping-Andersen, 1990, 1996, 1999), but also across Political Science and Sociology journals (Table A2 and A3 online Appendix).

The four top cited articles are all published in journals explicitly geared towards comparative analysis, that is, WP, CPS and JESP. Other journals with a high share of comparative articles over the total, that is, SP and the ESR, feature two articles in the top 20. The remainder were published in generalist journals with a lower share of comparative articles, that is, the APSR, AJS, the ASR and the JSP (Table 2).

Looking across disciplines, JESP dominates the Social Policy ranking with 14 articles, followed by SP with five. The JSP and SPA are much less represented (Table 2). In Political Science, the APSR has published 12 among the 20 most cited comparative articles, followed by WP with seven and CPS with one (Table A2, online Appendix). In Sociology, the AJS has the lead with 11, followed by the ASR with six and the ESR with three (Table A3, online Appendix).

From a methodological point of view, among the top five greatest hits, we find three studies employing case study as the method of analysis (Pierson, 1996; Ferrera, 1996; Lewis, 1992), one based on descriptive statistics (Laakso and Taagepera, 1979) and one using regression analysis (Cameron, 1978). In the top 20 greatest hits, eleven of them are based on regressions, four use simple descriptive statistics and no formal methods; four are based on case studies and one uses a word scoring technique (Table 1). This means that the most cited articles rely heavily on traditional approaches and techniques and, in particular, case study methodology seems core in classical comparative empirical work. This is a clear reflection of the origins of comparative method, grounded in nineteenth-century classics, based on non-formalized case study research (for example, Tocqueville, 1960; Weber, 1930).

At the disciplinary level, within Social Policy there is a strong prevalence of studies based on descriptive statistics (n=10), followed by case studies (n = 4) and articles based on regression techniques (n = 4). This confirms a general trend previously highlighted: within comparative articles published in Social Policy journals there is a stronger tendency to use descriptive statistics rather than formal methods in comparison to Political Science and Sociology (Figure 8). We suggest this is a reflection of the large prevalence of comparative institutional analysis and a more inductive way to compare than in Political Science and Sociology journals (Table 2). In Political Science and Sociology journals, studies based on regression techniques are prevalent (13 and 10 out of 20 respectively). A difference between the greatest hits in Political Science and Sociology is that in the latter, there is a slightly larger array of methods represented in the top 20 greatest hits, for example, QCA and randomized experiments. This is a reflection of the largest heterogeneity in Sociology, where more comparative studies on different subnational units of analysis are also published (Table A4, online Appendix). Interestingly, the methodologies employed across the greatest hits are also somehow heterogeneous within disciplines and across journals. For example, in Political Science journals, regression techniques are prevalent in a generalist journal like the APSR, while case studies are found only in WP.

Overall, we find a good representation of the methods we quantitatively mapped out: case studies, QCA, regression techniques, factorial techniques, the simple use of descriptive statistics, and the use of other less common methodologies in the comparative field, that is, word scoring techniques, block modelling techniques and the growing interest in experimental designs.

Case studies are most commonly employed across our greatest hits to investigate varieties or highlight similar trends across countries more or less explicitly following prescriptions on the method of agreement and difference from John Stuart Mill. With the exception of Gastil (1971), these case studies are situated at the cross-national level of analysis. Several of those employ The Three Worlds as their starting point, for example, Pierson (1996), Ferrera (1996), Lewis (1992, 2001), Bettio et al. (2006), Knijn and Kremer (1997), Alber (1995), while other works are focused on comparative organizational arrangements (Hamilton and Biggart, 1998), the expansion of the varieties of capitalism framework (Nölke and Vliegenthart, 2009), the study of public opinion and policy in an international context (Risse-Kappen, 1991), and the explanation of the neoliberal shift in politics across nations (Fourcade-Gourinchas and Babb, 2002).

QCA occupies an interesting position in the field of comparative methodology, as it bridges case-based small-N analysis and large-N quantitative work and also the qualitative and quantitative divide. Among the top cited articles, we found only one example of QCA (the technique has been first proposed Charles Ragin in 1987 (Ragin, 1987). Cress and Snow (2000) innovatively use QCA to compare the results gathered using ethnographical methods in eight American cities in their study of homeless organizations.

The use of regression techniques is well-represented in our greatest hits, with contributions spanning across four decades (Cameron, 1978; Inglehart, 1971; Hellman, 1998; Mishler and Rose, 2001; Huber et al., 1993; Korpi and Palme, 1998, 2003; Bingham Powell, 1986; Allmendinger, 1989; Anderson and Guillory, 1997; Van Oorschot, 2006; Albertini et al., 2007; Castles, 2003; Iversen and Soskice, 2001; Boix, 1999; Bradley et al., 2003; Schraeder et al., 1998; Phillips, 1974; Svallfors, 1997; Alderson and Nielsen, 2002; Wimmer et al., 2009; Scheepers et al., 2002; Massey and Eggers, 1990; Thoits, 1986; Massey and Denton, 1988). Regression techniques tend to be used in large-N contexts (although there are examples of studies using fewer countries) and within our sample, we found a large use of OLS and logistic regressions, as well as growing interest in multilevel modelling (MLM) and time series cross-sectional analyses (TSCS). In comparison to other methods, regression techniques are used in a more versatile way to support comparative analysis at different scales, for example, cross-national, across states, cities, and more local areas.

Within the greatest hits there are no articles solely based on SEM, factorial techniques or cluster analysis. However, we found these techniques in mixed-method research designs (prevalently in conjunction with regression techniques). Hence, mixed methods are used to provide more accurate – multidimensional – approaches geared toward the establishment of accurate description, association and causation. In our qualitative sample, the combination of regressions and different SEM and factorial techniques is used to explore complex subjects, like the impact of different welfare regime structures on social capital (Van Oorschot and Arts, 2005), determinants of political trust (Mishler and Rose, 2001), determinants of social welfare spending (Hicks and Swank, 1992; Iversen and Cusack, 2000), and trends in residential segregation in US metropolitan areas (Massey and Denton, 1987).

Despite an increasing sophistication and formalization in the use of comparative methods, many important works in the comparative field employ descriptive statistics only (Laakso and Taagepera, 1979; Krasner, 1976; Macintyre et al., 1993; Korpi, 2000; Bonoli, 1997; Anttonen and Sipilä, 1996; Gornick et al., 1997; Scruggs and Allan, 2006; Deacon, 2000; Walby, 2004; Daly, 2005; Pavolini and Ranci, 2008; Alford et al., 2005; Massey et al., 1994; Mann, 1970; Black, 1970; Portes and Sassen-Koob, 1987). However, the share of research articles based on descriptive statistics within the greatest hits is decreasing over time.

Finally, there is a small array of articles among our greatest hits employing less familiar techniques. For instance, word scoring technique has been employed to extract policy positions from the analysis of political texts in the United Kingdom, Ireland and Germany considering discourses as data in the form of words (Laver et al., 2003). Black modelling techniques have been used in conjunction with regression techniques to explore world system theory (Wallerstein, 1974) at the cross-national (Snyder and Kick, 1979) and city level (Alderson and Beckfield, 2004). Additionally, experimental designs have been used to generate new insights, such as the relationship between national cultural contexts and levels of trust (Yamagishi et al., 1998). The work by Yamagishi and colleagues is particularly innovative at this time drawing comparisons between the United States and Japan.

Conclusion

Comparative scholarship is rapidly growing across disciplines. The upward trend is strongly pronounced in the field of Social Policy, as reflected in the establishment of new comparative journals – JESP and SP – and the increasing comparative output published by older journals like SPA and JSP. This growing comparative trend is also evident in Sociology and Political Science, particularly in journals such as ESR, WP and CPS. Interestingly, the AJS, APSR and ASR (flagship American journals) include significantly less comparative output compared to the European journals in our sample, and this does not seem to change much over time. JESP is the most comparative journal in our sample, two thirds (66%) of the articles published here in 2020 are comparative in nature.

The year 1990 appears to be a turning point in comparative inquiry, most evident in Social Policy. We provocatively call this development ‘the Esping-Andersen effect’, suggesting The Three Worlds of Welfare Capitalism symbolizes an important transformation in the discipline. The debate over welfare regimes has fuelled the comparative effort. Our qualitative analysis of the greatest hits also shows the influence of Esping-Andersen’s work which is cited and debated from different perspectives. Of course, the influence of The Three Worlds is much stronger in Social Policy, where among the 20 most cited articles, 17 directly refer to Esping-Andersen’s work.

We find comparative researchers are increasingly turning to formal comparative methods and techniques, rather than relying purely on descriptive analysis, prevalent in the earlier decades. The quest for rigour and formalization in comparative research may not surprise, as the norms in the social sciences have changed significantly over the past half-century – the landscape is unrecognizable with the comparative turn.

The comparative turn is associated with increased formalization in the uses of method, most evident in Sociology. The methodological toolbox is becoming more organized, evident in the annual increase in the use of regression techniques, case studies and CHA, SEM and factorial techniques, QCA and fuzzy-sets and clustering techniques, and also by the growing number of works employing mixed- and multi-method research designs. Regression techniques dominate. However, among the top five most cited articles in our sample, case study designs appear three times. So despite the increased use of regression analysis and the growing trend towards large-N studies, classical comparisons and small-N case studies continue to have a strong and lasting influence on today’s research effort.

Such considerations are of course present in Sartori’s original argument (our point of departure), reflecting our empirical findings. The increasing complexity of methods and techniques now being employed has contributed to the development and rigour of social science research. The comparative turn opened up new possibilities and new directions for research, for example, the increasing N size and the diverse range of country-level macro-units employed in comparative research designs are testament to this (see Figure A1 and Table A5, online Appendix). The growing importance and formalization of the comparative method – largely inspired by nineteenth-century classics – is clearly evident in our review. Nevertheless, the comparative turn is not simply related to the growing application of sophisticated techniques and computing power (important though this is), but rather a growing appreciation of the way to look at, examine and investigate the world beyond the analysis of single case studies. Despite notable disciplinary differences, the use(s) of the comparative method has now become ‘mainstream’.

Supplemental Material

Supplemental Material - Comparative mainstreaming? Mapping the uses of the comparative method in social policy, sociology and political science since the 1970s

Supplemental Material for Comparative mainstreaming? Mapping the uses of the comparative method in social policy, sociology and political science since the 1970s by Emanuele Ferragina and Christopher Deeming in Journal of European Social Policy.

Footnotes

Acknowledgements

We are grateful to the research assistants based at Sciences Po, Paris and the University of Strathclyde, Glasgow who assisted with development of the database, downloading all of the articles, creating a library, and conducted the initial reviews (Taysheona Denise Brodie, Edouard Crocq, Emmaleena Käkelä and Federico Plantera). Also, to Yannick Savina who wrote the syntax command in R that we used to capture the citation counts in Google Scholar.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article benefits from the support provided by the ANR and the French government under the ‘Investissements d’Avenir’ programme LABEX LIEPP (ANR-11-LABX-0091, ANR-11-IDEX-0005–02) and the IdEx Université Paris Cité (ANR-18-IDEX-0001) and institutional research funding awards from Sciences Po Paris and the University of Strathclyde Glasgow for the project Methodologies for Comparative Analysis.

Supplemental Material

Supplemental material for this article is available online.

ORCID iD

Emanuele Ferragina

Notes

References

Alber

(1995) A framework for the comparative study of social services. Journal of European Social Policy 5(2): 131–149.

Albertini

Kohli

Vogel

(2007) Intergenerational transfers of time and money in European families. Journal of European Social Policy 17(4): 319–334.

Allmendinger

(1989) Educational systems and labor market outcomes. European Sociological Review 5(3): 231–250.

Alderson

Beckfield

(2004) Power and position in the world city system. American Journal of Sociology 109(4): 811–851.

Alderson

Nielsen

(2002) Globalization and the great U-turn. American Journal of Sociology 107(5): 1244–1299.

Alford

Funk

Hibbing

(2005) Are political orientations genetically transmitted? The American Political Science Review 99(2): 153–167.

Anderson

Guillory

(1997) Political institutions and satisfaction with democracy. American Political Science Review 91(1): 66–81.

Andreski

(1965) The Uses of Comparative Sociology. Berkeley, CA: University of California Press.

Anttonen

Sipilä

(1996) European social care services: is it possible to identify models? Journal of European Social Policy 6(2): 87–100.

10.

Armer

(1973) Methodological problems and possibilities in comparative research. In: Armer

Grinmshaw

(eds), Comparative Social Research. New York: Wiley, pp. 49–79.

11.

Arts

Gelissen

(2002) Three worlds of welfare capitalism or more? Journal of European Social Policy 12(2): 137–158.

12.

Bettio

Simonazzi

Villa

(2006) Change in care regimes and female migration. Journal of European Social Policy 16(3): 271–285.

13.

Bingham Powell

(1986) American voter turnout in comparative perspective. American Political Science Review 80(1): 17–43.

14.

Black

(1970) Production of crime rates. American Sociological Review 35(4): 733–748.

15.

Bonoli

(1997) Classifying welfare states. Journal of Social Policy 26(3): 351–372.

16.

Boix

(1999) Setting the rules of the game. American Political Science Review 93(3): 609–624.

17.

Bradley

Huber

Moller

, et al (2003) Distribution and redistribution in postindustrial democracies. World Politics 55(2): 193–228.

18.

Cameron

(1978) The expansion of the public economy. American Political Science Review 72(4): 1243–1261.

19.

Castles

(2003) The world turned upside down. Journal of European Social Policy 13(3): 209–227.

20.

Cress

Snow

(2000) The outcomes of homeless mobilization: the influence of organization, disruption, political mediation, and framing. American Journal of Sociology 105(4): 1063–1104.

21.

Daly

(2005) Gender mainstreaming in theory and practice. Social Politics 12(3): 433–450.

22.

Deacon

(2000) Eastern European welfare states. Journal of European Social Policy 10(2): 146–161.

23.

Deeming

(2017) The lost and the new ‘Liberal World’ of welfare capitalism: a critical assessment of Gøsta Esping-Andersen's The Three Worlds of Welfare Capitalism a quarter century later. Social Policy and Society 16(3): 405–422.

24.

Emmenegger

Kvist

Marx

, et al (2015) Three worlds of welfare capitalism: the making of a classic. Journal of European Social Policy 25(1): 3–13.

25.

Esping-Andersen

(1990) The Three Worlds of Welfare Capitalism. Princeton, NJ: Princeton University Press.

26.

Esping-Andersen

(ed), (1996) Welfare States in Transition. London: Sage.

27.

Esping-Andersen

(1999) The Social Foundations of Postindustrial Economies. Oxford: Oxford University Press.

28.

Ferragina

Deeming

(2022) Methodologies for comparative social policy analysis. In: MA

Yerkes

Nelson

Nieuwenhuis

(eds), Changing European Societies: The Role for Social Policy Research. Cheltenham: Edward Elgar, pp. 218–235.

29.

Ferragina

Filetti

(2022) Labour market protection across space and time: a revised typology and a taxonomy of countries’ trajectories of change. Journal of European Social Policy 32(2): 148–165.

30.

Ferragina

Seeleib-Kaiser

(2011) Welfare regime debate: past, present, futures? Policy & Politics 39(4): 583–611.

31.

Ferrera

(1996) The ‘Southern Model’ of welfare in social Europe. Journal of European Social Policy 6(1): 17–37.

32.

Fourcade-Gourinchas

Babb

(2002) The rebirth of the liberal creed. American Journal of Sociology 108(3): 533–579.

33.

Gastil

(1971) Homicide and a regional culture of violence. American Sociological Review 36(3): 412–427.

34.

Gornick

Meyers

Ross

(1997) Supporting the employment of mothers. Journal of European Social Policy 7(1): 45–70.

35.

Hamilton

Biggart

(1998) Market, culture, and authority. American Journal of Sociology 94: S52–S94.

36.

Hellman

(1998) Winners take all: the politics of partial reform in postcommunist transitions. World Politics 50(2): 203–234.

37.

Hicks

Swank

(1992) Politics, institutions, and welfare spending in industrialized democracies, 1960–82. American Political Science Review 86(3): 658–674.

38.

Huber

Ragin

Stephens

(1993) Social Democracy, Christian Democracy, constitutional structure, and the welfare state. American Journal of Sociology 99(3): 711–749.

39.

Inglehart

(1971) The silent revolution in Europe. American Political Science Review 65(4): 991–1017.

40.

Iversen

Cusack

(2000) The causes of welfare state expansion. World Politics 52(3): 313–349.

41.

Iversen

Soskice

(2001) An asset theory of social policy preferences. The American Political Science Review 95(4): 875–893.

42.

Jacsó

(2012) Using Google Scholar for journal impact factors and the h-index in nationwide publishing assessments in academia. Online Information Review 36(3): 462–478.

43.

Knijn

Kremer

(1997) Gender and the caring dimension of welfare states. Social Politics 4(3): 328–361.

44.

Korpi

(2000) Faces of inequality. Social Politics 7(2): 127–191.

45.

Korpi

Palme

(1998) The paradox of redistribution and strategies of equality. American Sociological Review 63(5): 661–687.

46.

Korpi

Palme

(2003) New politics and class politics in the context of austerity and globalization, 1975–95. The American Political Science Review 97(3): 425–446.

47.

Krasner

(1976) State power and the structure of international trade. World Politics 28(3): 317–347.

48.

Laakso

Taagepera

(1979) ‘Effective’ number of parties. Comparative Political Studies 12(1): 3–27.

49.

Laver

Benoit

Garry

(2003) Extracting policy positions from political texts using words as data. The American Political Science Review 97(2): 311–331.

50.

Lewis

(1992) Gender and the development of welfare regimes. Journal of European Social Policy 2(3): 159–173.

51.

Lewis

(2001) The decline of the male breadwinner model. Social Politics 8(2): 152–169.

52.

Macintyre

Maciver

Sooman

(1993) Area, class and health. Journal of Social Policy 22(2): 213–234.

53.

Mann

(1970) The social cohesion of liberal democracy. American Sociological Review 35(3): 423–439.

54.

Mishler

Rose

(2001) What are the origins of political trust? Comparative Political Studies 34(1): 30–62.

55.

Martin-Martin

Orduna-Malea

Harzing

, et al (eds), (2017) Can we use Google Scholar to identify highly-cited documents? Journal of Informetrics 11(1): 152–163.

56.

Massey

Denton

(1987) Trends in the residential segregation of Blacks, Hispanics, and Asians: 1970–1980. American Sociological Review 52(6): 802–825.

57.

Massey

Denton

(1988) Suburbanization and segregation in US metropolitan areas. American Journal of Sociology 94(3): 592–626.

58.

Massey

Eggers

(1990) The ecology of inequality, 1970–1980. American Journal of Sociology 95(5): 1153–1188.

59.

Massey

Goldring

Durand

(1994) Continuities in transnational migration. American Journal of Sociology 99(6): 1492–1533.

60.

Moore

Jr (1966) Social Origins of Dictatorship and Democracy. London: Penguin.

61.

Nölke

Vliegenthart

(2009) Enlarging the varieties of capitalism. World Politics 61(4): 670–702.

62.

Pavolini

Ranci

(2008) Restructuring the welfare state: reforms in long-term care in Western European countries. Journal of European Social Policy 18(3): 246–259.

63.

Phillips

(1974) The influence of suggestion on suicide. American Sociological Review 39(3): 340–354.

64.

Pierson

(1996) The new politics of the welfare state. World Politics 48(2): 143–179.

65.

Portes

Sassen-Koob

(1987) Making it underground: comparative material on the informal sector in western market economies. American Journal of Sociology 93(1): 30–61.

66.

Przeworski

Teune

(1970) The Logic of Comparative Social Enquiry. New York: Wiley.

67.

Ragin

(1987) The Comparative Method. Berkeley, CA: University of California Press.

68.

Ragin

(2014) The Comparative Method. Moving beyond Qualitative and Quantitative Strategies. Oakland, CA: University of California Press.

69.

Risse-Kappen

(1991) Public opinion, domestic structure, and foreign policy in liberal democracies. World Politics 43(4): 479–512.

70.

Rokkan

(1966) Comparative cross-national research. In: Merritt

Rokkan

(eds), Comparing Nations. New Haven, CT: Yale University Press. pp. 3–26.

71.

Sartori

(1970) Concept misformation in comparative politics. American Political Science Review 64(4): 1033–1053.

72.

Scheepers

Gijsberts

Coenders

(2002) Ethnic exclusionism in European countries public opposition to civil rights for legal migrants as a response to perceived ethnic threat. European Sociological Review 18(1): 17–34.

73.

Schraeder

Hook

Taylor

(1998) Clarifying the foreign aid puzzle. World Politics 50(2): 294–323.

74.

Scruggs

Allan

(2006) Welfare-state decommodification in 18 OECD countries. Journal of European Social Policy 16(1): 55–72.

75.

Skocpol

(1979) States and Social Revolutions. Cambridge: Cambridge University Press.

76.

Snyder

Kick

(1979) Structural position in the world system and economic growth, 1955–1970. American Journal of Sociology 84(5): 1096–1126.

77.

Svallfors

(1997) Worlds of welfare and attitudes to redistribution. European Sociological Review 13(3): 283–304.

78.

Thoits

(1986) Multiple identities: examining gender and marital status differences in distress. American Sociological Review 51(2): 259–272.

79.

Tocqueville

(1960) De la Démocratie en Amérique. London: Macmillan.

80.

van Oorschot

(2006) Making the difference in social Europe. Journal of European Social Policy 16(1): 23–42.

81.

van Oorschot

Arts

(2005) The social capital of European welfare states. Journal of European Social Policy 15(1): 5–26.

82.

Walby

(2004) The European Union and gender equality. Social Politics 11(1): 4–29.

83.

Wallerstein

(1974) The Modern World-System. New York: Academic Press.

84.

Weber

(1930) The Protestant Ethic and the Spirit of Capitalism. London: Unwin University Books.

85.

Wimmer

Cederman

Min

(2009) Ethnic politics and armed conflict. American Sociological Review 74(2): 316–337.

86.

Yamagishi

Cook

Watabe

(1998) Uncertainty, trust, and commitment formation in the United States and Japan. American Journal of Sociology 104(1): 194.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.42 MB