Opportunities and Threats in Reviewing Entrepreneurship Theory and Practice

Meta-Analysis

Meta-analysis combines the results of empirical studies using statistical methods for the synthesis of the literature. It has a key position in the evolution of knowledge in the entrepreneurship literature. Meta-analysis has been able to address and solve a number of lively debates in the field of entrepreneurship, for example, whether entrepreneurs should plan (Brinckmann, Grichnik, & Kapsa, 2010), whether personality is an important ingredient in the entrepreneurial journey (Rauch & Frese, 2007), and whether human capital and knowledge helps entrepreneurs (Unger, Rauch, Frese, & Rosenbusch, 2011). Often meta-analysis is stimulated by debates raised in qualitative (narrative) reviews and, notably, sometimes a meta-analysis arrives at different conclusions from the original narrative reviews. Because meta-analyses often fail to explain all the variance in reported effect sizes, they often include subgroup analyses to explain the remaining variance and raise many new questions for future research. Such standard meta-analyses provide an overall correlation between two variables at a time and combine this analysis with a subsequent subgroup analysis aiming to test whether the strength of the correlation differs depending on methodological or theoretical characteristics. While these standard meta-analyses have many advantages and have contributed to important debates in the field of entrepreneurship, they can have methodological as well as theoretical flaws. Methodologically, effect size estimates in this type of meta-analysis are often not accurate as there might be systematic errors (although many can be corrected), third variables that affect the effect size estimate, and correlated moderator variables. Theoretically, standard meta-analysis reporting bivariate relationships is unable to address multivariate theory. However, theories in entrepreneurship typically address multivariate relationships. Therefore, a standard bivariate meta-analysis may not meet the mission of Entrepreneurship Theory and Practice.

Meta-analysis can provide the type of contribution required by Entrepreneurship Theory and Practice through meta-analytical regression analysis (MARA) and meta-analytic structural equation modeling (MASEM). MARA is a multivariate extension of a subgroup moderator analysis (Schmidt, 2017). It is similar to a simple regression analysis, using the effect size estimate as a dependent variable in a regression analysis and the moderator variables as independent variables. MARA provides opportunities to test multiple continuous and categorical moderators and control variables at the same time, and is, therefore, able to test multivariate models and theories and to assess the boundary conditions of these theories. Unfortunately, entrepreneurship research does not have strong categorizations of moderator variables (Welter, 2011), which is an important weakness to overcome as a meta-analysis needs to conceptualize moderators at the level of the study in a meta-analysis. The exceptions are institutional variables that have been used in many meta-analyses (e.g., Saeed, Yousafzai, & Engelen, 2014). A creative example of this approach can be found in Bae, Qian, Miao, and Fiet (2014) who studied the relationship between entrepreneurship education and intentions that a previous meta-analysis had already established (Martin, McNally, & Kay, 2013). However, using MARA and controlling for pre-education entrepreneurial intentions revealed that the hypothesis that entrepreneurship education affects intentions must be rejected. Rather, students with intentions are more likely to self-select into entrepreneurship education suggesting that reversed causal influences take place. In this way, the meta-analysis provided a contribution to the debate about whether or not entrepreneurship education is effective. MARA is increasingly used in meta-analyses in the domain of entrepreneurship. However, it is important to remind that MARA builds on the same assumptions of any regression analysis. For example, the sample size needs to be sufficiently large in relation to the predictor variables (Schmidt, 2017). Considering that the average effect size in management is computed from 18 primary studies (Aguinis et al., 2011), it is clear that most meta-analyses do not have the sample size required to conduct MARA.

MASEM provides a second way for meta-analyses to make a stronger contribution (Viswesvaran & Ones, 1995). MASEM combines meta-analysis with structural equation modeling to test multivariate structural relationships between independent and dependent variables. MASEM requires a correlations matrix that consists of bivariate meta-analyses to use as an input file for a structural equation model. This method is powerful for theory development and theory testing as it allows testing of mediator variables to assess the mechanism by which the independent variables affect the outcomes. Therefore, MASEM can provide the type of contribution that top journals are looking for. Challenges in MASEM include the small number of studies in some subsets of analysis and analysis based on heterogeneous effect sizes, which both reduce the generalization of findings. A good example of a MASEM is a meta-analysis that compares different models of entrepreneurial intentions (Schlaegel & Koenig, 2014). The analysis uses MASEM to compare the theory of planned behavior with the entrepreneurial event model. The results do not only show that the theory of planned behavior explains more variance than the entrepreneurial event model but that an integrated mediation model explains most of the variance in entrepreneurial intentions. The study addressed a lively debate on competing intention models in the domain of entrepreneurship.

In general, meta-analyses in entrepreneurship and management often suffer from major limitations such as effect size heterogeneity and the validity of studies included in the meta-analyses, although many of these issues can be corrected. Essentially, most research in entrepreneurship is not experimental. Therefore, it is often more appropriate to look for variations in outcomes rather than for true effect sizes. Nevertheless, identifying variations in outcomes across a number of studies helps to accumulate knowledge in the domain of entrepreneurship.

Systematic Literature Reviews

Systematic literature reviews use detailed and rigorous methods to summarize results of entrepreneurship studies (Tranfield et al., 2003). Systematic literature reviews are conducted frequently in the entrepreneurship literature. For example, conducting a keyword search in Google Scholar in June 2019 combining the term “systematic literature review” with “entrepreneurship” had more than 9,000 results, with some of the reviews published in top tier management and entrepreneurship journals (e.g., Stephan, 2018; Shepherd et al., 2019). Reviews have become more standardized, transparent, and reproducible, and thus systematic, following the suggestions made by Tranfield et al. (2003). This type of review often highlights gaps in the literature, helps to develop and advance theoretical models, and presents new perspectives on emerging issues leading to valuable suggestions for future research (Denyer & Tranfield, 2006). Some reviews even suggest propositions for future research (Zahra, Sapienza, & Davidsson, 2006). Another advantage is that systematic literature reviews are inclusive as they can combine both quantitative and qualitative studies. These reviews provide important contributions helping to develop theory and research practices in entrepreneurship.

However, systematic literature reviews do face challenges. Ultimately, systematic reviews remain a frequency count similar to many forms of narrative reviews and they often have the same problems associated with narrative reviews including subjective decisions about which studies to include, how to analyze the studies, and how to draw conclusions, the limited information-processing capacities of the reviewers, and limited criteria to help minimize potential biases (Rauch & Frese, 2006). Often, the information remains descriptive. For example, knowing that seven studies examined a multidimensional conceptualization of failure (Cacciotti & Hayton, 2015) provides little information about the validity of such an approach. In addition, frequency counts stress the role of direct relationships between constructs (e.g., Newman, Obschonka, Schwarz, Cohen, & Nielsen, 2019), although we know from other forms of reviews that moderator variables are often present (Rauch & Frese, 2007). Another issue with systematic reviews is that while the literature search and study location process is often systematic, and therefore replicable, the coding and analysis are often less systematic. This provides some flexibility for the reviewers but at the expense of bias and misinterpretation and, therefore, insights can be idiosyncratic. Therefore, the interpretation of some systematic literature reviews requires caution. Nevertheless, many carefully conducted systematic literature reviews have provided valuable contributions to the field and such reviews will continue to do so. A good example for a systematic literature review is provided by Stephan (2018). The review provides an extensive documentation of the coding procedure and it includes coding scheme and a discussion of coding decisions. Thereby, it allows for replication. The review also provides a framework that does not only map previous research but also helps to develop theory and to identify areas for future research.

Bibliometric Analysis

Bibliometric reviews tend to look back and describe what has been done, who has influenced the field, or which networks have been established in the scientific entrepreneurship literature. Bibliometric analysis uses a set of techniques to quantitatively and qualitatively analyze the literature in a field, including citation analysis and content analysis (Garfield & Welljams-Dorof, 1992). Bibliometric analysis is achieving momentum because there are more and more tools available to analyze databases and citations in a quantitative way. The aim is often to look for regularities in a field, for example, by conducting a citation analysis, and to uncover the structure and theoretical foundations of a field, which also helps develop predictions. Information related to the content of the literature can also be examined. Such an overview provided by bibliometric analysis can be valuable particularly if a field is fragmented, involving different and competing conceptualizations, methodologies, and frameworks and when there is a lack of agreement on many key issues.

There are numerous articles in entrepreneurship using bibliometric analysis. For example, Andrade-Valbuena, Merigo-Lindahl, and Olavarrieta (2019) counted 24 published articles in the domain of entrepreneurship using bibliometric analysis, showing that it is a common form of review in the entrepreneurship literature. However, these articles are usually not published in the top journals of the field, possibly because most approaches use the method to describe a specific area in the discipline (Xi, Kraus, Filser, & Kellermanns, 2015) or count publications and citations in the field (Lampe et al., 2019). While it can be insightful to uncover central research topics and theoretical foundations in entrepreneurship, these reviews are descriptive and it is, therefore, difficult for them to contribute to and advance major debates in the field. Yet, it is unclear whether bibliometric analysis is merely a method for categorizing information or if it is useful to explain and predict phenomena. To a certain extent, a theory-driven approach to bibliometric analysis involving hypothesis testing is possible. For example, one bibliometric study presented a model and hypotheses predicting scholarly influence in management (Podsakoff, MacKenzie, Podsakoff, & Bachrach, 2008). In particular, examining structures and the dynamics in the field, examining the content of publications, and establishing correlations with external indications such as the institutional environment might extend the contribution of bibliometric reviews. For example, De Bakker, Groenewegen, and Den Hond (2005) combined bibliometric analysis with various methods to analyze their datasets on corporate social responsibility and compare three contrasting views about the development of the field: progression, variegation, and normativism. The study not only shows how the field proceeded in terms of citations but also how it developed over time.

The Synthesis of Qualitative Research

The synthesis of qualitative research is a research design for synthesizing primary qualitative data from case studies and using tools such as causal network technique, content analysis, and cross-case analysis for aggregating the literature. This synthesis shares some features with the systematic literature reviews discussed earlier, such as being systematic and organized, transparent and explicit, and replicable, and synthesizing the results (Briner & Denyer, 2012). This type of review is becoming more prominent as some researchers are uneasy about the dominance of quantitative reviews, which ignore insights from qualitative research (Hoon, 2013; Rauch et al., 2014). The synthesis of qualitative research is more common in medical research (Paterson, 2012), although there are examples in both management research (Mintzberg, Raisinghani, & Théorêt, 1976; Miller & Friesen, 1977) and entrepreneurship research (Habersang et al., 2019). There are a number of different approaches for synthesizing qualitative research to meet the different aims of the reviews. Some reviews synthesize qualitative studies with the aim to test theories and to generalize the findings of qualitative research (Rauch et al., 2014), while other syntheses of qualitative research are more interpretive and aim to develop new theory (Hoon, 2013).

This type of review can provide a contribution if applied with rigor and care. Most importantly, it can address calls for a process view in entrepreneurship (Baron & Markman, 2004) as well as calls to contextualize the domain (Welter, 2011). Specifically, case studies often provide rich and contextualized information about how and when specific conditions interact with the context and aggregating such information helps develop evidence about such processes in entrepreneurial firms (Habersang et al., 2019). It is difficult to cover these issues in a quantitative review where processes can best be covered by coding time points and where the statistical aggregation either eliminates context conditions or addresses them by coding context at the level of the study or the country. Therefore, the synthesis of qualitative research adds to the generation of knowledge in the field. Qualitative research often addresses issues that cannot easily be studied in quantitative research. For example, it is difficult to sample failed firms as many of them simply disappear from data files. But there is a rich body of case descriptions of failed firms. One study synthesized 43 published case studies on organizational failure (Habersang et al., 2019). The well-documented analysis used a combination of inductive and deductive steps to analyze the cases, starting with within-case analysis and aggregating these results in a cross-case analysis. The results revealed that the interplay of distinct rigidity and conflict mechanism explains different pathways of organizational failure.

The quality of the synthesis of qualitative studies is dependent on the skills of the coders and might be more challenging than in quantitative reviews or in bibliometric reviews where software may perform the coding. Interpretive syntheses are often difficult to replicate as they involve a number of decisions and explanations available for phenomena appearing in qualitative case studies. Both types of synthesis of qualitative research require comparative studies.

Historiometric Analysis

Historiometric analysis is a unique approach as it does not aim to analyze the scientific literature but rather it analyzes historical information, which may be extracted from material such as biographies, autobiographies, diaries, oral histories, obituaries, letters, or life stories (Denzin, 1989). Since this approach does analyze primary data collected by other people, such as a biographer, this approach can be classified as a review. Although this method is suitable for a number of different research questions, historiometric analysis of biographies is specifically well suited to address three issues. First, similar to the synthesis of qualitative research, historiometric analysis can address process issues. This method is well suited to look specifically at long-term processes. For instance, biographies enable reviewers to analyze how the life of a prospective entrepreneur unfolds as the combination of trajectories and transitions initiating turning points in life and redirecting paths. Second, historiometric analysis measures the context and situation-specific aspects of entrepreneurship. Third, it can be used to analyze unique or rare samples. Entrepreneurship provides a number of examples of unique entrepreneurs and their unique activities, and exceptional samples cannot be studied by relying on representative samples. For example, it is a stylized fact that only a small percentage of firms are very successful and research has tried to identify and study these firms. However, since the number of these firms is small, it is difficult to identify them by relying on traditional sampling procedures. Finally, as all these issues are not easily addressed with other forms of review, historiometric analysis provides opportunities to make substantial contributions to the field. However, while this method has been used, for example, in organizational behavior (Bass, Avolio, & Goodheim, 1987; Simonton, 1977), it has, to my knowledge, only occasionally been used in entrepreneurship, such as analyzing the hotel industry (Morrison, 2001; Nickson, 1997), single entrepreneurs (Reveley, 2010), or multiple entrepreneurs (Duchek, 2018; Schoenberger, 2001; Smeth, 2005; Villette & Vuillermot, 2009). For example, resilience, the ability to go on with life, or to continue living a purposeful life after hardship or adversity bringing the organization back to normal functioning (Tedeschi & Calhoun, 2004) develops over time depending on prior experiences, and individual and situational factors. A historiometric analysis, such as the one provided by Duchek (2018), is well suited to cover such long term processes.

To synthesize historiometric information, researchers have used deductive as well as inductive methods for the review. Content analysis is a deductive way of analyzing historiometric information (Crayne & Hunter, 2017). An advantage is that the information extracted can be quantified and quantitative methods for data analysis can be applied. Quality criteria such as validity and (inter-rater) reliability can be used to evaluate the quality of the analysis. However, it is also possible to analyze the research material in an inductive way, using techniques such as grounded theory, meta-synthesis, and meta-ethnography, all aiming to make contributions beyond those achieved in the original studies (Hoon, 2013, p. 527). A problem with inductive reviews is that the coding relies on reviewer skills, and reviewers’ interpretations and explanations make replication difficult.

The method is not without problems as the historiometric database moves often between fact and fiction, CV and narrative, and fact-based reconstruction and imaginative construction (Denzin, 1989). Beside these problems with the material, the iterative process of research refinement and interpretation has challenges and biases (Crayne & Hunter, 2017). Therefore, the analysis depends heavily on both the reviewer and the data sources themselves. The data sources might contain information gaps or errors and are often based on conventions of how historical material is presented. Therefore, establishing the validity of inferences becomes a critical exercise in the analysis. Historical data are still correlational and, thus, the internal validity might be low. For example, the relationships reported might be affected by omitted variables and, therefore, any causal interpretation should be conducted with extreme care. Thus, the analyzer must be well aware of these issues and take care to avoid them entirely (Crayne & Hunter, 2017, p. 26). For a top-tier publication it is imperative to implement multiple measures to reduce potential overinterpretation of historiometric data (Crayne & Hunter, 2017).

Future Trends in Knowledge Accumulation

There are additional forms of knowledge accumulation specifically promising for the field of entrepreneurship. Some have been used in specific contexts, but have not been systematically developed as a specific form of the review. In the field of entrepreneurship, it is particularly worthwhile to consider two specific trends: the use of large datasets and the use of computerized (text) analysis.

Datasets play an important role in the entrepreneurship literature. Some datasets have been developed specifically in the context of entrepreneurship at a global scale (e.g., Panel Study of Entrepreneurial Dynamics, Global Entrepreneurship Monitor [GEM]) or on entrepreneurship dynamics within specific countries (e.g., Kauffman Firm Survey, Comprehensive Australian Study of Entrepreneurial Emergence). Other databases that have not been developed specifically for the context of entrepreneurship have been used to test entrepreneurship hypotheses (e.g., Organization for Economic Co-operation and Development Employment Outlook, European Community Household Panel). These databases provide unique opportunities for reviews in the entrepreneurship literature. First, reviews can summarize the results of studies using one specific dataset. An example is the study by Bosma (2013), who reviewed academic papers published between 2004 and 2011 that used GEM data. Such analysis helps identify some stylized facts, such that early-stage entrepreneurial activity varies across countries, that there is a U-shaped relationship between entrepreneurship activity and economic growth, and that a considerable proportion of entrepreneurial activity is performed out of necessity, particularly in less developed economies. Second, and alternatively, one can review results from multiple different datasets to address one specific research problem. For example, researchers have used different datasets to study the relationship between entrepreneurship and earnings. Åstebro and Chen (2014) compared results across 24 commonly used databases. They found that 12 databases supported the proposition that entrepreneurs earn less than employees and that 12 other databases did not find support for this proposition. These differences are surprising because many of these databases are representative of the populations under investigation and, therefore, should lead to comparable results. Obviously, comparing such databases indicates that there are complexities involved in studying entrepreneurial rewards (Carter, 2011, p. 40) and these complexities involve methodological issues (Carter, 2011) as well as theoretical moderators (such as income underreporting by entrepreneurs) affecting the results (Åstebro & Chen, 2014). Accordingly, this type of review helps to address lively debates in the field of entrepreneurship. Finally, databases not related to entrepreneurship can often provide relevant information. For example, Stephan (2018) reviewed the literature on entrepreneurs’ mental health and well-being and many studies included in the review were panel studies that did not sample entrepreneurs but representative samples of the population, allowing Stephan to study a phenomenon that is highly relevant for entrepreneurship. This type of review can be useful as it is possible to address issues that are not commonly addressed in entrepreneurship research, such as drug abuse or even suicide (e.g., Heinz, Freeman, Harpaz-Rotem, & Pietrzak, 2017). Thus, reviewing databases can provide significant contributions to the field by providing stylized facts, resolving debates, and addressing new research questions. Notably, analyzing databases involves the risk that reported results might not be independent if they appear in different publications relying on the same database, a problem that needs to be adjusted for.

Computerized analysis is another trend that uses predominantly text analysis tools based in machine learning to articulate meaning embedded in the text. Since such analytical tools can test a large amount of data, they provide new opportunities to review text sources that could not have been analyzed with more traditional text analytical tools. For example, in entrepreneurship, researchers have used computerized text analysis to analyze shareholder letters of firms (Short, Broberg, Cogliser, & Brigham, 2010), press releases (von Bloh, Broekel, Özgun, & Sternberg, 2019), and crowdfunding campaigns (Kaminski & Hopp, 2019). Computerized tools offer several advantages compared to more traditional forms of review. First, they help to identify relevant data, which is important given the large and growing number of scientific publications, and there are several automated processes available to locate data (O'Mara-Eves, Thomas, McNaught, Miwa, & Ananiadou, 2015). Second, computerized analysis helps analyze new data sources that were difficult to access because of the sheer amount of data involved, such as the data information repository made available by the internet or analysis that relies on multiple types of data, such as texts, videos, or verbal information (Kaminski & Hopp, 2019). Finally, there are different sets of rules—algorithms—available to analyze the data, ranging from basic methods that look, for example, at frequencies to more complex models such as natural language processing, which analyzes multiple word phrases and enables researchers to capture the syntactic relations that bind words to produce meaning (Pandey & Pandey, 2019), or sentiment analysis that helps understand an opinion about a given subject from written or spoken language. Computerized text analysis provides opportunities to arrive at new interpretations and assigned meanings. One important decision in the analysis refers to the choice of using supervised versus unsupervised machine learning to analyze the data. Supervised methods allow the researcher to construct a classification scheme first, whereas unsupervised methods read all the texts first and then suggest which classification scheme(s) might apply. In the domain of entrepreneurship, this type of research is just evolving and has potential to contribute to knowledge accumulation in the field as it can process huge amounts of data including data sources that might have been untapped, and it can assign meanings to the data that can only be extracted with computerized text analysis. An example of applying computerized analysis is Kaminski and Hopp (2019). Using computerized text analysis, the authors analyzed 20,188 crowdfunding campaigns. The study used two algorithms—neural network and natural language processing—to analyze text, speech, and video material. The results indicated that positive language and showcasing the product were most predictive for receiving funding. Like all methods, computerized text analysis also has disadvantages. Using big data creates ethical issues about the access to such data. It is also difficult to define the boundaries of a research question and thus to define the corpus of text documents that needs to be analyzed. While this is always an issue with text analysis, in computerized text analysis it is unclear what the requirements are so that the method works well. Finally, there are numerous algorithms available for the actual data processing and analysis; each of them has uncertainties regarding reliability and how different algorithms affect the outcomes. Finally, computerized text analysis involves many subjective decisions and, thus, is often less objective than assumed.

In summary, this editorial shows that there are a number of different forms of knowledge accumulation. Some of them are well established and some more recent forms are developing as there is no agreed upon methodology. While a researcher may have personal preferences for one or another form of review, this does not mean that one form is better than others. Rather, the value of a certain form of review depends on the goals the review wants to achieve, whether generalization, theory testing, theory development, or mapping the field. A diversity of approaches for a review is required and helps accumulate the diversity of knowledge typically found in entrepreneurship research.

Some methods for the review help establish actionable design templates to combine the scientific evidence with practice recommendations (Tranfield et al., 2003), helping to achieve what Strokes (1997) called use-inspired research: combining scientific rigor and practical relevance. Thus, the accumulation of knowledge in the field of entrepreneurship is important for both the theoretical advancement of the field and for developing practice implications.

Concluding Remarks

Reviews can provide important contributions for entrepreneurship and are, therefore, attractive to many top-tier journals in the field such as Entrepreneurship Theory and Practice. At the same time, editors receive too many poorly conducted reviews, with many not accepted. Many reviews face a dilemma between summarizing what has been done in past research and contributing to advancing the field of research. This editorial shows how reviews can go beyond purely descriptive aims and discusses how reviews can provide a substantial contribution to the entrepreneurship literature. The contribution or worth of a review depends on several issues: the quality of the material being reviewed, the type of review conducted, the rigor with which the review is conducted, and the reviewers’ skills and competencies.

The five types of reviews discussed here—meta-analysis, systematic literature review, bibliometric review, the synthesis of qualitative research, and historiometric analysis—reflect those that are most popular in the management literature, but this list is certainly not exhaustive. There are many different forms for a review serving different aims, with some even combining qualitative and quantitative research (Dixon-Woods, Agarwal, Young, Jones, & Sutton, 2004; Paterson, 2012). There are also new developments in the field involving large datasets and computerized (text) analysis that provide new opportunities to conduct reviews in the domain. Selecting an appropriate method for a review depends on the aims of the researcher, but these aims also need to meet the aims of the journal the review is submitted to. The intention is to inspire creativity in reviews, not to discourage the submission of reviews. Reviews can be powerful and can produce novel insights into phenomena in the domain of entrepreneurship, helping achieve the mission of Entrepreneurship Theory and Practice. I hope this editorial provides clarity for authors aiming to contribute to the field of entrepreneurship by conducting a literature review.