Examining Psychological Mediators in Entrepreneurship: Experimental Designs,Remedies,and Recommendations

Abstract

Psychological mediators underlie many entrepreneurship phenomena. Unfolding psychological mechanisms enhances our understanding of theoretical relationships in entrepreneurship. This paper first reviews the current state of entrepreneurship studies examining psychological mediators and identifies the hurdles that push researchers away from employing randomized experiments to unfold the causal relationships underlying mediation. To alleviate these hurdles, we then propose parsimonious yet rigorous experimental designs that make experiments testing psychological mediators in entrepreneurship feasible and cost efficient. In addition, when manipulating the mediator is not feasible, we theorize and identify two remedies a single experiment can use to examine the causal chain underlying mediation.

Keywords

experimental methods/simulation research methods manuscripts: specialty areas entrepreneurship psychology manuscripts: general areas mediation

Introduction

Psychological factors predict many entrepreneurial outcomes and play an important role in the entrepreneurial process (Frese & Gielnik, 2014). The cognitions, regulatory processes, and motivations of entrepreneurs often serve as proximal factors to starting a business or to business success (Baum et al., 2014). In addition to being predictive, such psychological factors are also theorized as essential mediating mechanisms that underlie a causal relationship (Stone-Romero & Rosopa, 2011) between other important, focal variables in entrepreneurship (cf. Gielnik et al., 2020). For example, it is argued that entrepreneurial self-efficacy causes entrepreneurial passion as a potential mediator, which in turn predicts persistence with the business (Cardon & Kirk, 2015). While this mediation relationship is theoretically sound, it is also likely that the person’s passion increases self-efficacy, which leads to persistence. Empirically verifying the cause and effect between the independent variable, the psychological mediator, and the dependent variable is difficult in entrepreneurship—especially through oft-employed correlational research methods. As such, empirical work on the psychological mechanisms in entrepreneurship often acknowledges reverse causality and alternative explanations as the study limitations (e.g., Bischoff et al., 2020). The same theoretical and empirical challenge is true for other theoretical relationships in entrepreneurship (e.g., Stenholm & Renko, 2016). In this respect, whether the theorized mediators that are invoked in entrepreneurship research indeed mediate these focal relationships remains unknown.

Disentangling the cause and effect in mediation empirically can contribute to both theory building and theory testing. As Colquitt and Zapata-Phelan (2007, p. 1284) argued, empirical tests on mediation “involve adding a new ‘what’ (i.e., a construct or variable) to an existing theory in order to describe ‘how’ a relationship or process unfolds.”Anderson et al. (2019, p. 3) likewise argued that “the next frontier in theory testing research involves improving our ability to make causal predictions about entrepreneurship phenomena.” For instance, if a methodologically rigorous study examines a theoretical mediator but finds no mediation effects, such results may signal an alternative mechanism that has not been theorized or inform a boundary condition of the theory that the theoretical mediation does not function in certain circumstances. Such research also has important implications for practice.

And while current entrepreneurship research has made great strides theoretically and empirically (Maula & Stam, 2020), the current state of mediation research in entrepreneurship is overly reliant on statistical analysis. While it is a valuable tool, statistical mediation analysis is performed post hoc and requires the researchers to make several assumptions, such as the independent variable, the mediator, and the outcome occurring in sequence (see Imai et al. (2011) for a complete discussion). However, the reliance on assumptions about what might be the case in terms of relationships between psychological factors and other antecedents and outcomes in entrepreneurship does not in itself provide empirical evidence of mediation. That is, we ought not to use assumptions about theoretical relationships to form the basis of tests regarding the empirical veracity of those theoretical relationships (Bullock et al., 2010). Therefore, this paper addresses causality in mediation in the design of studies to test the role of psychological factors in entrepreneurship and does so in the hope of enhancing the researchers’ confidence in providing empirical evidence of mediation in causal inference.

As has been written about extensively, randomized control experiments are the gold standard of causality (Hsu et al., 2017; Shadish et al., 2002; Williams et al., 2019). However, even some existing experimental research in entrepreneurship does not warrant strong claims regarding causality. Burnette et al. (2020), for example, examined the mediation effect of entrepreneurial self-efficacy on the relationship between the students’ growth mindset and interest in the entrepreneurship career. In a randomized experiment, they manipulated the growth mindset and measured self-efficacy and career interest. We salute these authors for their use of experiments in a great research paper. However, since both the mediator and the dependent variable were measured, the causality and therefore the causal nature of the mediation effect remains unverified despite the fact that the authors rightly justified the mediation model with relevant theory. This is a limitation in the paper. And while the authors diligently sought to address issues of causality through robustness tests, the limitation nonetheless remains—even if not explicitly acknowledged.

Indeed, as part of developing our ideas, in this paper we review research examining psychological mediators in entrepreneurship and find that only 35% or so of the papers explicitly list causality as a study limitation. For nearly all of the other 65% of papers, this limitation around causality remains unacknowledged. While these papers should be applauded for doing something to understand psychological mediators, we argue that such limitations around causality should be acknowledged and that pathways to addressing this should be a focus of research in entrepreneurship. Of course, we note that this problem is not unique in entrepreneurship and also presents in other fields, such as psychology, which has given more detailed and rigorous attention to understanding the role of mediation analysis in experimental research. Editors of psychology journals have thus called for more rigorous research examining the complete causal chain underlying mediation (see Cialdini, 2009; Pirlott & MacKinnon, 2016; Smith, 2012 for more detail). In the field of entrepreneurship, which presents similar challenges to the field of psychology, efforts made to improve examinations of causality in mediation studies should also be encouraged and appreciated.

As we review entrepreneurship studies on mediation in this paper, we note that limitations on claims around causality continue to exist for some key practical reasons in entrepreneurship. We identify two of these reasons as the confounding issue (i.e., the manipulation of the independent variable confounds the manipulation of the mediator) and the complicating issue (i.e., the manipulation of the mediator complicates the proposed theoretical model) that make using experiments to examine mediation in entrepreneurship difficult, particularly when the mediator is a psychological factor.

In this paper, we thus focus on how to design experiments to examine psychological mediators in entrepreneurship. While two rigorous designs of experiments have been suggested for empirically examining mediation, we discuss each design and identify the confounding and complicating issues that arise when using these experimental designs to examine a psychological mediator. Specifically, we seek to address the challenges of demonstrating causality in mediation involving psychological factors in entrepreneurship research. As part of the effort in doing so, we review entrepreneurship research that is focused upon psychological factors as mediators and identify the potential methodological limitations that might constrain its theoretical contribution. We then review the existing experimental designs that can be used to test mediation (Pirlott & MacKinnon, 2016; Spencer et al., 2005; Stone-Romero & Rosopa, 2008, 2011) and elaborate on why they may produce the confounding and the complicating issues. We then propose several fractional factorial designs for use in entrepreneurship research that can also enable scholars to investigate the causal paths via a psychological mediator in the hopes of making empirical studies on psychological mechanisms in entrepreneurship more accurate, rigorous, and accessible. Furthermore, we offer two additional remedies for researchers to enhance confidence in making causal inferences, especially for situations where the causality between the mediator and the outcome is questionable.

This paper makes several contributions to entrepreneurship. First, we contribute by cautioning entrepreneurship scholars about the causal chain underlying mediation and the pitfalls of the correlational nature of mediation analyses. Through a review and identification of more than 200 entrepreneurship papers testing mediation in nine management and entrepreneurship journals from 2010 to 2022, we find 119 papers that theorized mediation of a psychological variable. Only 19 of these papers used randomized experiments to investigate the causality of a psychological mediator. We analyze the experimental design of these papers, review the existing recommendations, and identify the confounding and complicating issues that explain why it is challenging to examine the complete causal paths via a psychological mediator. This enables us to shed further light on the methodological limitations that hinder theoretical advancement in explaining the psychological mechanisms underlying many known relationships in entrepreneurship.

Second, we contribute by proposing a novel design option for experiments in entrepreneurship. We articulate how these alleviate the aforementioned issues and make the experiments more accessible to entrepreneurship researchers who study psychological mediators. Additionally, our proposed designs require fewer experimental groups than the designs previously proposed. This can help make experimental work in entrepreneurship more feasible. In this respect, our work is helpful to broader efforts to enable experimental methods to be utilized in entrepreneurship (Williams et al., 2019) as well as in other fields, especially considering the efforts and costs associated with data collection and random assignment in experiments (Short et al., 2010).

Third, in the case where it is unethical or infeasible to manipulate a psychological mediator, we contribute by proposing two remedies that can strengthen the researchers’ confidence in making causal inferences for a mediation effect. This approach can thereby enable entrepreneurship researchers to respond to the call from Low and MacMillan (1988, p. 155) “to pursue causality more aggressively.”. This is especially important because of the role that mediation plays in explaining the causal mechanisms in entrepreneurship.

In what follows, we first visit some key concepts in mediation (Hayes & Preacher, 2014; MacKinnon et al., 2012; Spencer et al., 2005; e.g., Stone-Romero & Rosopa, 2011; Zhao et al., 2010) to establish a general foundation for our theorizing. We then review the existing studies examining mediation in the field of entrepreneurship. We further propose several design choices for experiments and provide a set of recommendations for undertaking these experiments.

The Concept of Mediation

To begin, a mediator exists as a variable that is both caused by an antecedent variable and affects a consequent dependent variable. In this way, a psychological mediator represents an underlying psychological mechanism between focal variables (MacKinnon et al., 2012). In having this effect, a psychological mediator will fully or partially transmit the effect of an independent variable to a dependent variable (Aguinis et al., 2017; Stone-Romero & Rosopa, 2008). We provide a brief example of mediation to illustrate the concept of mediation.

Let us consider the basic mediation model of “passion contagion” where the entrepreneur’s displayed passion (X) affects an employee’s passion (M), which in turn enhances that employee’s affective commitment to the business (Y) (Hubner et al., 2020). In this model, the employee’s passion is the psychological mediator. Figure 1a shows the total effect (path c′) of X on Y. Figure 1b shows partial mediation where we see both the indirect effect from X to M to Y (paths a and b) and the direct effect of X on Y (path c). The total effect (c′) equals the direct effect (c) plus the indirect effect (a × b). When there is no direct effect (c equals zero), the total effect comes purely from the indirect effect and the model becomes full mediation (Figure 1c).¹ Full mediation occurs when the effect of X on Y becomes non-significant in the model where the mediator is presented. This is opposite to the effect caused by a suppressor. A suppression effect occurs when the effect of X becomes larger or more significant when a suppressor is presented in the model (Cheung & Lau, 2007).

Figure 1.

(a) Illustration of total effect, (b) illustration of partial mediation effect, (c) illustration of full mediation effect.

What we hope to emphasize is that there are at least two causal paths that must be verified for mediation: (a) X leads to M (path a) and (b) M leads to Y (path b) (Stone-Romero & Rosopa, 2011). As we noted earlier, examining the causality in the relationships that underlie mediation (X causes M and M causes Y) is difficult, particularly when M is a psychological factor, and when the manipulation of M would lead to the aforementioned confounding and complicating issues.

Existing Entrepreneurship Studies Testing Psychological Mediators

To develop an understanding of how the current state of empirical research on psychological mediators in entrepreneurship deals with sequential causality underlying mediation, we examined over 10 years of research (2010–2022) in seven entrepreneurship journals and two management journals² that publish entrepreneurship papers examining mediation. The keywords that we utilized in our search include “mediate,”“mediator,”“mediation,” and “mediating.”

We then examined each article to determine whether: (a) it was an entrepreneurship paper, (b) it hypothesized and tested a mediating effect, and/or (c) the mediator was a psychological factor. The papers that did not meet those three requirements were removed from consideration. The search returned 242 papers, of which about 123 papers examined a mediator that was not a psychological factor and thus were excluded from our review. Consequently, 119 papers were retained and reviewed for whether they used experimental methods and how mediation of a psychological variable was tested. This evidence supports our argument that psychological factors as the underlying mechanism/mediator in theory are common and important. Indeed, these papers have over 13,000 citations (see Table 1). In this respect, the impact of these studies in the literature is substantial.

Table 1.

Entrepreneurship Papers Examining Psychological Mediators Between 2010 and 2022.

Area of mediators	Examples	Number of papers*	Total number of citations**	Number of citations since 2016
Cognition	Self-efficacy, intention, cognitive creativity, perceptions, etc.	95	11,004	9,751
Emotion	Moods, psychological distress, frustration, etc.	11	357	357
Wellbeing	Work–family balance, job satisfaction, burnout, etc.	13	1,921	1,495

The complete list of the papers is available upon request.

The citation numbers were updated in August 2022.

In examining these papers, we found that a vast majority of them adopted a correlational method (e.g., primary survey or secondary data) to test a psychological mediator, and in doing so, acknowledged their inability to rule out alternative causal explanations in the mediation effect. Only 19 papers (16%) employed experiments to examine the psychological factors as mediators (see Table 2).³ This outcome is not a surprise to us. While experiments in entrepreneurship research have been increasing in recent years, the use of experiments remains rare as a result of challenges to their implementation (Williams et al., 2019). What this means is that a substantial portion of research on the psychology of entrepreneurship cannot empirically speak to the causal aspects of mediation. Given the impact of such research (over 13,000 citations), we see this as an important challenge to address in the field.

Table 2.

Entrepreneurship Papers Using Experiments to Examine Psychological Mediators Between 2010 and 2022.

Authors	Sample	Methods	Theorized relationships	Manipulation of the independent variable	Psychological mediator
1. Allison et al. (2022)	Study 1: 320 participants recruited from Amazon Mechanical Turk	Randomized experiment	Perceived preparedness and passion mediate the relationship between valence-arousal congruence and funding performance.	Valence-arousal congruence conditions	Measured perceived preparedness and passion
2. Burnette et al. (2020)	A sample of 238 undergraduate students at a large public university	Randomized experiment	Entrepreneurial self-efficacy mediates the relationship between growth mindset and entrepreneurship career interest.	Growth-mindset intervention	Measured entrepreneurial self-efficacy
3. Cornelis et al. (2022)	A sample of 514 participants recruited from Amazon Mechanical Turk	Randomized experiment	Product and founder perceptions mediate the relationship between crisis response strategy and funding intentions.	Crisis response strategy	Measured product and founder perceptions
4. Engel et al. (2020)	Study 1: 81 entrepreneurs from three startup hubs in the Netherlands	Randomized experiment for the IV–DV relationship	Self-compassion mediates the relationship between loving-kindness meditation and sustainable decision-making.	Loving-kindness meditation	Measured self-compassion
4. Engel et al. (2020)	Study 2: 114 entrepreneurs recruited from Prolific	Randomized experiment for the IV–Mediator–DV relationship		Loving-kindness meditation	Measured self-compassion
5. Engel et al. (2021)	A sample of 87 entrepreneurs recruited through social media groups of entrepreneurs	Randomized experiment	Self-compassion mediates the relationship between loving-kindness meditation and fear of failure.	Loving-kindness meditation	Measured self-compassion
6. Gielnik et al. (2017)	A sample of 227 students (784 observations) from a University in Nairobi, Kenya	Randomized experiment	The mediation causal chain of entrepreneurship training—entrepreneurial self-efficacy—passion—business creation	Entrepreneurship training	Measured entrepreneurial self-efficacy, measured passion
7. Gish et al. (2019)	Study 2: a sample of 101 small business owners from around the world	Randomized within-subject experiment for the Mediator–DV relationship	Participants’ attention to structural alignment mediates the effect of sleep restriction on the formation of third-person opportunities.	Sleep restriction	Measured cognitive ability to attend to structural similarity
7. Gish et al. (2019)	Study 3: a sample of 89 upper-level business school students from a large research university from the United States	Randomized between-subjects experiment for the IV–Mediator–DV relationship		Sleep restriction
8. Hubner et al. (2020)	Study 2: a sample of 321 individuals recruited from Amazon Mechanical Turk	Randomized experiment	The employee passion response mediates the effect of the entrepreneur’s passion experience on the employee’s affective commitment.	The display of the entrepreneur’s passion	Measured passion of the employees
9. Ivanova et al. (2018)	A sample of 258 entrepreneurship students from a Dutch university	Randomized experiment	Controllability appraisal mediates the effects of emotions on opportunity beliefs.	Induced emotions	Measured controllability appraisal
10. Johnson et al. (2018)	Study 2: a sample of 73 amateur investors in the eastern region of the United States	Randomized experiment	The investor perceptions of the entrepreneur trustworthiness and competency mediate the effect of the entrepreneur gender on the investor funding decision.	Gender of the pitching entrepreneur	Measured perceptions of trustworthiness and competency of the entrepreneur
11. Kollmann et al. (2017)	Study 2: 204 real nascent entrepreneurs identified via a well-established and representative German online panel	Randomized experiment for the IV–Mediator–DV relationship	Fear of failure mediates the effects of obstacles on opportunity evaluation.	Vignettes to manipulate perceived obstacles	Measured fear of failure
11. Kollmann et al. (2017)	Study 3: 355 real nascent entrepreneurs identified via a well-established and representative German online panel	Randomized experiment for the IV–Mediator–DV relationship		Vignettes to manipulate perceived obstacles	Measured fear of failure
12. Lerner (2016)	A sample of 134 upper-level business students enrolled in a large marketing course required for all business majors	Randomized experiment	Perceived founder quality mediates the effect of founder disinhibition on perceived success likelihood.	Vignettes to manipulate founder disinhibition	Measured perception of founder quality
13. Mahmood et al. (2019)	Study 3: a sample of 200 individuals recruited on Amazon Mechanical Turk	Randomized experiment	Crowdfunders’ perception of innovativeness mediates the effects of logos on crowdfunders’ investment decisions.	Logos	Measured crowdfunders’ perception of innovativeness
14. Rose et al. (2020)	Study 1a: a sample of 103 individuals recruited through Amazon Mechanical Turk Study 1b: a sample of 101 individuals recruited through Amazon Mechanical Turk	Randomized experiment for the IV–Mediator–DV relationship Randomized experiment for the IV–Mediator–DV relationship	Crowdfunders’ psychological distance mediates the effects of product development stage and time to delivery on crowdfunders’ willingness to support.	Product development stage, time to delivery	Measured psychological distance
15. Schellong et al. (2019)	A sample of 166 adult consumers recruited from Germany	Randomized experiment	Consumer perception of a firm as doing good mediates the effect of the firm brand on consumer happiness.	Vignettes to manipulate the firm brand	Measured perception of doing good
16. Shane et al. (2020)	A sample of 19 subjects	Randomized experiment	Investor neural engagement during the exposure to a pitch mediates the effect of displayed entrepreneurial passion on investor interest.	Pitch videos to manipulate displayed passion	Measured neural engagement
17. Stevenson et al. (2022)	Study 1: 125 entrepreneurs resulting in 375 persistence decisions	Randomized experiment	Affective activation and cognition-based action intentions mediate the relationship between positive market validation information and persistence.	Market and expert validation	Measured affective activation and cognition-based action intentions
18. Souitaris et al. (2020)	Second experiment: 190 MBA students	Randomized experiment	Frustration mediates the effect of a founder’s power on the likelihood of fully exiting initial public offerings.	Vignettes to manipulate the founder power	Measured frustration
19. Wieland et al. (2019)	805 participants recruited from Amazon’s Mechanical Turk	Randomized experiment	Perceived risk, anticipated social resources, and self-efficacy mediate the effect of venture-gender congruency on venture desirability.	Vignettes to manipulate venture-gender congruency	The three mediators were measured

Note. Only experimental studies testing the hypothesized psychological mediators are listed.

Moreover, among the 19 papers that utilized experiments, 15 conducted only one experiment that randomly manipulated the independent variable yet measured the mediator and the dependent variable. Spencer et al. (2005) and Pirlott and MacKinnon (2016) called such an experiment the measurement-of-mediator experiment (MME). Because establishing evidence for mediation also implies the establishment of a causal sequence that contains two or more causal paths (Judd & Kenny, 2010; MacKinnon et al., 2012; Stone-Romero & Rosopa, 2008), a criticism of the measurement-of-mediator design is that this design is not necessarily able to verify the causal relationship between the mediator and the dependent variable. Caution must thus be taken when making causal inferences in studies using a single MME. Later in this paper, we will discuss two remedies that researchers employing a single MME may use to enhance confidence in causal inferences for the mediating effect.

Furthermore, we noted that only one paper by Gish et al. (2019) employed multiple randomized experiments (an MME and another randomized experiment to manipulate the mediator) that were able to verify the complete causal chain underlying the mediation effect. Our review reveals that verifying the complete causal chain underlying psychological mechanisms as the mediation effect in entrepreneurship is rare. Even though two rigorous designs of experiments for mediation tests have been proposed (MacKinnon et al., 2012; Stone-Romero & Rosopa, 2008): the Two Independent Randomized Experiments Design and the Full Factorial Design, these designs are generally underutilized by entrepreneurship researchers. We believe that this might be in part due to a lack of understanding regarding why these may be necessary and a lack of clear guidance for how these can apply in psychological work in entrepreneurship research. In the next section, we thus examine these two experimental designs that exist to test for mediation and discuss the strengths of each design, the weaknesses, and the methodological limitations for examining psychological mediators in entrepreneurship. We then propose several more parsimonious designs—the fractional factorial designs—that are ideally suited for entrepreneurship research where access to samples of entrepreneurs is a substantial challenge (e.g., Short et al., 2010).

Design Choices for Experiments to Test Mediation

Use of Two Independent Randomized Experiments Design

In this section, we highlight two rigorous experimental designs that can be used to test mediation in entrepreneurship. In the first, the Two Independent Randomized Experiments Design, two separate experiments are used to examine the two causal paths independently (Stone-Romero & Rosopa, 2008). This design has been proposed (Stone-Romero & Rosopa, 2008, 2011) to test the relationships between the independent variable (X), the mediator (M), and the dependent variable (Y). The first experiment manipulates X and measures or observes M (M_obs) and Y. The second experiment manipulates M_man and measures or observes Y. This design is conceptualized in Table 3. The results of the two experiments provide a basis to examine the causal effects of:

X on M_obs (first experiment)

X on Y (first experiment)

M_man on Y (second experiment)

Table 3.

Two Independent Randomized Experiments Design.

First experiment	R	X	M _obs	Y
Second experiment	R		M _man	Y

Note. R = random assignment; X = the independent variable manipulated; M_obs = the mediator measured or observed; M_man = the mediator manipulated; Y = the dependent variable.

The strength of this design is that it provides a clear independent examination of the causal chain of X → M → Y. Despite several strengths of this approach, there are serious caveats for using this design to examine a psychological mediator. Specifically, there are a number of shortcomings that should be considered when deciding to use this design in entrepreneurship research. First, conducting two independent experiments influences the feasibility of the research project (not only in terms of the costs of the research project but also in terms of the time required for participants) because there are two experiments instead of one. This may be compounded when multiple experiments are already required before considerations of the causal chain underlying mediation. Second, as the two causal paths are examined in different experiments with different samples, the strength of the mediation effect cannot be examined directly (Stone-Romero & Rosopa, 2011). Third, and most importantly, this design becomes problematic when the mediator is a psychological factor, which cannot be manipulated directly (Bullock et al., 2010).

This introduces the complicating issue that must be addressed. Take again, for example, Hubner et al.’s (2020) model of passion contagion for example. Hubner et al. (2020) used a single MME along with a cross-sectional survey to examine their mediation model. If we were to use the design of the two independent experiments, the additional experiment would require the manipulation of the psychological mediator, the employee’s passion. While this may be doable by having the employee receive entrepreneurship training (Gielnik et al., 2017) to enhance their passion, such manipulation would also affect other psychological factors, such as the entrepreneur’s entrepreneurial self-efficacy (Gielnik et al., 2017). Consequently, the model would become much more complicated than originally proposed. Manipulating the employee’s passion via any way other than the entrepreneur’s displayed passion would “muddy the waters.” Thus, the complicating issue arises when manipulating the psychological mediator through a variable that is not theorized and included in the researchers’ original model. This is because there may be no direct manipulation of a psychological variable/mediator. Most of the time, only indirect manipulations are available for a psychological variable and therefore may inevitably cause a variable that is not in the original model.

In our review of 19 experimental papers examining psychological factors as the mediator, only one study adopted the two independent experiment design. Gish et al. (2019) examined the sleep restriction (X)—cognitive attention (M)—opportunity evaluation (Y) relationship and conducted a correlational survey, an MME, and the additional experiments that manipulated cognitive attention (M) only and measured Y. In the additional experiment, they found a clever way to manipulate cognitive attention by providing information in different business scenarios. Given the limitations of the two independent randomized experiment design, the full factorial experiment design has also been proposed (Pirlott & MacKinnon, 2016) as a way to examine mediation effects. It integrates two experiments into one to manipulate X and M concurrently and adds additional control groups that measure or observe M.

Full Factorial Design

In the second rigorous experimental design that can be used to test mediation in entrepreneurship, the full factorial design (MacKinnon et al., 2012; Pirlott & MacKinnon, 2016), X and M are manipulated concurrently with two additional groups included as the control, where M is measured rather than manipulated. This design can be conceptualized as a 2 (X: high and low) × 3 (M: high, low, and varying freely) between-subjects design, which includes all the possible combinations of different levels of X and M. The full factorial design can be conceptualized in Table 4. Such a design allows researchers to examine the causal effects of:

M_man on Y (e.g., Groups A vs. C or B vs. D)

X on M_obs (Groups E vs. F)

X on Y (e.g., Groups A vs. D, B vs. C, or E vs. F).

Table 4.

Full Factorial Design.

Group A	R	X _h	M _man-h	Y
Group B	R	X _l	M _man-l	Y
Group C	R	X _h	M _man-l	Y
Group D	R	X _l	M _man-h	Y
Group E	R	X _h	M _obs	Y
Group F	R	X _l	M _obs	Y

Note. R = random assignment; X_h = X manipulated at the high level; X_l = X at the low level; M_man-h = M manipulated at the high level; M_man-l = M at the low level; M_obs = M varies freely and is observed; Y = the dependent variable.

In our review, none of the entrepreneurship studies examining a psychological mediator utilized the full factorial design. However, we acknowledge an entrepreneurship paper examining a non-psychological mediator with a similar design and thus use their experiments to illustrate this design. In the work of Gielnik et al. (2015), the first experiment, which manipulated both X (effort) and M (venture progress) and measured Y, was similar to this full factorial design. The difference is that Gielnik et al.’s (2015) experiment did not have the “control groups” where M was varied freely (i.e., Groups E and F as discussed here). In this respect, the causality between X and M in the first experiment from Gielnik et al. (2015) could not be verified in a strict sense. Consequently, Gielnik et al. (2015) conducted a second experiment (see their footnote 2) to establish the causality between X and M. In this regard, the Gielnik et al. (2015) study combined the full factorial design and the two independent experiment design. If Gielnik et al. (2015) had included additional conditions in their first experiment, this would have then precluded the need for another experiment.

There are many advantages to this design. First, any causal effects between the variables of interest can be disentangled. This means that the causality of the different aspects of the relationship can be understood, as well as (to a degree at least) the magnitude of the effect of mediation. This can be especially helpful for beginning to get at the complexity of relationships in entrepreneurship theory. Second, this design enables researchers to explore any potential interacting or moderating effects of X and M_man on Y thereby including or ruling out alternative explanations for Y in reporting the study results. This approach could provide added nuance to the underlying mechanisms that are the focus of the study in entrepreneurship research utilizing mediation tests.

Several issues regarding this design must be addressed. First, a drawback of this design is that it requires a minimum of six experimental groups and thus a large sample size. As is the case in the two independent randomized experiments design, this need for a large sample can influence the feasibility of conducting such an experiment—which is already a challenge in entrepreneurship research (e.g., Short et al., 2010). Second, and most importantly, a confounding issue exists related to manipulating M independently of X in the same experiment (Bullock et al., 2010). As Pirlott and MacKinnon (2016) suggested, this design requires the researchers to manipulate the mediator independently of the manipulation of the independent variable in a single experiment to examine mediation. However, the manipulation of the mediator could be confounded by the manipulation of the independent variable, particularly when the mediator is a psychological variable. Take again, for example, the model of self-efficacy, passion, and persistence. If theory suggests that self-efficacy increases passion, manipulating the study participants’ self-efficacy should also make them more passionate about entrepreneurship, which would undermine the manipulation of passion (e.g., manipulation of low passion). How can the researchers manipulate strong self-efficacy and low passion in the same experiment if self-efficacy is supposed to increase passion? The manipulation of the independent variable affects the participant’s psychological factor as the mediator and confounds the manipulation of that psychological mediator.

We suggest that resolving confounding manipulations is one reason that many entrepreneurship researchers may shy away from using this design. To further illustrate this point, let us again consider the mediation model of effort-passion-intention—a modified version of Gielnik et al. (2015). Since effort increases passion theoretically, the participant’s passion should change in accordance with the manipulation of effort received in Groups A, B, C, and D. As a result, in these groups the passion manipulation would be confounded by the participant’s passion that could in turn be affected by the manipulation of effort. Put differently, it would be difficult to manipulate strong effort and low passion and then make them completely independent (see Newport, 2016).

The reason that the confounding issue arises is, again, because there is no direct manipulation of a psychological variable/mediator. Consequently, the psychological mediator is usually strongly associated with its theorized predictor. Manipulating the independent variable may then increase or decrease the psychological mediator thereby potentially confounding any other possible manipulation of the mediator. In the original model of Gielnik et al. (2015), the mediator is venture progress, which is not a psychological variable and can be manipulated directly. Accordingly, the full factorial design to test the Gielnik et al. (2015) model or something similar to this is potentially feasible. When the mediator is a psychological factor, one way to manage this confounding issue is to employ robust and rigorous manipulation checks (Ejelöv & Luke, 2020; Fiedler et al., 2021), both in a pilot study and the experiment itself (see Grégoire et al., 2019), to understand the extent to which the confounding issue may exist in an experiment. When the confounding issue is not found to exist in a pilot study, the researcher can more confidently utilize this design. However, when the confounding issue is found to exist in a way that may undermine the results, we suggest alternative designs that can serve as further remedies, as we now discuss.

Fractional Factorial Designs

Because of some of the challenges that exist with existing experimental designs for testing mediation in psychological research in entrepreneurship, here we suggest the fractional factorial design as being more parsimonious in nature and as the beginning to address the confounding and complicating issues in entrepreneurship, making it suited for testing mediation in entrepreneurship research. To address the confounding issue as to the independent manipulations of X and M and to enable more research in entrepreneurship that makes possible causal inferences about theoretical relationships, we propose to use only part of the experimental groups that are required to be present in the full factorial design. Such designs are fractional factorial designs. Below we discuss two types of fractional factorial designs. First, there is the fractional factorial design that is a four-group design. Then we discuss a fractional factorial design that is a three-group design.

In the four-group fractional factorial design, we argue that only four groups in Table 4 (Groups A, B, E, and F) are needed to establish M on Y, X on M, and X on Y. We illustrate this modified design in Table 5. Note that this design does not use all the combinations of X and M. While the two groups (Group C: X_h and M_man-l; Group D: X_l and X_man-h) in Table 4 are not included in Table 5, this does not affect the test of the mediation. Mediation requires the causal effects of X on M, M on Y, and possibly X on Y. Groups A and E enable the researchers to examine M_man on Y. In both groups, X is manipulated at the same level (e.g., high). The difference is with M, which is manipulated at a high level in Group A but observed or measured in Group E. The researchers can use simple analysis techniques such as ANOVA or regression to then examine the effect of M_man-h (Group A), compared to M_obs (Group E), on Y. If there is a significant difference, then the causal effect of M on Y is evident.

Table 5.

Four-Group Design (With the Enhancement Manipulation).

Group A	R	X _h	M _man-h	Y
Group B	R	X _l	M _man-l	Y
Group E	R	X _h	M _obs	Y
Group F	R	X _l	M _obs	Y

Note. R = random assignment; X_h = X is manipulated at the high level; X_l = X at the low level; M_man-xh = M is manipulated at a high level; M_man-l = M is at a low level; M_obs = M varies freely and is observed; Y = the dependent variable.

In a similar vein, Groups E and F allow the examination of the causal effects of X on M_obs and on Y. In both groups, M and Y are measured. X is manipulated at the high level in Group E and the low level in Group F. Using ANOVA or regression, the researchers can attribute any between-group difference in M or Y to the manipulation of X. Therefore, the complete causal paths underlying mediation are examined with this fractional factorial design. To verify this causal chain, the researchers may use a more state-of-art technique such as Andrew Hayes’ PROCESS procedure (Hayes, 2009). The two missed groups will be needed only when the researchers want to explore the moderation effect of X and M on Y (Pirlott & MacKinnon, 2016).

For example, see Kollmann et al.’s (2017) second experiment that manipulated perceived loss of financial resources (X) and measured fear of failure (M) and opportunity evaluations (Y). In this particular experiment, the causal effect of M on Y was not established. We wonder how future research could establish this mediation effect in a single experiment, however, since fear of failure can be affected by many things. Indeed, we see several ways to manipulate fear of failure corresponding to the effect of X. For example, providing information on the economic downturn may elicit fear of failure and strengthen the effect of X. Researchers could then map those manipulations into this fractional factorial design. The four groups would contain Group A, which provides information on the large loss of financial resources (a high level of X) and economic downturn to elicit fear of failure (a high level of M),⁴ Group B, which provides information on the small or no loss of financial resources (a low level of X) and economic growth (a low level of M), and then Group E and Group F, where X is then manipulated at the two levels and no information is provided on the economic environment (M varied freely). Theoretically, comparing Groups A and E or B and F can then establish the causal effect of M on Y. Comparing Groups E and F can establish the causal effects of X on M and Y. However, whether the mediation effect is found partly depends on the strength of the manipulations.

We argue that there are several advantages to this design. First, it reduces the minimum number of experimental groups to 4. This is helpful because the required sample size is reduced, which helps alleviate a substantial challenge faced in entrepreneurship research (Short et al., 2010). Second, it resolves the confounding issues and mitigates the problems that the independent variable and the mediator are often intertwined and can hardly be manipulated independently as we discussed before. In this design, X and M are manipulated concurrently, corresponding to each other in the two experimental groups.

With respect to the three-group fractional factorial design, we note that a close evaluation of Table 5 indicates that the complete causal chain can be examined without using all four groups. The revised design is illustrated in Table 6. In this three-group factorial design, the causal effects of X on M and Y can be examined by comparing Groups E and F. The causal effect of M on Y can be examined by comparing Group A, where M is manipulated at a high level consistent with X, with Group E, where M varies freely and is observed or measured. Because X is manipulated the same way in both Groups A and E, X can be considered constant. If there is any change in Y, it is attributable to the difference between the manipulation of M in Group A and the omission of the M manipulation in Group E. The causal effect of M on Y can thus be examined. Consequently, the complete causal chain underlying mediation can be revealed.

Table 6.

Three-Group Design (With the Enhancement Manipulation).

Group A	R	X _h	M _man-h	Y
Group E	R	X _h	M _obs	Y
Group F	R	X _l	M _obs	Y

Note. R = random assignment; X_h = X is manipulated at the high level; X_l = X at the low level; M_man-xh = M is manipulated at the high level; M_man-l = M is at the low level; M_obs = M varies freely and is observed; Y = the dependent variable.

Again, take Kollmann et al.’s (2017) experiment as an example. Applying this type of fractional factorial design would require only three groups to verify the complete causal chain underlying mediation. Group A provides information on the large loss of financial resources (a high level of X) and the economic downturn to elicit fear of failure (a high level of M). Group E provides information on the large loss of financial resources (a high level of X) but nothing about the economic environment (M varied freely). Group F provides information on the small loss of financial resources (a low level of X) but nothing about the economic environment (M varied freely). Comparing Groups A and E establishes the causal effect of M on Y. Comparing Groups E and F establishes the causal effects of X on M and Y. The three causal paths in mediation can be tested.

While the three-group fractional factorial design discussed above is the most parsimonious and cost-efficient experimental design for testing mediation, it has an important caveat. Because M is manipulated at a high level in one group and observed rather than manipulated at a low level in another group, the observed M can be relatively high, thereby making the effect size small and the results not statistically significant. Comparing the two groups that manipulate M at the extreme levels as in the other fractional factorial designs should derive a larger effect size and be able to reveal the treatment effect more profoundly. Therefore, we suggest that researchers use this three-group design only in specific conditions, such as when the treatment effect of manipulating M—at only one level to contrast the participants’ baseline M (i.e., varied freely)—is theorized to be sufficiently large.

As discussed previously, testing mediation rigorously is challenging as it requires verification of two or more causal paths, which involves randomization and manipulation of both the independent variable and the mediator. Regardless of what design choice the researcher adopts, sometimes it is unethical or unfeasible to manipulate the mediator. For example, in research that argues that moral disengagement mediates the relationship between motivation for financial gains and making unethical decisions (Baron et al., 2015), it would be unethical or infeasible to manipulate or increase the study participant’s moral disengagement as the mediator. We suspect that it is, in part, for this reason that many of the experimental studies testing mediation in entrepreneurship utilize a single MME, in which the authors manipulate the independent variable and measure the mediator and the dependent variable in one experiment as a way to test the mediation effect. Of course, in such studies, the questionable causality between the mediator and the dependent variable is acknowledged as a limitation. In the following section, we further detail the challenges that arise from using a single MME to test mediation and then propose two remedies for researchers to improve on causal inferences for mediation effects in such circumstances.

Challenges of Using a Single MME to Test Mediation and Suggested Remedies

As we have described previously, the use of an MME is not ideal. But as we have also noted, there are times when this may be the only option. When using an MME that measures both M and Y after the manipulation of X, three possible mediation models are implied. The first model is the target mediation model, which would suggest the mediation effect of M on the X-Y relationship, or the X → M → Y model, regardless of any direct effect of X on Y. The second model is the reverse mediation model, which would suggest the mediation effect of Y, or the X → Y → M model, regardless of any direct effect of X on M. The third model is the parallel mediation model with bilateral causal effects between M and Y, which would suggest both X → M → Y and X → Y → M (see Figure 2). In this design, even if the experiment that manipulates X and measures M and Y finds that M and Y change in accordance with X, we do not know if X causes Y through M or if X causes Y directly and then affects M. That is, whether the changes in M and Y are from paths a and b₁* or from paths c and b₂* is still in question.

Figure 2.

Illustration of parallel mediation.

As an example, Wieland et al. (2019) used a manipulation involving a masculine-typed venture versus a feminine-typed venture and measured entrepreneurial self-efficacy as a mediator and venture desirability as the dependent variable. A limitation of the design employed is that it cannot verify whether entrepreneurial self-efficacy leads to venture desirability or the other way around. As helpful as this prior research is for entrepreneurship theory, such use of a single MME in experimental methods is unable to determine which model is correct.

Broadly speaking, prior research has articulated three requirements for making causal inferences (Preacher, 2015; Shadish et al., 2002): (a) temporal precedence or sequence, (b) observed covariance, and (c) no alternative explanations for the causal relationship under investigation. Causality can be inferred when these three requirements are met. While condition (b) of the mediation requirements, observed covariance between M and Y, can be satisfied by a single MME, the requirement of (a) temporal precedence between M and Y is not met as M and Y are measured at the same time. Additionally, (c) alternative explanations such as reverse mediation cannot be ruled out.

It may be argued that the temporal precedence, and hence causality, can be inferred if M is measured before Y (cf. Aguinis et al., 2017; Stone-Romero & Rosopa, 2011). But this argument is not necessarily true because the manipulation of X may have simultaneously affected both M and Y before the measurements take place. The requirement of temporal precedence between M and Y cannot be met simply by measuring M before Y. The concerns mentioned above lead to the conclusion that a single MME cannot test and warrant mediation effects in a strict sense (Pirlott & MacKinnon, 2016; Stone-Romero & Rosopa, 2011). However, there are cases where temporal precedence can still be established in a single MME. We now discuss two remedies that can be used to enhance causal inferences in situations in entrepreneurship requiring a single MME because prior experimental designs are not possible.

Remedy 1: Create Cognitive Precedence

The first remedy is to create cognitive precedence for M and Y in a single MME so that the manipulation of X will affect M but not Y until the measurement of Y takes place. Lemmer and Gollwitzer (2017, p. 748) called it the “conceptual timing criterion.” We see an example of this approach in the entrepreneurship literature. As noted, Gish et al. (2019) used an experiment to examine the mediation effect of cognitive attention (M) on the relationship between the participant’s sleep deprivation (X) and the quality of the business idea he/she generated (Y). They manipulated the participant’s sleep deprivation by asking him/her to play video games, watch movies, or do any activities all night and then reported to the lab the following morning. After answering the questions about his/her cognitive attention, the participant was asked to generate a business idea, which was later rated by two independent research assistants. As the participants had not developed a business idea until their cognitive attention was measured, it was impossible that the development of a business idea affected cognitive attention. Such a design created cognitive precedence between M and Y and satisfied the requirement of temporal precedence because the manipulation of X (sleep deprivation/quality) only affected M (cognitive attention) and not Y (the idea quality) until the participant is contextualized in Y. When cognitive precedence between M and Y is established, reverse causality between M and Y is unlikely.

This approach can be extended to other entrepreneurship studies examining mediation. For example, Williamson et al. (2018) used the experience sampling method to examine the mediating effect of mood on the relationship between sleep quality and the entrepreneurs’ innovative behavior. If the researchers would examine the complete causal chain underlying the proposed mediation effect, they could conduct a single MME with cognitive precedence as discussed in the Gish et al. (2019) experiment. In this potential MME, the participant’s mood (M) could be measured in the morning after sleep deprivation (X). The participant would not be made aware of the innovative task (Y) he/she will be asked to perform soon until after the participant’s mood (M) is formed and measured. In this case, it is impossible that the participant’s innovative behavior (Y) could occur before the induced mood (M).

While we have discussed the conditions where (a) temporal precedence and (b) observed covariance can be satisfied in a single MME, we have not yet addressed (c) non-alternative explanations for M to mediate the X-Y relationship. Indeed, it is possible that X affects M and an unobserved variable, and the effect on Y is caused by that unobserved variable rather than M. That is, if the researchers find a way to capture the “unobserved” variable and include it in the model, it will wipe out the effect of M and becomes an alternative explanation for the X-Y relationship. To alleviate this concern, we suggest that researchers can test for full mediation (Figure 1c). Full mediation means that the effect of X on Y is completely transmitted through M. That is, when M is presented in the model, there is no direct effect (no path c in Figure 1b) or any other possible paths. Zhao et al. (2010, p. 210) argued that in indirect-only mediation (i.e., full mediation), any omitted mediator is “unlikely.” When there is full mediation, alternative explanations, such as unobserved variables are ruled out⁵ and (c) is satisfied.

The discussion above leads to a conclusion: the three requirements of causality underlying mediation can be satisfied with a carefully designed, single randomized experiment that manipulates X and measures M and Y (i.e., MME). Specifically, such an experiment can warrant mediation when (a) there is cognitive precedence between M and Y, that is, measuring M before contextualizing participants in Y and (b) M fully mediates the effect of X on Y. However, if both theory and the results of the study show partial mediation only, creating cognitive precedence alone is not sufficient to robustly establish the mediation and causality between M and Y. In such a case, researchers are encouraged to use other remedies or conduct another experiment to further test mediation.

Remedy 2: Qualify X as an Instrumental Variable

In psychological experiments, it may be difficult (although admittedly not necessarily impossible) to directly manipulate a psychological factor, such as an emotion or cognition. Consequently, researchers often manipulate this type of variables indirectly or externally (e.g., Ivanova et al., 2018) and then measure the participant’s self-reports on the variable of interest as the manipulation check and the dependent variable. If the manipulation check shows that the manipulation is successful, the researchers claim the causal effect between the two variables (see Figure 3). For example, there is no direct manipulation of mood. Accordingly, indirect manipulations such as sleep deprivation or weather can be used to manipulate mood (Dushnitsky & Sarkar, 2022; Williamson et al., 2018). Researchers then use a manipulation check (a self-reported measure of mood) (M) to verify that mood is changed by those indirect manipulations (X). If both mood and the dependent variable (Y) are measured, however, then how can researchers make the causal inference for mood and the dependent variable?

Figure 3.

Causal effects with a manipulation check.

To answer this question, let’s consider a recent MME undertaken by Dushnitsky and Sarkar (2022), who found, but did not hypothesize, that weather affected the investor’s investment evaluations through the investor’s mood. They showed the study participants (e.g., investors) the pictures of a local sunny day or cloudy day, depending on the group to which the participants were randomly assigned, to manipulate the weather effect (X) and measured the participants’ mood (M). They then asked the participants to watch a pre-recorded business pitch and make investment evaluations (Y). The researchers found that a sunny day led to the investor’s positive mood as well as a positive evaluation of the business pitched. To argue for the causal effect of mood on the investment evaluations, we follow Pearl and Mackenzie (2018) to consider “counterfactuals” or “what-ifs” when examining causal effects. That is, would a sunny day still lead to a positive evaluation of the investors had their mood not been affected? It should not because no other theory suggests a direct relationship between weather and investment evaluations when holding mood constant. Therefore, the only explanation for the weather effect is mood (M). To verify this empirically, the researchers could use the subgroups of the participants with the same level of mood in both experimental groups (sunny day and cloudy day) and examine whether the weather effect still exists. While the means of mood between the two experimental groups should be significantly different, some participants across the two groups may still have the same level of mood. For these participants, the disappearance of the weather effect would indicate that mood blocks the effect of weather on the investment evaluations (i.e., the blockage effect (Pirlott & MacKinnon, 2016)). In this case, it is evident for mood to be the mediator.

This type of theorizing is consistent with the notion of instrumental variables, which are often used for mediation studies in medical research (Baiocchi et al., 2014; Didelez & Sheehan, 2007), economics and public policy research (Becker, 2016; Gathergood, 2013; Imai et al., 2011), and biological and personality research (Briley et al., 2018; Didelez & Sheehan, 2007). In those fields, longitudinal studies or natural experiments are used to examine causality between a predictor variable (P)⁶ and an outcome variable (Y). In such a setting, an unmeasured confound (U) is likely to affect P and Y (Figure 4), as P is not randomly manipulated in those studies. Lacking randomization makes the causal inference questionable. In response, methodologists provide the tool of instrumental variables to verify causality between P and Y.

Figure 4.

Illustration of an instrumental variable.

There are three requirements for a variable to qualify and serve as an instrumental variable (Z): randomization, relevance, and exclusion (Baiocchi et al., 2014; Becker, 2016; Pearl, 2009; Sajons, 2020). Randomization reflects the notion that the instrumental variable Z is randomly occurred to subjects and should not have an effect on Y. Relevance refers to a strong association between the instrumental variable (Z) and the predictor (P). The exclusion principle requires the effect of Z on Y, if any, to be exclusively through P and nothing else (Baiocchi et al., 2014; Becker, 2016). If the researchers are able to find a qualified Z, then the causality between P and Y can be established. If we remove U from Figure 4, the figure becomes identical to full mediation (Otter et al., 2018; Shrout & Bolger, 2002) as illustrated in Figure 1c or 3, in that Z maps onto X and P maps onto M. For further information regarding instrumental variables, we suggest Becker (2016), Pearl (2009), and Baiocchi et al. (2014) as helpful additional resources.

Returning to the MME experiment by Dushnitsky and Sarkar (2022), the three qualifications for instrumental variables are met: the weather condition is randomly assigned to study participants (randomization principle); this condition is highly correlated with mood (the relevance principle); and the statistical relationship between the weather manipulation and investment evaluations is found when no other theory suggests a direct relationship between weather and investment evaluations (the exclusion principle). Therefore, the only explanation for the weather effect is mood (M). According to the theory of instrumental variables, the mediation effect of mood in this MME should be deemed as evidence of causality.

While the use of instrumental variables to test causality is encouraged in economics, medical health, and biology (Baiocchi et al., 2014; Becker, 2016; Briley et al., 2018; Didelez & Sheehan, 2007; Gathergood, 2013), a major challenge in entrepreneurship is that the independent variable is not always qualified as the instrumental variable. We suggest that entrepreneurship researchers carrying out an MME first check whether their independent variable qualifies as an instrumental variable. They can do so by (a) examining whether the independent variable meets the three requirements for instrumental variables in theory and (b) empirically testing the blockage effect of the mediator using the subgroups of the participants, as we discussed earlier. In the case where the independent variable does not qualify as the instrumental variable, researchers are advised to seek other remedies, such as creating cognitive precedence or conducting a reverse mediation test, as we now discuss in the context of the broader recommendations and suggestions we have covered herein.

Discussion

Examining mediation makes important contributions to theory building and testing (Colquitt & Zapata-Phelan, 2007). Since the mediation effect contains two or more causal paths, testing is challenging. To verify the causal paths, multiple experiments are usually recommended (MacKinnon et al., 2012; Stone-Romero & Rosopa, 2008). However, sometimes it is unethical or unfeasible to manipulate the mediator in a randomized experiment. Most critically, the approach of multiple experiments requires a manipulation of M independently from the manipulation of X. Even if such an independent manipulation of M is feasible, this approach may invoke unobserved causes of M and Y as a result of the second manipulation (i.e., the complicating issue as we discussed earlier). In this case, the original theoretical model becomes underspecified. Consequently, many studies utilize a single MME, which manipulates the independent variable and measures the mediator and the dependent variable in one experiment, to test the mediation effect and commonly acknowledge the questionable causality between the mediator and the dependent variable as a limitation. A problem with a single MME is that many unobserved variables could also precede or come after the proposed independent variable, and these unobserved “mediators” could only be revealed post treatment. As such, using a single MME to test mediation is often deemed as not meeting the requirement of empirical rigor by top-tier entrepreneurship journals.

Recommendations

We argue that there are nonetheless certain conditions in which researchers can use a single MME to examine mediation. We identify these conditions and propose corresponding remedies. In the conditions where the remedies do not apply, we classify several parsimonious yet still rigorous experimental designs for testing mediation. Our recommendations are provided as follows. These steps are illustrated in the decision tree (Figure 5).

Figure 5.

Decision tree for conducting experiments to test mediation in entrepreneurship.

First, we suggest researchers determine whether X and M can be manipulated independently in the same experiment. If so, we recommend using the full factorial design so that every relationship between X, M, and Y, including moderation, can be clearly investigated. The downside is that a large sample size will be required given six or more experimental groups. Second, if it is not possible to attain a large sample size or manipulate X and M independently, we suggest researchers consider whether it is possible to manipulate M in a way to enhance rather than suppress X. If it is possible, we recommend using our four-group or three-group fractional factorial design. The choice between the four-group and three-group designs depends on the strength of the treatment effect as discussed earlier. Third, if it is infeasible or unethical to manipulate M at all, we suggest researchers consider whether M would fully mediate the relationship between X and Y theoretically. If partial mediation is expected and other mediators are likely but not included in the theoretical model, we suggest researchers consider revising the model. If full mediation is likely in theory, we recommend using an MME along with the proposed remedies, (a) creating cognitive space, and (b) finding a way to manipulate X as the instrumental variable. In those specific conditions, an MME may still verify the causal chain underlying mediation.

General Limitations of Experiments and Alternative Methodologies

The randomized experiment is one of the most rigorous research methods to examine causality (Diener et al., 2022; Hsu et al., 2017). However, it is not without limitations. For example, it is possible that the random assignment is not completely random (Neuberg, 2003; Pedersen & Larson, 2016). Even if the researchers blindly and randomly assign study participants into experimental groups, there is no guarantee that the participants in each group are qualitatively the same (Bullock et al., 2010). With this in mind, researchers are encouraged to confirm randomization by examining individual differences between groups on other variables that are theoretically unrelated to the manipulations. Any significant differences between individuals on such variables would indicate the possibility of unsuccessful randomization, and these variables then need to be included as control variables in the analysis. When this occurs, claims regarding causality are not warranted. It is also not uncommon that some participants in a research study fail the manipulation checks (Diener et al., 2022; Grégoire et al., 2019). And the resulting question of whether to include these participants in the analyses in turn also casts doubt on whether or not a researcher can make causal inferences based on the results of the study.

Additionally, randomized experiments are not always feasible (Diener et al., 2022). In some cases, experiments are not feasible for testing certain research questions regarding mediation. A helpful guide for where experiments may not be appropriate for testing theory is elucidated through the work of Kim et al. (2016) who suggested that entrepreneurship research needs to more thoroughly understand levels of analysis—especially meso-level structures that exist between micro and macro levels of analysis. As an example, they note how temporality needs to be taken into multi-level accounts. This relates to the limitations of experiments in capturing causality. Since the process from the environment to individual cognition to action involves time (Johnson & Schaltegger, 2020), it would be difficult to confirm theorized causality empirically in a strict sense (Berglund & Korsgaard, 2017). While researchers may employ a field experiment for a period of time (Hsu et al., 2017), many factors can happen during this period and thus confound the cause of the dependent variable and the effect of the manipulation. As a result of these limitations with respect to causality, alternative methods have been proposed, such as process tracing, mathematical modeling, simulations (cf. Hedström & Wennberg, 2017), difference-in-difference methods (Lechner, 2011), and longitudinal mediation data (MacKinnon et al., 2007). Each of these methods offers a different pathway to enabling causal inferences to be made under specific conditions and has strengths and limitations. These methods thus represent a broad set of approaches to enabling researchers to understand causality. Through our work, we hope to articulate more clearly how researchers can be mindful and clear regarding limitations and assumptions for using experiments, as well as these other methods, to claim causality in theorizing mediation.

Conclusion

Traditional wisdom suggests that the most rigorous approach to examining mediation is via sophisticatedly designed experiments. While this is true in many cases, it is not always feasible or ethical to manipulate the mediator in research that investigates the role of psychological factors in entrepreneurship. This is perhaps the major reason why many existing studies employ an MME to test mediation. It is not surprising that journal reviewers reject papers using a single MME because the verification of the causal chain is deemed as not complete with a single MME. We hope the experimental designs and the remedies for MMEs proposed in this paper will help entrepreneurship researchers make causal inferences more confidently, alleviate reviewers’ concerns, and help propel theory and practice forward in the psychology of entrepreneurship specifically and in entrepreneurship research in general.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Dan K. Hsu

J. Robert Mitchell

Xian Cao

Notes

Author Biographies

Dan K. Hsu is an Associate Professor at North Dakota State University and a Research Fellow for the Center for Entrepreneurship and Family Business. His research interests include experimental designs and entrepreneurs’ decision-making, emotions, cognitions, and well-being.

J. Robert Mitchell is a Professor at Colorado State University and an Adjunct Research Professor at the Ivey Business School. Rob received his PhD from Indiana University. His research explores how the socially-situated cognition of entrepreneurs, managers, and other stakeholders can enable the creation of value through new/existing firms.

Xian Cao is an Assistant Professor of Entrepreneurship at Ball State University. Her research focuses on female entrepreneurs, the motivation of entrepreneurship, and research methods.

References

Aguinis

Edwards

J. R.

Bradley

K. J.

(2017). Improving our understanding of moderation and mediation in strategic management research. Organizational Research Methods, 20(4), 665–685.

Allison

T. H.

Warnick

B. J.

Davis

B. C.

Cardon

M. S.

(2022). Can you hear me now? Engendering passion and preparedness perceptions with vocal expressions in crowdfunding pitches. Journal of Business Venturing, 37(3), 106193. https://doi.org/10.1016/j.jbusvent.2022.106193

Anderson

B. S.

Wennberg

McMullen

J. S.

(2019). Enhancing quantitative theory-testing entrepreneurship research. Journal of Business Venturing, 34(5), 1–13.

Baiocchi

Cheng

Small

D. S.

(2014). Instrumental variable methods for causal inference. Statistics in Medicine, 33(13), 2297–2340.

Baron

R. A.

Zhao

Miao

(2015). Personal motives, moral disengagement, and unethical decisions by entrepreneurs: Cognitive mechanisms on the “slippery slope”. Journal of Business Ethics, 128(1), 107–118.

Baum

J. R.

Frese

Baron

R. A.

(2014). The psychology of entrepreneurship. Psychology Press.

Becker

S. O.

(2016). Using instrumental variables to establish causality. IZA World of Labor.

Berglund

Korsgaard

(2017). Opportunities, time, and mechanisms in entrepreneurship: On the practical irrelevance of propensities. Academy of Management Review, 42(4), 730–733.

Bischoff

K. M.

Gielnik

M. M.

Frese

(2020). When capital does not matter: How entrepreneurship training buffers the negative effect of capital constraints on business creation. Strategic Entrepreneurship Journal, 14(3), 369–395.

10.

Briley

D. A.

Livengood

Derringer

(2018). Behaviour genetic frameworks of causal reasoning for personality psychology. European Journal of Personality, 32(3), 202–220.

11.

Bullock

J. G.

Green

D. P.

S. E.

(2010). Yes, but what’s the mechanism? (don’t expect an easy answer). Journal of Personality and Social Psychology, 98(4), 550–558.

12.

Burnette

J. L.

Pollack

J. M.

Forsyth

R. B.

Hoyt

C. L.

Babij

A. D.

Thomas

F. N.

Coy

A. E.

(2020). A growth mindset intervention: Enhancing students’ entrepreneurial self-efficacy and career development. Entrepreneurship Theory and Practice, 44(5), 878–908.

13.

Cardon

M. S.

Kirk

C. P.

(2015). Entrepreneurial passion as mediator of the self–efficacy to persistence relationship. Entrepreneurship Theory and Practice, 39(5), 1027–1050.

14.

Cheung

G. W.

Lau

R. S.

(2007). Testing mediation and suppression effects of latent variables: Bootstrapping with structural equation models. Organizational Research Methods, 11(2), 296–325. https://doi.org/10.1177/1094428107300343

15.

Cialdini

R. B.

(2009). We have to break up. Perspectives on Psychological Science, 4(1), 5–6.

16.

Colquitt

J. A.

Zapata-Phelan

C. P.

(2007). Trends in theory building and theory testing: A five-decade study of the academy of management journal. Academy of Management Journal, 50(6), 1281–1303.

17.

Cornelis

E. F.

Baker

A. M.

Ahsan

(2022). Mitigating a crisis of confidence: The effect of crisis response strategies on reward-based crowdfunding success. Strategic Entrepreneurship Journal, 16(1), 67–96.

18.

Didelez

Sheehan

(2007). Mendelian randomization as an instrumental variable approach to causal inference. Statistical Methods in Medical Research, 16(4), 309–330.

19.

Diener

Northcott

Zyphur

M. J.

West

S. G.

(2022). Beyond experiments. Perspectives on Psychological Science. 17(4), 1101–1119. https://doi.org/10.1177/17456916211037670

20.

Dushnitsky

Sarkar

(2022). Here comes the sun: The impact of incidental contextual factors on entrepreneurial resource acquisition. Academy of Management Journal, 65(1), 66–92.

21.

Ejelöv

Luke

T. J.

(2020). “Rarely safe to assume”: Evaluating the use and interpretation of manipulation checks in experimental social psychology. Journal of Experimental Social Psychology, 87, 103937.

22.

Engel

Noordijk

Spoelder

van Gelderen

(2021). Self-compassion when coping with venture obstacles: Loving-kindness meditation and entrepreneurial fear of failure. Entrepreneurship Theory and Practice, 45(2), 263–290. https://doi.org/10.1177/1042258719890991

23.

Engel

Ramesh

Steiner

(2020). Powered by compassion: The effect of loving-kindness meditation on entrepreneurs’ sustainable decision-making. Journal of Business Venturing, 35(6), https://doi.org/10.1016/j.jbusvent.2019.105986.

24.

Fiedler

McCaughey

Prager

(2021). Quo vadis, methodology? The key role of manipulation checks for validity control and quality of science. Perspectives on Psychological Science, 16(4), 816–826.

25.

Frese

Gielnik

M. M.

(2014). The psychology of entrepreneurship. Annual Review Organizational Psychology and Organizational Behavior, 1(1), 413–438.

26.

Gathergood

(2013). An instrumental variable approach to unemployment, psychological health and social norm effects. Health Economics, 22(6), 643–654.

27.

Gielnik

M. M.

Cardon

M. S.

Frese

(2020). The psychology of entrepreneurship: New perspectives. Routledge.

28.

Gielnik

M. M.

Spitzmuller

Schmitt

Klemann

D. K.

Frese

(2015). “I put in effort, therefore I am passionate”: Investigating the path from effort to passion in entrepreneurship. Academy of Management Journal, 58(4), 1012–1031.

29.

Gielnik

M. M.

M. A.

Funken

Bischoff

K. M.

(2017). Boosting and sustaining passion: A long-term perspective on the effects of entrepreneurship training. Journal of Business Venturing, 32(3), 334–353.

30.

Gish

J. J.

Wagner

D. T.

Grégoire

D. A.

Barnes

C. M.

(2019). Sleep and entrepreneurs’ abilities to imagine and form initial beliefs about new venture ideas. Journal of Business Venturing, 34(6), 105943.

31.

Grégoire

D. A.

Binder

J. K.

Rauch

(2019). Navigating the validity tradeoffs of entrepreneurship research experiments: A systematic review and best-practice suggestions. Journal of Business Venturing, 34(2), 284–310.

32.

Hayes

A. F.

(2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76(4), 408–420.

33.

Hayes

A. F.

Preacher

K. J.

(2014). Statistical mediation analysis with a multicategorical independent variable. British Journal of Mathematical and Statistical Psychology, 67(3), 451–470.

34.

Hedström

Wennberg

(2017). Causal mechanisms in organization and innovation studies. Innovation, 19(1), 91–102.

35.

Hsu

D. K.

Simmons

S. A.

Wieland

A. M.

(2017). Designing entrepreneurship experiments: A review, typology, and research agenda. Organizational Research Methods, 20(3), 379–412.

36.

Hubner

Baum

Frese

(2020). Contagion of entrepreneurial passion: Effects on employee outcomes. Entrepreneurship Theory and Practice, 44(6), 1112–1140.

37.

Imai

Keele

Tingley

Yamamoto

(2011). Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies. American Political Science Review, 105(4), 765–789.

38.

Ivanova

Treffers

Langerak

(2018). Emotional paths leading to opportunity desirability and feasibility beliefs through controllability. International Small Business Journal, 36(5), 546–573.

39.

Johnson

M. A.

Stevenson

R. M.

Letwin

C. R.

(2018). A woman’s place is in the… startup! Crowdfunder judgments, implicit bias, and the stereotype content model. Journal of Business Venturing, 33(6), 813–831.

40.

Johnson

M. P.

Schaltegger

(2020). Entrepreneurship for sustainable development: A review and multilevel causal mechanism framework. Entrepreneurship Theory and Practice, 44(6), 1141–1173.

41.

Judd

C. M.

Kenny

D. A.

(2010). Data analysis in social psychology: Recent and recurring issues. In Fiske

S. T.

Gilbert

D. T.

Lindzey

(Eds.), Handbook of social psychology (5th ed., Vol. 5, pp. 115–139). John Wiley & Sons, Inc.

42.

Kim

P. H.

Wennberg

Croidieu

(2016). Untapped riches of meso-level applications in multilevel entrepreneurship mechanisms. Academy of Management Perspectives, 30(3), 273–291.

43.

Kollmann

Stöckmann

Kensbock

J. M.

(2017). Fear of failure as a mediator of the relationship between obstacles and nascent entrepreneurial activity—An experimental approach. Journal of Business Venturing, 32(3), 280–301.

44.

Lechner

(2011). The estimation of causal effects by difference-in-difference methods. Foundations and Trends® in Econometrics, 4(3), 165–224.

45.

Lemmer

Gollwitzer

(2017). The “true” indirect effect won’t (always)stand up: When and why reverse mediation testing fails. Journal of Experimental Social Psychology, 69, 144–149.

46.

Lerner

D. A.

(2016). Behavioral disinhibition and nascent venturing: Relevance and initial effects on potential resource providers. Journal of Business Venturing, 31(2), 234–252.

47.

Low

M. B.

MacMillan

I. C.

(1988). Entrepreneurship: Past research and future challenges. Journal of Management, 14(2), 139–161.

48.

MacKinnon

D. P.

Coxe

Baraldi

A. N.

(2012). Guidelines for the investigation of mediating variables in business research. Journal of Business and Psychology, 27(1), 1–14.

49.

MacKinnon

D. P.

Fairchild

A. J.

Fritz

M. S.

(2007). Mediation analysis. Annual Review of Psychology, 58, 593–614.

50.

Mahmood

Luffarelli

Mukesh

(2019). What’s in a logo? The impact of complex visual cues in equity crowdfunding. Journal of Business Venturing, 34(1), 41–62.

51.

Maula

Stam

(2020). Enhancing rigor in quantitative entrepreneurship research. Entrepreneurship Theory and Practice, 44(6), 1059–1090.

52.

Neuberg

L. G.

(2003). Causality: models, reasoning, and inference, by Judea Pearl, Cambridge University Press, 2000. Econometric Theory, 19(4), 675–685.

53.

Newport

(2016). So good they can’t ignore you. Hachette UK.

54.

Otter

Pachali

M. J.

Mayer

Landwehr

(2018). Causal inference using mediation analysis or instrumental variables-full mediation in the absence of conditional independence. Available at SSRN 3135313.

55.

Pearl

(2009). Causal inference in statistics: An overview. Statistics Surveys, 3, 96–146.

56.

Pearl

Mackenzie

(2018). The book of why: The new science of cause and effect. Basic Books.

57.

Pedersen

W. S.

Larson

C. L.

(2016). State anxiety carried over from prior threat increases late positive potential amplitude during an instructed emotion regulation task. Emotion, 16(5), 719–729.

58.

Pirlott

A. G.

MacKinnon

D. P.

(2016). Design approaches to experimental mediation. Journal of Experimental Social Psychology, 66, 29–38.

59.

Preacher

K. J.

(2015). Advances in mediation analysis: A survey and synthesis of new developments. Annual Review of Psychology, 66, 825–852.

60.

Rose

Wentzel

Hopp

Kaminski

(2020). Launching for success: The effects of psychological distance and mental simulation on funding decisions and crowdfunding performance. Journal of Business Venturing, 36(6), 106021.

61.

Rucker

D. D.

Preacher

K. J.

Tormala

Z. L.

Petty

R. E.

(2011). Mediation analysis in social psychology: Current practices and new recommendations. Social and Personality Psychology Compass, 5(6), 359–371.

62.

Sajons

G. B.

(2020). Estimating the causal effect of measured endogenous variables: A tutorial on experimentally randomized instrumental variables. The Leadership Quarterly, 31(5), 101348. https://doi.org/10.1016/j.leaqua.2019.101348

63.

Schellong

Kraiczy

N. D.

Malär

Hack

(2019). Family firm brands, perceptions of doing good, and consumer happiness. Entrepreneurship Theory and Practice, 43(5), 921–946.

64.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

65.

Shane

Drover

Clingingsmith

Cerf

(2020). Founder passion, neural engagement and informal investor interest in startup pitches: An fMRI study. Journal of Business Venturing, 35(4), 105949. https://doi.org/10.1016/j.jbusvent.2019.105949

66.

Short

J. C.

Ketchen

Jr, D. J.

Shook

C. L.

Ireland

R. D.

(2010). The concept of “opportunity” in entrepreneurship research: Past accomplishments and future challenges. Journal of Management, 36(1), 40–65.

67.

Shrout

P. E.

Bolger

(2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7(4), 422–445.

68.

Smith

E. R.

(2012). The beginning of a new editorial term at the attitudes and social cognition section of the Journal of Personality and Social Psychology (JPSP) is a natural occasion for reflection on this journal’s mission. Introduction. Journal of Personality and Social Psychology, 102(1), 1–3.

69.

Souitaris

Zerbinati

Peng

Shepherd

(2020). Should I stay or should I go? Founder power and exit via initial public offering. Academy of Management Journal, 63(1), 64–95.

70.

Spencer

S. J.

Zanna

M. P.

Fong

G. T.

(2005). Establishing a causal chain: Why experiments are often more effective than mediational analyses in examining psychological processes. Journal of Personality and Social Psychology, 89(6), 845–851.

71.

Stenholm

Renko

(2016). Passionate bricoleurs and new venture survival. Journal of Business Venturing, 31(5), 595–611.

72.

Stevenson

Allen

Wang

(2022). Failed but validated? The effect of market validation on persistence and performance after a crowdfunding failure. Journal of Business Venturing, 37(2), 106175. https://doi.org/10.1016/j.jbusvent.2021.106175

73.

Stone-Romero

E. F.

Rosopa

P. J.

(2008). The relative validity of inferences about mediation as a function of research design characteristics. Organizational Research Methods, 11(2), 326–352.

74.

Stone-Romero

E. F.

Rosopa

P. J.

(2011). Experimental tests of mediation models: Prospects, problems, and some solutions. Organizational Research Methods, 14(4), 631–646.

75.

Wieland

A. M.

Kemmelmeier

Gupta

V. K.

McKelvey

(2019). Gendered cognitions: A socio-cognitive model of how gender affects entrepreneurial preferences. Entrepreneurship & Regional Development, 31(3–4), 178–197.

76.

Williams

D. W.

Wood

M. S.

Mitchell

J. R.

Urbig

(2019). Applying experimental methods to advance entrepreneurship research: On the need for and publication of experiments. Journal of Business Venturing, 34(2), 215–223.

77.

Williamson

A. J.

Battisti

Leatherbee

Gish

J. J.

(2018). Rest, zest, and my Innovative and best: Sleep and mood as drivers of entrepreneurs’ innovative behavior. Entrepreneurship Theory Practice, 43(3), 582–610.

78.

Zhao

Lynch

J. G.

Jr. Chen

(2010). Reconsidering Baron and Kenny: Myths and truths about mediation analysis. Journal of Consumer Research, 37(2), 197–206.