Abstract
Management educators regularly innovate by designing and/or integrating experiential exercises into their classrooms. Many then endeavor to publish their exercises in academic journals, providing readers with a roadmap for how to use them in their classes. To convince colleagues that an activity helps students learn, authors will often include evidence demonstrating its impact. Importantly, there is a dearth of research exploring the types of evidence authors use to do so. We analyzed 185 articles presenting in-class management-related experiential exercises, published across seven journals between 1 January 2018 and 1 June 2024. In so doing, we contribute to the literature by synthesizing the ways in which authors demonstrate the effectiveness of their activities. We offer both a portrait of current practices and framework of strategies (by source, type, and preparation required) that prospective authors may draw on as they reflect on impactful ways to structure upcoming data collection efforts. We are, however, mindful that most authors in our sample were affiliated with institutions in North America. As norms regarding evidence for experiential exercises are collectively shaped by journal expectations and authors, we reflect on how our methodological choices contribute to shaping practices about what “counts” as credible evidence in management education.
Keywords
Introduction
Building on the growing interest in pedagogical innovation, there has been a notable rise in the use of experiential learning exercises as a specific and impactful approach to foster greater interaction in the classroom and student learning (Bradford, 2019). This is driven by a broader recognition of the benefits such exercises offer in business schools, particularly in helping students acquire essential job skills, enhancing classroom engagement during lessons, and improving overall academic performance (e.g. Armstrong and Mahmud, 2008; Clark and White, 2010; Leal-Rodriguez and Albort-Morant, 2019), in addition to providing a way for course material to become more relevant and easy to comprehend (e.g. Paul and Mukhopadhyay, 2005; Roberts, 2018). Indeed, meta-analytic findings by Burch et al. (2019) demonstrate the positive effects of experiential learning on learning-related outcomes, based on quantitative studies employing rigorous treatment and control group designs.
Given the benefits that experiential learning offers, many management educators regularly innovate in their classrooms, drawing on experiential learning theory (e.g. Kolb, 2015) to design and/or integrate new in-class experiential activities to help their students better understand and assimilate course material. Kolb (2015), for example, defines experiential learning as “the process whereby knowledge is created through the transformation of experience” (p. 49) and proposes a learning cycle, comprised of four learning modes (concrete experience, reflective observation, abstract conceptualization, and active experimentation) that helps to optimize the learning process. Thus, when incorporating experiential exercises in the classroom, educators can draw on this learning cycle (e.g. Kolb, 2015) by integrating as many of the learning modes as possible into their activities – providing students with a concrete experience (whether a simulation, role-play activity, game, etc.), prompting them to consider the meaning that underlies the experience, helping them to then forge connections between the experience and more abstract notions or concepts (such as course material), and finally, by providing them with an opportunity to put this reflection and knowledge into practice.
However, introducing experiential exercises into the classroom can be somewhat unnerving, as piloting new in-class activities carries a degree of potential risk. The literature highlights a number of facilitation challenges, such as exercises not proceeding as planned, leading to instructor frustration (e.g. Edelson et al., 2019; Schibrowsky and Peltier, 1995) or strong emotional reactions from students that create complex classroom dynamics (e.g. Clancy and Vince, 2019; Elmes, 2019). Notably, students sometimes experience negative emotions (Wright et al., 2022) or become so immersed in the experience itself that they fail to engage in critical reflection, analysis, or connections to theoretical frameworks (e.g. Sanderson, 2021; Wright et al., 2018). More generally, educators must be mindful of the fact that students can have varying levels of preference for “sensory-based” activities in the classroom – as Bas et al. (2023) note, while some learners are receptive and find such experiential activities enriching, others may resist or disengage, often due to discomfort, uncertainty, or prevailing norms that devalue sensory ways of knowing. Instructors may therefore ponder: What if the activity does not work as envisioned? What if students do not find it sufficiently valuable to justify the class time dedicated to it? And if the exercise fails to meet expectations, will it negatively impact the class environment and momentum established to that point in the semester?
When instructors do decide to integrate experiential activities in their classrooms, they can either design new exercises themselves or draw inspiration from instructional innovations published in a variety of business and management education–related journals. In such articles, authors share the novel activities they developed – ranging from in-class exercises and simulations to state-of-the-art course structures – so that readers may implement them in their own classrooms. In essence, authors writing instructional innovation articles offer readers a tested roadmap for how to implement a novel classroom activity. Their facilitation guidance helps to reduce some of the uncertainty associated with trying something new in the classroom, although it is never a guarantee that an innovation will translate across courses or contexts. Nonetheless, many of the aforementioned risks and considerations remain, most notably: Will this activity contribute meaningfully to my class, and will it help my students learn?
As such, it is imperative that authors of articles presenting experiential exercises provide compelling arguments to readers – that is, potential adopters of activities – that their activities work and will help students gain a deeper understanding of focal concepts. This argument is supported by the literature on innovation adoption, which consistently emphasizes that uptake is driven less by objective superiority and more by perceived value. Early work on organizational innovation adoption (e.g. Damanpour and Evan, 1984) and later scholarship in technology and product innovation (e.g. Kaur Kapoor et al., 2014) suggest that the rate at which innovations are adopted depends on how potential users interpret their attributes. Central among these attributes is the perception that the innovation offers meaningful benefits relative to existing alternatives.
This emphasis on perception is critical in an educational context. For example, Stroh et al. (2023) argue that instructional innovations gain traction when they clearly articulate the value they create for both instructors and students. Their work highlights that perceived value is multidimensional: it includes pedagogical effectiveness (e.g. improved conceptual understanding), efficiency (e.g. ease of implementation and time demands), and alignment with instructors’ teaching identities and course constraints. In other words, adoption decisions are shaped not only by whether an innovation “works” but also by whether instructors believe it works better than what they currently use, and whether it does so at an acceptable cost in terms of time, preparation, and classroom risk.
This insight aligns with broader innovation theory, which posits that a superior innovation offers a compelling value proposition to the user (Cooper, 2013) and must demonstrate a clear relative advantage over the idea it supplants (Rogers, 2010). In management education, that relative advantage may take the form of stronger student engagement, clearer links between theory and practice, or improved assessment outcomes. However, unless these advantages are made explicit and supported with credible evidence, potential adopters may default to familiar practices.
Accordingly, management educators introducing new in-class experiential exercises can strengthen their persuasive appeal by explicitly articulating the activity’s value proposition and substantiating it with robust evidence drawn from multiple sources (e.g. student performance data, reflective feedback, comparative assessments). Doing so may mitigate readers’ perceptions of pedagogical risk, which is reflected in some of the broad, illustrative concerns mentioned earlier that instructors may have about time investment, classroom control, or learning trade-offs. As perceived risk decreases and perceived value increases, the likelihood of adoption rises. While it can be difficult to fully track in-class use of new activities, Schmidt-Wilk (2019) notes that evidence of classroom uptake is one meaningful indicator of the impact of a publication.
Although some research has begun to examine how teaching innovations are presented in specific outlets, such as Mesny and Dubé’s (2025) recent exploration of teaching innovations in the Journal of Management Education, there is a notable dearth of research on the type(s) of evidence of effectiveness used to introduce new management-related in-class experiential exercises across the field. This is surprising, given the importance of sections that overview new teaching innovations to many management education journals. Moreover, as Mesny and Dubé (2025: 20) caution, insufficient rigor in the presentation of new teaching innovations (where the authors explored rigor in the context of both the integration of relevant literature in support of an innovation and evidence of effectiveness) can undermine legitimacy.
Yet, what is effectiveness and what constitutes robust evidence of effectiveness, with respect to experiential exercises? To guide our answer to the first part of this question, we turned to the Oxford Learner’s Dictionaries (2026, no pagination), which defines effectiveness as “the fact of producing the result that is wanted or intended; the fact of producing a successful result.” The second part of the question is more complex, as the type and magnitude of data required to convince readers that an innovation is “producing the result that is wanted or intended” (Oxford Learner’s Dictionaries, 2026) is influenced by contextual factors. For example, journals that publish experiential exercises have different requirements related to the nature of evidence they will consider, and it is fully reasonable that authors tailor their approach to the specifications of their target journal. Thus, the type and amount of effectiveness data needed will, to some extent, depend on publication expectations. Given this, we feel it is important to expressly state that our research is neither intended as a criticism of the experiential exercises published in the field nor of the requirements of the journals that provide an outlet for our management education colleagues to share their creations with the community. Rather, our objective is to contribute to the literature on experiential learning and pedagogical innovation by documenting how authors substantiate the effectiveness of their management-related, in-class experiential exercises and identifying the common types of evidence they use. Through these insights, we offer instructors a clear portrait of the different ways in which authors who have published recent experiential exercises demonstrate the impact of their innovations, with the hope that this will help authors to select appropriate and rigorous strategies when collecting evidence of effectiveness, based on the needs of their future projects.
Regrettably, a lack of clarity on the strategies that can be used to demonstrate the effectiveness of experiential exercises may also function as a barrier to entry for authors looking to publish their management-related experiential activities for the first time. As Middleton (2024: 4) aptly observes, “[m]anagement research has typically been a ‘second career.’” Moreover, business and management faculty can face a daunting pressure to publish (e.g. Alday, 2025; De Rond and Miller, 2005), which may limit the time they have at their disposal to delve into the norms and expectations for collecting data to support the effectiveness of their teaching innovations. That said, Mason et al. (2024) argue that pedagogy, pedagogical innovation, and management education research more generally are critical areas of our profession to support.
It is quite understandable that time constraints and other pressures may impede educators who are unfamiliar with the management education literature from exploring how to craft their data collection to document and provide evidence for their teaching innovations, so as to publish and share them with the field. A structured framework, drawn from current practices in the field, therefore represents a practical resource to address a critical gap in the management education literature, ultimately pushing conversations and reflections regarding the presentation of evidence in support of new experiential exercises forward.
Drawing on our own experiences of both designing and publishing in-class experiential activities, we recognize that we, as authors, experience a strong sense of personal investment and attachment to the materials we create – a sentiment that is likely shared by many of our fellow management educators. In addition to improving quality of evidence, rigorous data collection may prompt creators to actively seek different perspectives on its effectiveness. This encourages reflexivity, defined by Cunliffe (2016: 741) as “[q]uestioning what we, and others, might be taking for granted – what is being said and not said – and examining the impact this has or might have” about the new in-class exercises we introduce to our students.
To shed light on this underexplored topic, we selected several well-respected journals that publish instructional innovations on management-related topics and compiled a database of 185 articles published between 1 January 2018 and 1 June 2024. As experiential activities may take many forms (e.g. Foster et al., 2021; Wright et al., 2022), it is likely that the evidence required will differ based on the type of innovation presented. As such, it was first necessary to clearly define the scope of the present project. We opted to constrain our focus to in-class exercises, 1 allowing us to capture a wide breadth of in-class activities (games, role-plays, simulations, etc.), while excluding experiential learning activities that center on interactions among students or other stakeholders occurring primarily outside the classroom (such as at-home assignments or service-learning projects). We then read and coded each article to isolate the strategy (or strategies) used to demonstrate that the exercise was effective.
Methods
Article search process, inclusion/exclusion criteria, and sample
To construct our database, we selected a number of peer-reviewed journals that publish instructional innovations and experiential activities on management-related topics – namely, the Journal of Management Education, Management Teaching Review, Organization Management Journal, Decision Sciences Journal of Innovative Education, Simulation & Gaming, The International Journal of Management Education, and The Journal of Education for Business. 2 We chose these journals in consideration of the types of articles that they publish (i.e. outlets that include articles that introduce experiential exercises, though the length and format requirements of such articles may vary by journal) and the applicability of their aims and scope for our project. Moreover, as the Academy of Management Learning & Education has published at least one article centered on an experiential exercise (see the article by Schyns et al., 2011), we opted to include it in our search process for completeness, despite the fact that it is not a specific submission type at the journal. All of these journals are ranked by SCIMAGO and seven are included in the Australian Business Deans Council Journal Quality List.
We then performed a search via the website of each journal for articles meeting at least one of the following criteria: (1) articles explicitly labeled as instructional innovations or as an equivalent section for a given journal (such as teaching briefs), (2) articles that present specific experiential classroom activities, such as simulations, or (3) articles that reference experiential learning or related terms. As our goal was to capture current practices and trends in management education, we limited our search to new experiential exercises focused on management-related topics and decided to consider articles assigned to a journal issue between 1 January 2018 and 1 June 2024. This provided us with 6.5 years of data.
Through this initial process, we identified 230 potential articles, which we compiled into an Excel database. Of those, 45 were ultimately excluded from our analysis for one of the following three reasons: (1) the article focused on experiential learning; however, it was not a new in-class exercise or it did not have a substantive in-class component (n = 28; such as articles describing an entire course or module, or a resource review), (2) the article presented an experiential activity, but the subject was outside the scope of the present research (n = 8; such as marketing exercises), and (3) the article was an empirical study (n = 9).
Our final sample comprised 185 articles from seven peer-reviewed journals (Table 1) that regularly publish in-class experiential exercises on management topics. These articles were authored by 374 scholars (averaging 2.02 authors per article) – the majority of which were affiliated with institutions in North America (n = 290 from the United States, n = 28 from Canada), with 36 authors affiliated with institutions in Europe, 10 affiliated with institutions in the Middle East, 6 affiliated with institutions in Oceania, and 4 authors affiliated with institutions in Asia.
Sample composition by journal.
Note. Although Academy of Management Learning & Education (AMLE) was included in the search criteria, no articles met the inclusion requirements; therefore, the sample is limited to the seven journals listed above. Despite the fact that AMLE was included in the search, as no articles are included in the final list, we consistently refer to our sample within the manuscript as drawn from seven journals.
Coding
To systematize our coding process, we developed and continuously updated (as needed) a codebook with definitions of strategies used by authors to demonstrate the effectiveness of their in-class experiential exercises (Table 2). First, drawing on our knowledge of the experiential exercise format across a variety of pedagogical journals, as well as our own experiences, we composed a list of strategies that we anticipated would emerge during the data collection process. Thus, in the spirit of self-reflexivity (Cunliffe, 2011), we fully acknowledge that our personal experiences designing experiential activities, as well as our engagement with the management education literature as readers of pedagogical journals that publish such innovations, shaped our approach to and interpretation of the strategies. That said, the codebook was developed iteratively as we added new codes, refined definitions to better reflect ways in which strategies were used by authors, and collapsed codes when relevant throughout the project, which sometimes required us to re-assess articles to ensure that codes were consistently applied across the sample.
Codebook – Strategies for assessing the effectiveness of experiential exercises.
In Figure 1, we offer an organizing framework that groups the final list of strategies into several categories, based on (1) the data source – differentiating information provided by instructors, students who participated in the activity, and additional stakeholders (such as colleagues and external judges), (2) the type of assessment – discerning formally structured feedback (e.g. collected via pre-test/post design designs or through surveys measuring self-reported learning and perceptions) from informally structured feedback (e.g. student comments during a debriefing session) as forms of evidence of effectiveness, and (3) the level of preparation required for data collection – ranging from minimum to more extensive efforts.
Strategies for assessing the effectiveness of experiential exercises – a snapshot.
For example, authors will often share their own observations of running the activity that highlight ways in which they have seen students learn from and enjoy the exercise. In terms of our framework, this represents an instructor-generated informally structured strategy, as it is grounded in the author’s perceptions and recollections.
As an illustration of student-generated data, authors may distribute a survey to students after the exercise, asking questions about how the activity helped them to learn certain concepts and/or their enjoyment of it. Other times, they might report spontaneous feedback offered by students about the exercise. In terms of our framework, the former represents a formally structured measure (a questionnaire designed to collect information on perceptions or learning), whereas the latter is an informally structured indicator of the impact of the exercise.
Finally, authors can also collect data from other stakeholders – such as clients, external judges, or by inviting colleagues not involved in running an activity to attend a session, observe the exercise, and take notes. As each of these strategies could be crafted as a formally or informally structured assessment, we group them together in our framework under the heading of stakeholder-generated data.
A third category emerged during the coding process, capturing the degree of preparation required to collect the data. For example, a pre-test/post-test design necessitates laying more extensive groundwork (choosing measures, creating a survey, developing a protocol), as compared to reporting spontaneous student feedback. Importantly, however, this is not a judgment of whether one strategy is better than another, but rather an observation/distinction based on the depth of the preparation required for different data collection efforts.
The coding process followed a series of steps. First, at the outset of the process, two of the authors independently read the same 10 articles from the database, then met to discuss and compare reflections. Our goal at this stage was to refine our codes and definitions, clarify our processes, and solidify our inclusion/exclusion criteria. This discussion was very fruitful – as examples, we agreed to add a new code, “longitudinal,” to capture effectiveness data collected across different phases of an activity and decided to record the type of experiential activity within the database, using whenever possible the description/terms presented by authors.
One of the authors then embarked on coding the remaining articles. However, we incorporated “spot checks” on two occasions. The first was in the preliminary stages of the coding process – the author selected 10 papers and asked another author to read and code them. The second was when the coding was complete – the author again selected 10 papers and asked the same team member to code them as well. This provided two opportunities to meet, confer, and validate our approach and resembles the validation strategy of “peer examination” (e.g. Merriam and Tisdell, 2016: 249), though in our case the colleague providing feedback was also a member of the research team. Overall, the team members were well-aligned and made minor modifications, as needed. This systematic process ensured consistency in the coding and provided opportunities for reflexivity and transparency among team members.
Results
Of the 185 articles in our sample, 149 (80.5%) included at least one measure of evidence of effectiveness. Across these articles, we coded 332 strategies, as summarized in Table 3.
Prevalence of codes by strategy.
Fifteen articles presented observations of both learning and enjoyment, and thus both types were coded for the same article.
Eight articles used post-exercise surveys that included questions related to both learning and perceptions (as determined by the coder), and thus both types coded for the same article. Percentages are calculated by dividing the number of times a strategy was coded by the total number of codes assigned.
Taking the full sample into account (N = 185), authors used an average of 1.79 strategies (SD = 1.36) to demonstrate the effectiveness of their experiential activity, with 36 articles presenting no evidence, 48 articles integrating one strategy, 49 articles using two strategies, 30 articles mobilizing three strategies, 14 articles incorporating four strategies, and 8 articles offering five pieces of evidence in support of the effectiveness of the exercise.
As shown in Table 4, we find that authors rely primarily on student-generated strategies. Nearly two-thirds of all strategies coded in the study (n = 208, 62.7%) reflect student-generated evidence, whether formally structured (n = 164) or informally structured (n = 44).
Prevalence of codes by source and type.
Note. Based on 332 codes across 149 articles where effectiveness data were presented. Articles where effectiveness data are not presented are excluded from this table.
Instructor-generated observations represent the most frequent single strategy used across the sample (n = 116, 34.9%; see Table 3). These typically involve authors reporting on their perceptions of the impact that an activity has on student learning and/or the degree to which participants appear to be engaged in the exercise. By contrast, stakeholder-generated strategies are rarely used (n = 8, 2.4%).
Common approaches to student-generated data
As the majority of the strategies coded in our study stem from student-generated measures, we next explored the most common student-generated data used by authors. Focusing first on formally structured measures, we note that articles in our sample drew more frequently on strategies requiring greater data collection preparation. Within that category, the most common strategies included the assessment of qualitative comments collected via data collection efforts (n = 31, 8.4% – though the depth to which these comments were analyzed varied), self-reported perception surveys (n = 24, 6.5%), and pre-test/post-test designs (n = 15, 4.1%). Nonetheless, strategies requiring less extensive data collection preparation were also widely used, with a particular emphasis on data from a graded assignment (n = 26, 7.1%), examples of output (n = 18, 4.9%), and references in course evaluations (n = 12, 3.3%).
Regarding informally structured measures, we observe that authors exclusively used strategies requiring minimal data collection preparation, with the most common approach being the collection of student feedback during the activity (n = 20, 5.4%).
Configurations of effectiveness measures
Of the 149 articles that included at least one measure of effectiveness, 72 articles limited their evidence to one or more strategies from a single source and type (Table 5). The remaining 77 articles offered strategies from more than one source and type, with 62 articles (41.6%) including strategies from two categories and 15 articles (10.1%) spanning three categories.
Prevalence and combinations of strategies by type and source (for articles that include at least one formally structured or informally structured strategy).
Note. Based on 149 articles in which at least one effectiveness strategy was coded.
The most prevalent groupings include both instructor-generated informally structured and student-generated formally structured strategies (n = 39), both instructor and student-generated informally structured measures (n = 15), and the most popular three-way combination capturing instructor-generated informally structured measures with both student-generated formally and informally structured strategies (n = 10).
Mapping experiential innovation types
Although we constrained our study to in-class experiential exercises, we recognize Wright et al.’s (2022: 988) argument that many different types of activities are grouped into the general category of “classroom activities” in the literature. The authors point out that by grouping these activities together, we risk overlooking important nuances in students’ experiences. Thus, to further explore our data, we wondered whether the most common strategies used to demonstrate effectiveness would vary as a function of the type of exercise presented in an article. Given the breadth of experiential activities identified in the literature (e.g. Foster et al., 2021; Wright et al., 2022), we sought to ascertain the most widespread types of experiential activities or characteristics within our sample, which we could then use to compare and contrast the largest number of articles from our database.
Reflecting on the sample, we determined that there were four main activity types or characteristics that would allow us to differentiate the largest number of exercises: (1) articles presenting in-class simulations (n = 32), (2) articles that include an arts-based or media component (n = 27), (3) articles introducing games (n = 24), and (4) articles with role-play components (n = 23). While these four categories do not capture all types and characteristics of innovations in our sample, and the assignment of an article to a category is based on the professional assessment of the researchers, this process allowed us to compare 106 articles, or 57.3% of the total dataset (Table 6).
Prevalence of codes for in-class simulations, arts/media, games, and role-play activities.
Note. In-class sim = in-class simulation.
Authors introducing games employed the largest number of strategies into their articles (M = 2.04, SD = 1.40), followed by those featuring arts/media (M = 1.96, SD = 1.26), in-class simulations (M = 1.59, SD = 1.50), and finally, role-play activities (M = 1.09, SD = 1.04). To ascertain whether any of the differences in means were statistically significant, we ran six independent samples t-tests. The results reveal that the mean number of strategies used to argue the effectiveness of game-based activities was higher than the average for role-play activities (t = −2.66, df = 42, p = .01, two-tailed). Moreover, the mean number of strategies of effectiveness included in the articles presenting activities featuring an arts/media component was higher than the average number of strategies used for role-play exercises (t = 2.70, df = 48, p = .01, two-tailed). The differences among all other pairs were not statistically significant.
Interestingly, role-play articles were also most likely to present no supporting evidence of effectiveness (n = 9, or 39.1% of articles in that category), followed by in-class simulations (n = 8, or 25% of articles in that category), and arts/media articles (n = 3, or 12.5% of articles in that category). Articles that introduced games were least likely to present no supporting evidence of effectiveness (n = 3, or 11.1% of articles in that category).
Discussion
Experiential educators navigate several roles in the classroom (Kolb et al., 2014). This is not an easy task – the effective facilitation of experiential learning activities requires a keen awareness of the interconnectedness of educator, participant, and contextual characteristics, to ensure student learning and support the well-being of all involved (e.g. Lundgren et al., 2023; Tomkins and Ulus, 2016; Wright et al., 2019, 2022). Experiential educators must therefore be mindful of the potential risks associated with the integration of experiential activities in the classroom (e.g. Bradford, 2019) and proactive in their attempts to mitigate them (e.g. Kisfalvi and Oliver, 2015; Wright et al., 2019). In this article, we understand “risk” primarily as the perceived risk faced by prospective adopters (i.e. educators considering implementing a new exercise), particularly in terms of time investment, classroom outcomes, and student well-being.
Adding to this complexity, research suggests that instructors who adopt existing exercises often invest substantial time to tailor them to their own styles and contexts (Agogué and Robinson, 2021). Therefore, for educators who venture beyond traditional methods to design and publish experiential activities, providing convincing evidence of the activity’s effectiveness can help alleviate risk-related concerns. This evidence increases the likelihood that fellow educators will feel confident enough to invest the time to pilot these exercises in their classrooms.
We build on this premise by documenting the approaches used by authors to establish appreciation and usefulness of their in-class experiential activities. In so doing, we offer both a portrait of current practices and a useful framework of strategies (by source, type, and preparation required) that prospective authors may use as they plan their upcoming data collection efforts. While our findings may also inform educators, reviewers, and editors interested in evaluation practices, our central contribution is to provide actionable guidance for prospective authors in planning their data collection and, crucially, in communicating evidence of effectiveness.
At the heart of the matter: Student learning
From the perspective of prospective authors, it is particularly important to reflect on and collect rigorous evidence to demonstrate the degree to which the activity helps students to learn course concepts. However, we would be remiss to not acknowledge the importance of assessing the impact of pedagogical interventions more generally, regardless of whether the ultimate objective is to publish them in academic journals. Ford et al.’s (2018) recent review of the literature (primarily in organizational contexts) on transfer of training underscores this point – in their article, the authors highlight the breadth of personal, design, and organizational variables that can impact the degree to which training transfers. This emphasizes the need to be proactive in evaluating its effects.
Foster et al. (2021) advocate for designing assessments that are strongly aligned with an exercise’s learning objectives, to track student learning. While exercise output and well-designed assessments can certainly be used as evidence of effectiveness for instructional innovation articles (see Figure 1), the need to ensure the relevance of pedagogical endeavors extends beyond those measures – the literature provides excellent guidance for management educators on how to achieve this (see, for example, Foster et al., 2021: 74). For prospective authors, aligning assessments with learning objectives also strengthens how evidence of effectiveness is communicated.
Instructors can also turn to the training and assessment literature for inspiration. Frameworks may be used, such as Kirkpatrick’s model, which differentiates several levels through which the impact of training may be explored (e.g. Kirkpatrick and Kirkpatrick, 2007) – Level 1: “Reaction,” Level 2: “Learning,” Level 3: “Behavior,” and Level 4: “Results” (see, for example Kirkpatrick and Kirkpatrick, 2007, who offer guidelines and examples of questions that can be used to assess each level in organizational training contexts). Comparing our portrait of current practices to this model, we observe that the strategies mobilized in our sample to evaluate the effectiveness of experiential exercises appear to fall under Levels 1 or 2, defined as “the degree to which participants find the training favorable, engaging and relevant to their jobs” and “the degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training,” respectively (Kirkpatrick and Kayser Kirkpatrick, 2016: 10). Given that the exercises in our sample are conducted within one or more course meetings in a given semester, this is unsurprising. At the same time, it is encouraging as it reinforces the argument that learning-focused data is achievable by way of a number of commonly used data collection strategies (see Figure 1).
While student perceptions of the activity and/or learning provide valuable information to educators and researchers, the literature also reinforces the significance of considering how learning is measured beforehand, as this impacts the conclusions that can be drawn from the data. Indeed, speaking of business and management education (BME) research more generally, Bacon (2016: 703) contends that “the confusion between perceived learning and actual learning has been one of the greatest impediments to the progress and legitimacy of BME research” (see Bacon, 2024; Bacon and Stewart, 2022 and Mesny and Dubé, 2025, for more on this point).
Exploring how authors present evidence of effectiveness: Key insights and take-aways
We reviewed and coded 185 articles presenting new in-class experiential exercises, to document the different strategies used by authors to demonstrate the effectiveness of the activities and synthesize the current practices in the field. Several interesting take-aways emerged, which carry practical implications.
First, the vast majority (80.5%) of the articles included at least one measure of effectiveness. Yet, while our codebook included 20 different strategies, we found notable variance and distribution of the evidence used across the sample. For example, we observed a heavy reliance on student-generated data, for both formally (n = 164) and informally (n = 44) structured strategies. The focus on student data is somewhat expected – as shown in Figure 1, there is a greater breadth of strategies that draw on participant data to choose from. Nonetheless, the most common method used by authors to argue the effectiveness of their exercises is instructor observations of enjoyment or learning derived from an activity, our sole instructor-generated strategy. This suggests that instructor observations are an important way that authors assess and communicate the impact of their activities. However, these observations are highly subjective, and the amount of space allocated to them in articles varies, ranging from a few words to an entire section. To refine this common strategy, instructors may focus on systematizing their observations. For example, instructors could record them across multiple iterations of an activity, to then code and analyze the data. This systematic approach would increase the rigor of using instructor observations as evidence. It would also provide a more reliable foundation for arguing the effectiveness of the experiential exercise.
Notably, two-thirds of the articles in our sample that included evidence of effectiveness used two or more strategies to demonstrate the value of their activities. Of those, 51.7% incorporated evidence from at least two of the six categories presented in Figure 1. This is somewhat lower than the proportion from Mesny and Dubé’s (2025) recent study of teaching innovations published in the Journal of Management Education, where 81% of the 36 articles in their sample included evidence of effectiveness employed at least two types. However, this discrepancy may be explained by the different coding and sampling foci of the two studies. Importantly, in both cases, we see that authors have a tendency to rely on multiple pieces of evidence to argue for the effectiveness of their innovations.
Second, we recognize that journals often differ regarding the norms and expectations around the evidence required to demonstrate the effectiveness of experiential activities (see e.g. Lund Dean and Forray, 2014), and it is therefore fully reasonable that authors tailor their evidence to the requirements of their target journal. However, while differing expectations may partly account for variations in our findings, we argue that a stronger focus on collecting robust evidence of effectiveness would benefit all stakeholders, as this would not only help authors better communicate the effects of their exercises on student learning but also mitigate some of the key risk-related concerns of potential end-users, particularly when evidence is clearly and convincingly communicated.
That said, we are mindful that more than four-fifths of the authors of the articles in our sample are affiliated with institutions in North America and that over three-fourths of the articles in the sample are written exclusively by authors affiliated with institutions in the United States. This suggests that the bulk of the instructional innovations published by our sample were developed for use within a North American classroom context.
The extant literature suggests that this is not a new observation. Beatty and Leigh (2010) analyzed the content published in three management education journals – Academy of Management Learning & Education; AMLE, Management Learning; ML, and the Journal of Management Education; JME – between 2002 and 2005. Although not specific to instructional innovations, as only a quarter of the articles from one of the three journals in the sample (JME) were experiential exercises, their findings reveal a notable percentage of authors from US-based institutions in content at both JME (82%) and AMLE (79%) during that time frame, with a much smaller percentage in ML (21%).
This is important, as norms related to evidence of effectiveness for instructional innovations are collectively shaped by both journal expectations and the authors themselves through their publications. In essence, methodological choices are not neutral; they contribute to shaping norms about what “counts” as credible evidence in management education. Authors who aspire to publish their experiential exercises should therefore be intentional about not only what conversations they join but also the standard of evidence they are helping to institutionalize in the field through their work. For prospective authors, this also underscores the importance of being intentional about what evidence is collected as well as how that evidence is communicated.
Building on this, we acknowledge that our synthesis provides a snapshot of the data collection strategies mobilized to assess evidence of exercise effectiveness that is drawn, in large part, from a similar context. Thus, prospective authors should strive to collect comprehensive data that are developed iteratively, drawn from diverse sources and methods, and closely tied to learning objectives (see, for example, Mesny and Dubé, 2025; Schmidt-Wilk, 2010) to evaluate their exercises, while also considering how such evidence can be effectively communicated to intended audiences. We hope that colleagues in the management education field will consider our framework to be a beneficial resource, supporting the design of robust data collection efforts, but we do not intend it to be prescriptive or limiting. Instead, we invite scholars from a wide range of institutional and geographic contexts to extend, challenge, and enrich this work. In particular, we encourage contributions from regions and perspectives that are underrepresented in our sample, as expanding the diversity of voices in the field is essential for advancing more inclusive, contextually grounded, and methodologically rigorous understandings of instructional effectiveness. This aligns with Cunliffe’s (2016) work on the critical role that reflexivity plays in the field. Thus, our framework is intended as a supportive tool for authors who plan to collect and communicate evidence of effectiveness, rather than a prescriptive standard for all educators.
Third, we wondered whether the number of strategies used by authors might differ based on the type of exercise presented. To this end, we identified four categories of activities that allowed us to compare and contrast the largest number of articles in our sample – namely, in-class simulations, role-plays, exercises with a media/arts component, and games. Our results revealed that articles introducing games integrated more evidence of effectiveness than articles featuring role-play components, and that in-class exercises incorporating arts or media elements also included more evidence of effectiveness than role-plays. Moreover, role-play articles tended to include the least amount of evidence and were most likely, of the four categories, to fail to include any evidence at all. Intrigued by this pattern, we reflected on possible explanations.
In their article, Wright et al. (2022) group a number of experiential activities into categories, based on two dimensions – potential to engender negative emotions and the degree of skill needed to facilitate it. Interestingly, role-play did not cleanly load onto a specific category. Management educators use role-play in various ways in the classroom (from enacting simple situations to stimulating complex, emotionally charged interactions), and the effectiveness of some role-plays may therefore be challenging to assess in the moment, as deep learning from the experience may be long-term in nature (K. Lund Dean, personal communication, 2025). Indeed, Pekrun et al. (2017) argue that individuals can experience many emotions over the course of epistemic activities. This can deeply influence how instructors facilitate role-play experiential activities as well (e.g. Fairbanks Taylor, 2018). Thus, it is possible that the intricacy and riskiness of role-play activities may impede management educators’ ability to collect immediate evidence of their effectiveness. This may also increase perceived risk for instructors, particularly when evidence of effectiveness is limited or difficult to communicate.
Given the widespread use of role-plays in management education (Clack and Ellison, 2018; Stevens, 2015), another possibility is that authors underestimate the importance of collecting evidence of their effectiveness, in particular when they seek to reproduce low-stakes interactions among students via experiential exercises in the classroom. While our data do not permit us to assess this potential explanation, it does suggest fruitful avenues for future research.
Limitations and directions for future research
Three main limitations must be acknowledged. First, as the evidence needed to demonstrate the effectiveness of instructional innovations will likely vary by activity type, we recognized the importance of clearly defining the scope of our project and constrained our investigation to in-class experiential exercises. Given this, our findings may not be generalizable to all types of experiential activities. This limitation, however, opens the door for an interesting opportunity for future research. Indeed, expanding the research focus to other types of instructional innovations (e.g. service-learning projects where the majority of the interactions and work are done outside of the classroom) would provide direction for more authors who aspire to collect evidence on, and ultimate publish, other innovations in the field.
Second, while our sample includes 185 articles from seven well-respected and peer-reviewed journals, there are other outlets where authors can publish their business and management-centered experiential exercises. Though sizable, our sample is comprised only of articles from the journals we included in our search process. Future research may consider other journals and sampling strategies, and/or broaden the scope of disciplines, which would expand the analysis to journals in other fields that publish instructional innovations (such as the Marketing Education Review).
We also note the need for clearer guidance on which strategies, and how many, might be most useful for internal analysis and publication, an area that offers promising directions for future research. Moreover, scholars may expand our framework to explore the types of data that provide the most compelling evidence of effectiveness, in consideration of different characteristics of an experiential exercise (such as its potential for stirring negative emotions, as discussed by Wright et al., 2022).
Conclusion
Management educators regularly innovate in their classes, designing in-class experiential exercises to help students better grasp course concepts and achieve learning objectives. Yet, these innovations do more than transmit content. They shape how learning is experienced, interpreted, and understood in practice. Recognizing the important contribution these make to the literature, several business and management education-focused journals have specific sections dedicated to the presentation of engaging and effective instructional innovations. Through these, authors can share their experiential activities with colleagues in the community, who can then adopt and adapt them for the courses. In this way, instructional innovations function as both teaching tools and an important context where assumptions about learning, effectiveness, and evidence are constructed and reproduced within the field.
Beyond presenting an overview of the mechanics and materials to run the activity, authors detail their experiences and provide arguments to convince readers that the exercise resonates with participants and facilitates learning. However, what counts as “evidence” of effectiveness is neither neutral nor fixed. It reflects specific assumptions about what learning is, how it can be observed, and whose perspectives are privileged. While some research has begun to explore how evidence of effectiveness is presented in specific outlets, such as Mesny and Dubé’s (2025) study centered on the Journal of Management Education, little research has examined the strategies to demonstrate effectiveness of new in-class experiential exercises, as presented in business and management education-focused journals more broadly. By foregrounding these practices, we shift attention from whether teaching innovations “work” to how claims of effectiveness are constructed, justified, and made credible within the management education community.
In this study, we identify and code a sample of 185 instructional innovation-style articles published over a 6.5-year period across seven peer-reviewed journals that publish in-class experiential exercises on management concepts. Rather than treating these articles as neutral reports of pedagogical success, we approach them as artifacts of how educators make learning visible, evaluable, and shareable. In so doing, we contribute to the experiential learning and management education literatures by addressing a notable gap in our understanding of the current practices used to demonstrate evidence of effectiveness, as well as providing a portrait of the different strategies that authors may select from, based on the source (instructor, student, stakeholder), type (formally and informally structured), and degree of preparation required for the data collection. Importantly, these strategies capture and shape what is recognized as learning and what is overlooked, inviting a broader reconsideration of how innovation is evaluated within management education. In contexts marked by different constraints (e.g. technological, institutional, and resource-related), demonstrating effectiveness may rely on different forms of evidence, raising questions about whose innovations are recognized and legitimized, while shifts between face-to-face and digital environments increasingly reshape how innovations are enacted and how their effectiveness is assessed.
Our hope is that our findings serve as a resource to prospective authors, helping them select robust strategies for their data collection efforts which, ultimately, may facilitate the publication and adoption of their experiential activities. At the same time, we encourage scholars, reviewers, and editors to critically reflect on the assumptions embedded in these practices and to remain open to diverse and context-sensitive approaches to evaluating teaching innovations.
Footnotes
Acknowledgements
We would like to sincerely thank Anaïs Scuussel, Mohamed-Khalil Ferchiou, and Claudèle Fortier for their work as Research Assistants on earlier versions of the research project leading to this manuscript. We would also like to thank Jennifer S. A. Leigh, Anne Mesny, and Kathy Lund Dean for their feedback during the development of this manuscript.
Ethical considerations
Not required, since only publicly available information was utilized.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors gratefully acknowledge an Insight Development Grant provided by the Social Sciences and Humanities Research Council (SSHRC) of Canada (Grant No. 430-2020-01035) received by Marine Agogué (Principal Investigator) and Melanie Robinson (Co-investigator).
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.