Conducting and Writing a Structured Literature Review in Human Resource Development

Differences Between the SLR and Other Review Types

Literature reviews can be a useful tool in the development and advancement of theory representing the first step in theory development models (Callahan, 2014; Rocco et al., 2022; Snyder, 2019). Four types of reviews—the SLR, meta-analysis, scoping review, and integrative literature review—involve an analysis of literature for the purpose of synthesizing the literature and generating new insights. Each type of review was mentioned in Wang’s editorial (2019) and published in HRDR (Cho, 2022). The distinctions between an SLR and these three other types of literature reviews are summarized in Table 1 and outlined in this section. We provide distinctions across such elements as the purpose, data selection process, types of data, method of analysis, and presentation of findings and discussion.

OPEN IN VIEWER

Table 1.

Distinctions Between a Meta-Analysis, Scoping Review, Integrative Literature Review, and Structured Literature Review.

Points of Distinctions	Meta-Analysis	Scoping Review	Integrative Literature Revactiew	Structured Literature Review
Purpose	Investigate statistical significance among empirical studies on the same topic	Identify breadth or scope of existing research on a topic	Integrate representative literature or different literature streams to make meaning	Comprehensive collection of literature meeting criteria to discover themes, patterns, trends, or gaps
Data selection process	Databases; specific studies	Databases, reference lists, and organizational documents	Databases; guided by the researcher’s point of view on the topic	Databases, specific journals, reference lists, google scholar ‘cited by’ function; researcher’s perspective bracketed
Types of data	Peer-reviewed quantitative empirical articles	All publications fitting the purpose except conceptual or theoretical	Representative literature on the topic	All publications meeting selection criteria based on the purpose
Analysis of data	Quantitative	Qualitative or quantitative	Critical reflective analysis	Content analysis, thematic analysis
Presentation of findings and discussion	Results of comparisons of elements of the empirical studies	Overview or map	Coherent conceptual structuring	Findings of the data analysis process

First, the SLR and the meta-analysis differ on two main issues. The SLR is primarily a method of reviewing content qualitatively, while the meta-analysis is primarily a method of statistical analysis (Cooper et al., 2009). Glass (1976), who coined the term meta-analysis, defined this method as “the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings” (p. 3). Additionally, the SLR treats the text in the documents as data to be analyzed to uncover patterns, trends, or gaps around a research topic. These literature types can include conceptual, empirical, peer or non-peer reviewed articles, chapters, and professional work. In contrast, meta-analyses seek to explore statistical significance among similar empirical studies on the same topic (Cooper et al., 2009). In a meta-analysis, researchers usually focus on effect size across several studies, including the heterogeneity of the effects and variation in the effect size (Cheung & Vijayakumar, 2016). For example, when individual medical studies may not show a statistically significant difference between several treatments, a meta-analysis of those studies could indicate such difference. There are potential issues with this approach. One issue is publication bias, or “the publication or non-publication of studies depending on the direction and statistical significance of the results” and the suppression of certain types of data (Rothstein et al., 2006, p. 3). The SLR provides one alternative for diminishing such issues by collecting documents using keywords and search methods that strive to be inclusive and may not support a preconceived outcome.

Second, the SLR and scoping review processes are similar in identifying relevant material for review, developing material selection criteria, and extracting textual data. However, there are three key differences between an SLR and scoping reviews: (a) the questions they seek to answer, (b) the use of conceptual or theoretical frameworks, and (c) the formulation of implications. The SLR addresses questions concerning the state of a phenomenon or concept, definitions, trends, issues, research intent, and long-term implications. A scoping review is primarily intended to provide an overview or a map of existing research identifying the volume, characteristics, and focus of the literature, in an area of interest (Munn et al., 2018). Therefore, a scoping review is unlikely “to address very specific research questions” (Arksey & O’Malley, 2005, p. 20) and, instead, seeks to develop a description of how the topic has been discussed (and evolved) in the extant literature. An SLR may also comprise conceptual or theoretical literature whereas such literature is excluded from scoping reviews and meta-analyses. Conceptual or theoretical literature is regarded as subjective and believed to expose the research to bias in a scoping review (Arksey & O’Malley, 2005; Munn et al., 2018). Finally, an SLR often develops implications for research, practice, theory, and policy, yet scoping reviews rarely intend to provide implications for practice or policy (Munn et al., 2018).

Third, the approaches to the SLR and the integrative literature review differ. The SLR uses a structured and detailed method that can be replicated while the integrative literature review has no “well-established format” (Torraco, 2005, p. 359). The integrative literature review relies on the researcher’s perspective on the relationship among the concepts in the research. This relationship can be steered or channeled either by “a guiding theory, a set of competing models, or a point of view about the topic” (Torraco, 2005, p. 359). Ideally, the SLR is not guided by a point of view on the topic, but rather preconceived notions are bracketed so the data can speak on their own. The SLR treats the documents collected in the sample as data sources. The data is extracted and analyzed.

The SLR is based on inclusion of all documents that fit the purpose of the study and meet the sampling criteria while the integrative literature review utilizes only “representative literature on the topic” (Torraco, 2005, p. 356), though this literature is discovered in a systematic way. The contribution of the SLR depends on findings generated by the data analysis process, while the contribution of the integrated literature review weighs heavily on “a coherent conceptual structuring of the topic” (Torraco, 2005, p. 360).

Finally, we concur with Foster (Wang, 2019) that the SLR is an empirical method of research, closer in structure to a meta-analysis. Both integrative literature reviews and SLRs report on where, when, how, why, and what literature is selected (Callahan, 2010), but an SLR goes beyond data selection descriptions to include data extraction, analysis, and management procedures.

In this section, we provided the distinctions between an SLR and three types of literature reviews (Cho, 2022; Wang, 2019). These distinctions illuminate the unique characteristics of an SLR and illustrate its potential to address HRD problems and further scholarship around key concepts. The SLR is a review method HRD researchers should add to their repertoire. The SLR presents a structured process for performing a comprehensive collection of literature that contains the data to be extracted and analyzed to discover themes, patterns, trends, or gaps. Through the analysis of documents, scholars generally use content or thematic analysis, producing themes or categories and sometimes using simple descriptive statistics to support a theme. With its use of all publications meeting selection criteria, the SLR strives to be inclusive and not guided by a preconceived outcome. In an SLR, a discussion of findings always results in implications for research, practice, or policy. In the next section, we describe the process of conducting and documenting an SLR.

The Research Problem

Manuscripts intended for publication begin with a sound foundation explaining why the topic is worth exploring (Jacobs, 2011). This section is common to all articles and is often called the introduction or research problem (Rocco & Plakhotnik, 2011). The research problem includes three elements: background to the problem, problem statement, and research purpose and questions. The background to the problem situates the issue to be addressed in the manuscript in a broader social, political, economic, or other context. The background to the problem should capture the reader’s attention and facilitate understanding of how the issue is related to other issues, structures, policies, and concerns in a broad sense and in the field.

The next element of the research problem, the problem statement, situates the issue in the field and involves a progressive sharpening and narrowing of scope, all while defining terms and refining concepts. The problem statement articulates a gap in the literature and makes a case for why the issue should be explored (Merriam & Simpson, 1995). This gap could be related to theory, methodology, or an inconsistency between theory and practice, among other issues (Jacobs, 2011). The knowledge gap can also be that a topic warrants examination through a theoretical lens to shed new insight on how the topic is understood. The problem statement should indicate how the SLR can be useful in addressing the gap or need. This is sometimes done by providing a short historical overview of how the research on a phenomenon has evolved in different fields and by pointing at a lack of comprehensive knowledge of the phenomenon in a particular field (see Plakhotnik & Rocco, 2011). A gap or need can also be addressed by clearly illustrating why it is important to see how concepts are discussed in the literature (see Collins, 2015; Meeker et al., 2020). A need for the SLR can also be explained by demonstrating, through skillful use of the literature, that a larger concept, such as diversity, is addressed, but the components that make up diversity (e.g., race, gender, or sexuality) ignore a particular aspect, such as disability (see Procknow & Rocco, 2016).

The last element of the research problem, a research purpose, clearly explains what the researcher will do to address the identified gap or issue. The purpose statement should focus on one phenomenon, point to the method of inquiry, describe the extent of the literature to be reviewed, and use action verbs (Creswell & Poth, 2016; Krippendorff, 2018). We recommend beginning with, “The purpose of this SLR is to…” The purpose statement is often, but not always, followed by research questions or propositions. When creating research questions, the researcher should ask what knowledge is needed to fulfill the research purpose. For example, Plakhotnik and Rocco (2011) provided the following research purpose and questions:

The purpose of this SLR was to find out how organizational culture has been studied within the Academy of Human Resource Development (AHRD).

The following research questions guided the study:

Research Question 1

How is organizational culture defined?

Research Question 2

What are the purposes for studying organizational culture?

Research Question 3

What are the implications for practice?

Research Question 4

How has the study of organizational culture evolved in the span of 16 years? (p. 76)

Therefore, answers to these four research questions helped the authors address the purpose of the paper and guided their analysis.

Conceptual or Theoretical Framework

A conceptual or theoretical framework may or may not be appropriate when conducting the SLR and is frequently not used. When there is no framework, the research design or method follows the purpose statement. This would be the case when the framework would be composed of the same literature that comprises the sample. For example, in their SLR, Wollard and Shuck (2011) set out to identify the antecedents of employee engagement. A review of the engagement literature or a presentation of engagement theory(ies) would have been redundant because this was the very literature to be examined. The majority of SLRs we have examined in other fields (e.g., Burgess et al., 2006; Van Waart et al., 2001) do not have a framework for this same reason.

A conceptual or theoretical framework is used to support the purpose, provide a baseline to examine the topic, particular issue, or phenomenon, and can serve as the analytical framework through which the data is analyzed. When appropriate, the framework is found after the purpose statement in the SLR.

Frameworks are intended to guide how researchers interpret, make meaning of, and situate data to make the findings relevant and valuable to others. This might be done by conducting a content analysis using a rubric and searching for terms that fit the rubric parameters in each article included in the sample. For example, if the theory being utilized proposes four phases to understanding a particular phenomenon, a rubric with a description or characteristics of each phase could be leveraged to identify, quantify, and/or categorize instances that represent the phase within the sample. For instance, in seeking to understand how scholars characterized social entrepreneurship, Huyler and colleagues (2022) conducted an SLR, which revealed how pragmatic, moral, or cognitive legitimacy (Suchman, 1995) helped organizations advance the practice of social entrepreneurship. A theoretical framework could also be used as the analytical framework by developing a set or series of a priori codes (Saldaña, 2021). Another SLR example, Collins (2015), examined literature “on masculine, male-dominated, gendered, and masculinized work contexts” (p. 418). The heading of the conceptual framework supporting this purpose became: “Hegemony, homosexuality, and multiplicity: masculinity at the crux of social identity” (p. 418). Hegemony, homosexuality, and multiplicity were used to place the concept of masculinity historically and in terms of sexuality and identity. Once a conceptual or theoretical framework has been presented, it should be interwoven into the discussion and often the implications of the SLR (Rocco & Plakhotnik, 2009).

Method

Researchers in HRD and other fields have not agreed on the name for this section or element of an SLR. For example, many authors use the term “method” in their SLRs published in HRDR (e.g., Lee et al., 2020; Plakhotnik & Rocco, 2011) and other journals in such fields as health management (Shah & Robinson, 2006), culture (Meeker et al., 2020), and higher education (McGill, 2019). Some use other terms, including “materials and methods” (e.g., Coetzee et al., 1998, p. 74), “review methodology” (Burgess et al., 2006, p. 705), or research design (e.g., Collins, 2015; Procknow & Rocco, 2016). We concur with the majority of SLR users and prefer to use the term “method.”

The method presentation is structured and provides a detailed description of all design decisions and the steps taken to conduct the SLR. Structured implies that the method section contains enough information that the study can be replicated. Replication of an SLR means the material selection and data extraction and analysis steps are clearly described. Responses to questions such as who (one or all authors), when (timeframe), where (databases and other repositories), how (the process), what (sample description), why (inclusion and exclusion critieria) should be addressed to ensure replicability (Callahan, 2014). Some authors portray the method as a series of steps or phases (see McGill, 2019). For every element in the method, explanations of what was done, how it was done, and why it was done help readers gain a clear understanding of method decisions and implementation.

Findings and Discussion

In an SLR, the findings section and discussion section can be presented separately as two sections of a manuscript or simultaneously as one section. This section should present the findings in a logical manner so that the section headings and subheadings demonstrate a relationship through the order in which they are presented and through the systematic and parallel construction of the heading names. The heading names should correspond to the themes, categories, trends, or issues found. In some cases, the data are presented under headings corresponding to the research questions (see Lee et al., 2020; Plakhotnik & Rocco, 2011).

The subsections within the findings and/or discussion section should begin with a definition of the organizing concept, such as a theme, category, trend, or issue. When relevant to the purpose or research questions, researchers provide the number of articles that support each organizing concept or information about the authors of these articles, for example, gender, race, or institution. This should be followed by a presentation of the data (excerpts or citations from the literature) that support and explain the concept (Rocco & Plakhotnik, 2011). Often tables are used to summarize the data supporting a concept or to summarize all the themes presented in the section (see Collins, 2015). Figures are used to illustrate relationships between concepts, and graphs can be used to illuminate trends over time (see Lee et al., 2020; Plakhotnik & Rocco, 2011). When using tables, figures, graphs, and other visual aids, each must be explained in the text; they do not stand alone.

As in any paper, the discussion section of an SLR is where meaning is made through interpretation of the findings (Rocco & Plakhotnik, 2011). The author(s) describes what was found, the importance of these findings, and the relationship of these findings to the purpose, research questions, and conceptual or theoretical framework used to support the study (Cetin & Hackam, 2005; Collins, 2015). The discussion section should address the research questions bringing in the findings and making connections to the framework (if one is used in the paper) and extant literature.

The decision to have separate sections presenting and discussing the findings or to merge the two sections is based on the most efficient and effective way to present and discuss the findings. Merging findings and discussion is best when the discussion section seems redundant of the presentation or description of the findings. This can happen when findings need to be repeated to be discussed meaningfully. There is not one correct way to present findings or to discuss them in an SLR. This decision should also be aligned with the journal requirements for article structure. Human Resource Development Review editors, for instance, require findings to be separate from their discussion because the latter is critical to describe the significance of the study, implications for research and practice, and limitations.

Implications: Taking the Discussion into the Future

In this section, the researcher answers the “So what?” question (Rocco & Plakhotnik, 2011) by suggesting how the findings of the research could be used in the field by scholars, practitioners, and policy makers. Implications take the findings to the next level; implications are oriented into the future, into what could or should be done. Structured literature reviews always provide implications. The most common implications in HRD include implications for research, practice, and policy. If the SLR has a theoretical framework, then implications for theory must be made. Implications for practice could be divided into implications for different types of practice, for example, for leaders and for training and development, career development, and organizational development. In an SLR around schools, higher education, and other types of instructional settings, researchers suggest implications for learning and teaching. What implications to suggest depends on the topic of the SLR, findings, and the publication outlet. For example, practice-oriented journals expect implications for practice while research-oriented journals expect implications for research and theory, but either type of journal could expect both. This section could have subsections for each type of implication. Sometimes, the researchers choose to summarize their implications in a form of a table or chart. For example, Collins (2015) provided implications for research and practice and presented the latter as a table to direct readers’ attention to his suggestions. Rocco et al. (2003) used a figure to illustrate the relationship between the trends and issues discussed and their relationship to research and practice.

In an SLR, the implications and discussion section could be merged (see Meeker et al., 2020) to increase the efficiency and effectiveness of the writing. When this route is taken, often the subheadings correspond to the research questions. The text then responds first to the research question and then ends the section with implications for practice, research, or policy or the response and implications are integrated around a single point.

Common Challenges

The researcher should be ready to spend weeks, and possibly months, organizing, scanning, and analyzing the documents, conducting data extraction and analysis, and documenting the process. When not enough time and effort are put into the planning, documenting, and execution of the research, a study that could potentially make a great contribution to the field fails: The author gets a rejection letter from the journal or cannot make meaningful discussion and implications and decides not to complete the paper.

As editors and reviewers for different journals, we have, unfortunately, come across such failed attempts at SLRs. One of the distinctive characteristics of an SLR is a detailed description of the method with potential for replicability. The more detailed the description of the method is, the more replicable and credible the SLR becomes, which increases its chances for publication. The findings of replicating studies will be different because they will be based on the new literature published since the original SLR.

Too often, authors omit vital information in the method section. For example, authors fail to justify their choice of databases, time frame, and scope of their search in terms of its internationality. Also, researchers often fail to explain how they reduced their documents from several hundred to the several dozen included in the sample. Here, it is ineffective to say those documents were simply carefully scanned or read; instead, it must be clear how the researcher decided to include or exclude an article from the sample. Another frequent omission is a description of the sample. In a SLR, the sample should be described in as much detail as in any paper that uses other methods of empirical research. When empirical research involves people in its sample, editors, reviewers, and readers want to know who these people are and their relevant characteristics. Similarly, in a SLR, the audience should know details about the documents that make up the sample. Such description could provide new unexpected observations about the sample, the phenomenon under investigation, and build a bridge or continuity between the data extraction and analysis process and the findings section.

In the findings and discussion section(s) of an SLR, another challenge is to state the themes, categories, trends, or issues clearly, creating corresponding headings and sub-headings and beginning each section with a definition of the theme or category. A poorly structured and insufficient description and discussion of the findings undermines the implications that authors suggest. When the audience struggles to understand the findings and their meaning, they question if and how implications are connected to the findings.

Applications

Conducting an SLR at the beginning of a larger project allows the researcher to acquire an understanding of the state of the topic, identify major works, authors, trends, issues, questions, and breakthroughs around the topic and, importantly, what may be missing from the knowledgebase. Therefore, similar to a scoping review, an SLR is useful for students and researchers beginning a line of research on a new topic. An SLR can be used to explore the historical development of a concept, conceptualize an issue for future theory development, or explore a new, emerging concept in one or several related fields.

This method could be used for a group project in which students, with guidance of a faculty member, conduct an SLR to become immersed in the literature on a topic and to learn the method. For example, the first author, Tonette, invited a group of her graduate students to participate in an SLR that examined how qualitative research was reported in papers published in the proceedings of the Academy of HRD Annual Conference. In each selected paper, the examination of several elements (e.g., purpose, sampling, and integrity measures) was divided among the six researchers. The findings were presented at four conferences and published in six papers of the conference proceedings (e.g., Bernier et al., 2005; Plakhotnik et al., 2005; Rocco et al., 2005). This project helped students understand the SLR, the quality of published qualitative research found in the proceedings of the Academy of HRD, difficulties and benefits of working as a team, and the joy of multiple publications based on one large project. This method can be used to support the literature review chapter in a dissertation.

Implications

An SLR could be used to ask and answer epistemological and ontological questions related to the nature, purpose, definitions, and boundaries of the field. For example, HRDR, a leading journal in HRD that aims to build the theoretical and conceptual knowledgebase of the field, has a section for publishing literature reviews, called Integrative Literature Reviews. While all types of literature reviews are published in this section, naming the section for one type of literature review is problematic. Scholars may believe only an integrative review can be published, is the preferred method, or that other types are not as important.

An SLR provides an understanding of a topic and, potentially, is the initial step in theory development. Findings could provide ideas for further development of an issue leading to one or more conceptual paper(s) that suggest a model or a typology (Rocco et al., 2022). Also, an SLR could be used to enhance and sharpen HRD’s connections to both established and emerging areas of practice and research previously un- or underexplored in the field. As the field matures, using an SLR to examine a concept in terms of its use in HRD can help delineate a future research agenda that is informed by our recent, but rich, history. As HRD is a field deeply connected to and informed by other fields, understanding how a concept is used in HRD, management, education, psychology, and/or other fields may bridge the gaps between the disciplines, help set the stage for a unified definition of a concept, or create an interesting baseline for future disciplinary and interdisciplinary research. Finally, in many other disciplines, SLRs are often highly cited in new investigations, in part because of their robust nature and the extent to which they often fully overview a topic or issue. Structured literature reviews make a good foundation for new work. If more HRD scholars choose to use the SLR as a method and publish their work, SLRs could be a useful tool for enhancing and raising the profile of our journals and therefore the field.