How to Undertake an Impactful Literature Review: Understanding Review Approaches and Guidelines for High-impact Systematic Literature Reviews

Abstract

Literature reviews lay the foundation for academic investigations, especially for early career researchers. However, in the planning phase, we generally lack clarity on approaches, due to which a lot of review articles are rejected or fail to create a significant impact. The systematic literature review (SLR) is one of the important review methodologies which is increasingly becoming popular to synthesize literature in any discipline in general and management in particular. In this article, we explain the SLR methodology and provide guidelines for performing and documenting these studies. Through systematic processes, these reviews offer suggestions to synthesize literature to identify research gaps and indicate research directions. Lastly, this article serves as a guide for researchers and academics in conducting an extensive literature review.

Keywords

Systematic literature review (SLR)literature review review process review methodology guidelines

Introduction

Important aspects of a systematic literature review (SLR) include a structured method for conducting the study and significant transparency of the approaches used for summarizing the literature (Hiebl, 2023). The inspection of existing scientific literature is a valuable tool for (a) developing best practices and (b) resolving issues or controversies over a single study (Gupta et al., 2018). The evaluation of published content such as literature or news in a certain domain includes several assessments, including area, outlet and topic. These reviews are commonly undertaken within a focused research field to explain synthesis and possibly propose extensions of theories (Jeyaraj & Dwivedi, 2020). The literature review is, thus, an important part of academic study which enables the reader to (a) understand the current state-of-the-art from previous research, (b) the length and breadth of the present work and (c) to identify and explore research gaps (Xiao & Watson, 2019). Among the different types of review, such as critical and post-publication reviews, the SLR is one of the more popular approaches to conduct reviews in a scientific and justified manner (Snyder, 2019). It facilitates academicians and experts to define their research objectives based on gaps in the existing literature and then design a study to address these gaps. The systematic (also known as ‘structured’) literature review has been introduced in management research as a potential technique to examine past research and contribute to the field (Tranfield et al., 2003). High-quality SLRs encourage policymakers and entrepreneurs to make better decisions and support scholars in synthesizing literature (Kraus et al., 2020). Hence, the SLR has received attention and gained popularity within the management domain. The initial papers have established norms and suggestions for how to develop review articles in management disciplines (Kraus et al., 2020). Eventually, the SLR closely substituted conventional reviews for individual management review papers (Jones & Gatrell, 2014). The two key benefits of the SLR are transparency in gathering information, and synthesis, which result in greater objectivity and reproducibility (Tranfield et al., 2003).

When we closely examine conventional literature reviews, it is important to note that they are sometimes characterized by a lack of organization, systematization and transparency. These reviews tend to involve a high level of subjectivity in the collection of data and analysis (Kraus et al., 2020). Likewise, SLRs are not new and first emerged at the end of the nineteenth century to evaluate articles (Petticrew & Roberts, 2008). In comparing and characterizing current circumstances, we find a significant change in the evolution of these reviews over time (Hart, 2018). This is most visible in the growing accessibility of literature in broad online databases, such as EBSCO, Scopus, ABI/Inform, and Web of Science (WoS), which enabled academicians to carry out much faster and more transparent review procedures by the end of the twentieth century (Kraus et al., 2020). However, Hart concluded in their study that the SLR is a proven method for providing an overview of a topic within a larger research endeavour and is considered a stand-alone process for producing findings and evidence (Kraus et al., 2020). Commonly, SLRs serve as the foundation for clinical practice and provide guidelines to undertake future interdisciplinary research, especially for early-career researchers (Gupta et al., 2018). These evaluations can also result in the establishment of new theories and directions for future research (Higgins et al., 2019). However, Hart (2018) criticized that several review articles are published which fail to guide future work for early career researchers, and we frequently face the same issue today. For example, in conventional research programmes at universities, sometimes the SLR does not receive the same attention as methods which verify empirical results, which indicates that a significant portion of the research community is more committed to subjectively summarizing the existing literature than synthesizing it in a systematic manner (Kraus et al., 2020).

The purpose of a review is to blend and synthesize existing information (Rowley & Slack, 2004). The contexts in which literature reviews are undertaken, differ among authors in terms of both processes and findings. According to Knopf (2006) and Okoli (2015), there are three main situations in which a literature review is required:

When an independent analysis of the relevant literature is needed for a specific topic.

When an introduction to an analytical paper outlines the groundwork or foundation for hypotheses.

When undertaking the initial phase of a larger research project, such as a dissertation.

In addition, the topic of the literature review must be timely, relevant and novel to be more effective. The editors must evaluate these studies from three perspectives: generalizability, rigorous methods and contribution relevance. The outcome of these review articles should attempt to make a strong theoretical contribution through the synthesis of existing literature; simply describing prior art is not considered a theory, but it does aid in determining theoretical contributions. Furthermore, data and analysis that are collected and performed to confirm the same theoretical framework in multiple settings (geography and time) fail to provide a substantial theoretical impact. In each context, the theory must explain a subset of a phenomenon or the relationships between concepts (constructs) that no one else has addressed. To summarize, a study should attempt to signal how it is original, concise, generalizable, reproducible, falsifiable, internally consistent and able to withstand the test of time. Hence, literature reviews need to move beyond descriptions of the data and synthesize a research topic in greater depth and uncover the edges of its nomological network. It is important to note that contrasting debates in a discipline often motivate the need for undertaking an SLR.

This article examines SLR in management research, including its constraints, its present situation and prospects to offer best practices. In addition, the study is intended for researchers, doctoral scholars and postgraduate students, especially in the management domain, and serves as a discussion of specific requirements. The purpose of this article is to explain how an early career researcher can undertake an SLR, to overcome its primary flaws by comparing. Further, we also attempt to contrast it to other relevant review methods. This article attempts to serve as an initial guide for researchers to deploy the SLR methodology by providing a methodological overview that highlights information sources and examines practical relevance concepts, thereby providing directions for future research. Thus, this study will explore the approach and practical considerations associated with an SLR. It presents a brief introduction to systematic reviews in its first section. The second section focuses on the evolution of the SLR and its comparison to traditional literature. The third section describes how to conduct and write the SLR process. Lastly, the study concludes with a discussion of the impact of SLRs and draws inferences by providing a set of guidelines for scholars to conduct the SLR process.

Evolution and Comparison of Review Methods

Although the fundamental concepts and components of modern SLRs can be traced back to between the eighteenth and nineteenth centuries (Chalmers et al., 2002), during the latter half of the twentieth century, SLRs established their current form and significance in contributions to health sciences. Also, in the 1960s and 1980s, a number of studies found significant discrepancies in physician practices that were eradicated by the continuation of research and the provision of improper care as a result (Chassin et al., 1987). This gave rise to the ‘evidence-based medicine’ movement, which was found to reinforce medical practice with the findings of the best available scientific research and reduce reliance on intuition and non-scientific rules (Sackett et al., 1996).

Besides, the advent of meta-analysis as a more comprehensive statistical review method is a preferred advancement for undertaking reviews on any theory (Kraus et al., 2020; Petticrew & Roberts, 2008). The possible benefits of comprehensive reviews based on evidence have been identified by policymakers (Petticrew & Roberts, 2008). Based on an in-depth investigation of the similarities and differences between management and other disciplines, editors encourage the expanding use of SLRs in management research (Tranfield et al., 2003).

SLRs have been recognized as a useful research tool for evidence-based decision-making (Kraus et al., 2020). According to Tranfield et al. (2003), the hierarchy of evidence in healthcare reveals that SLRs provide the highest degree of testimony and provide a summary of each of the individual studies. Petticrew and Roberts (2008) evaluate and compare a single-subject study to a sole or exclusive response in a quantitative study. Nevertheless, the synthesis of multiple perspectives can help to overcome biases and produce a reliable answer to any research question. The same procedure applies to the SLR, which leads to a high level of confidence in the review article’s ability to answer a research question (Kraus et al., 2020). Additionally, there are differences between SLRs depending on how the authors analyse reviews; the quality of an SLR varies drastically. Subsequently, the author must achieve an optimal balance between the scope and comprehensiveness of their review paper, taking into consideration the abundance of literature available for analysis (Kraus et al., 2020). Another critical factor which impacts the reporting of findings is the author’s tolerance for heterogeneity during the search process (Frank & Hatak, 2014). The SLR is a type of research that synthesizes previously published data using a systematic methodology (Tranfield et al., 2003). In addition, the SLR determines, assesses and synthesizes all available research pertinent to a specific research query, subject area or phenomenon of interest (van Dinter et al., 2021).

The SLR approach is often described as a searching, screening and reporting process of the existing literature that employs a transparent and reproducible method of searching for it, evaluating its quality and compiling it with a high level of objectivity when related concepts may not be strictly theoretical, say technology artefacts (Kar et al., 2022). The SLR provides a method for evaluating the quality and quantity of evidence on a query or topic of interest. The SLR can also be classified as a domain theory or a method-based review (Palmatier et al., 2018). Furthermore, Paul and Criado (2020) categorize a multitude of sub-forms of domain-based reviews, including structured theme-based reviews, framework-based reviews, bibliometric reviews, hybrid reviews and conceptual reviews, and introduced meta-analytical reviews, also known as meta-systematic reviews. Paul et al. (2021) explain the various types of literature evaluations, which are summarized below:

Domain-based Review

This review focuses on the growth of a field, publication or subject. This may pick up an emerging discipline which has started in a related discipline and introduce this into management literature (Paul et al., 2021). For example, bio-inspired computing and swarm intelligence, which are typically from computer science literature, were introduced to business management literature by Chakraborty and Kar (2017).

Structured Review

Structured theme-based reviews emphasize the development of concepts, which may include related theories, models, constructs, contexts and methodologies (Canabal & White, 2008; Kahiya, 2018). Typically, such information is presented in well-constructed tables in conventionally structured review articles, which is helpful for new researchers in comprehending the overall procedure (Paul et al., 2021). Researchers may explore opportunities pertinent to methods, theories and constructs based on the compiled data. This category of review comprises some of the standard review articles found in the existing literature.

Framework-based Review

This review relies on the known frameworks for domain evaluation. The popular framework-based reviews include the antecedents, decisions and outcomes (ADO) framework by Paul and Benito (2018), the theories, contexts, characteristics and methods (TCCM) framework by Grover et al. (2022) and the combined ADO-TCM framework by Lim et al. (2021). These frameworks include criteria for the ‘what’, ‘why’, ‘where’, ‘when’, ‘who’ and ‘how’ (Lim, 2020). Based on these frameworks, authors will be able to construct models that will help to achieve the best level of comprehension and coverage of the topic (i.e., breadth and depth) in their review. Therefore, framework-based reviews are frequently more useful and influential than all other types (Paul & Criado, 2020).

Bibliometric Review

Bibliometric reviews examine a vast amount of published research by employing statistical tools to identify trends, citations and co-citations of a specific topic by year, nation, author, journal, technique, theory and research problem (Paul & Criado, 2020). Software tools like VoS Viewer make it possible to create a graphical bibliometric review, which is widely used in several topics, such as customer experience, artificial intelligence and text mining (Kumar et al., 2021, 2023; Varsha et al., 2021). The concern frequently observed in bibliometric analyses is that a small number of publications contribute disproportionately to the total number of citations in the analysis (Paul & Criado, 2020). Also sometimes, bibliometric analysis misses to have a detailed discussion on the synthesis of the literature beyond the visualization of the data.

Theory-based Review

This review investigates the evolution of a particular theory in a specific field (Paul et al., 2021). Examples of such reviews include the self-determination theory in marketing research (Gilal et al., 2019) and technology adoption using the unified theory of the acceptance and use of technology (Williams et al., 2015).

Method-based Review

This review examines the evolution of an individual method in a review domain (Paul et al., 2021). Illustrations of such reviews include the development of common method bias in tourism research (Çizel et al., 2020), crowdsourcing in consumer research (Goodman & Paolacci, 2017), netnography in tourism research (Tavakoli & Wijesinghe, 2019) and structural equation modelling in marketing research (Hair et al., 2017).

Meta-analytical Review

This method examines statistical assessments of previous research in a specific domain (Paul & Barari, 2022). Through the systematic procedures that underpin literature reviews, researchers can identify all relevant variables and studies in the review field before performing quantitative synthesis to determine their statistical significance and relevance. Guidelines for undertaking meta-analysis reviews are well documented recently by Jeyaraj and Dwivedi (2020).

Meta-systematic Review or Evidence Summary

This technique combines previous systematic reviews in the subject area. The latest systematic review on behavioural control, which can establish a new theory, is an ideal meta-systematic review (Ilavarasan et al., 2017). Normally evidence summaries are used for policy decisions by combining insights of different types of literature reviews on a related theme, so that synthesis of different findings can be evaluated qualitatively and documented.

Grey Literature in Systematic Reviews

Grey literature is defined as knowledge documentation that does not result from the peer-review mechanisms that govern scientific journal publication (Lawrence et al., 2014). As a result, there are now more specific and broader perspectives on grey literature (Adams et al., 2017). Additionally, Schöpfel (2010) defines grey literature as a methodical approach with diverse documentation generated at all levels of government, academia, business and industry in both paper and electronic formats that are secured by intellectual property laws and of sufficient quality to be obtained and preserved by archives in libraries or institutional repositories, but which is not subject to regulation by commercial publishers, that is, where publishing is not the main business of the knowledge organization. However, because of digitization, the scope and impact of this type of literature have increased, and the need to include it in systematic reviews has become more apparent while improved indexing and administration are becoming an increasing concern for librarians (Jeffery, 2000). In addition, the heterogeneity of grey literature makes it less responsive to traditional methods of archiving, retrieval, analysis, synthesis, bibliographic data capture, data extraction and integration (Adams et al., 2017). Kepes et al. (2012) suggest a framework for the classification of grey content that may be incorporated into a meta-analytic review based on the concept of source availability. For example, current focus innovations, such as quantum computing, metaverse and generative artificial intelligence (GAI), may benefit from the review of grey literature (Chakraborty, 2023a, 2023b; Kar et al., 2023) simply because there may not be adequate discussions on this in academic literature.

Hybrid Reviews

This review may incorporate two or more subtypes of review from the same domain. In the first stage, researchers incorporate a framework that offers future directions in a descriptive kind of literature review referred to as a hybrid review. Hybrid reviews may combine bibliometric analysis followed by thematic discussions. During the subsequent phase, a review that is comparatively composite in nature can be generated through the integration of structured and bibliometric review principles (Dabić et al., 2020). A few unconventional examples of grey literature reviews combined with scientific literature reviews are blockchain in the insurance sector (Kar & Navin, 2021), the sharing economy in sustainability (Rana et al., 2023), online shopping (Shroff et al., 2023) and the diffusion of blockchain technology (Grover et al., 2019). Similarly, when a phenomenon is yet to be published widely, hybrid reviews combining grey literature with academic articles, gives a better understanding of the discipline (e.g., Kar et al., 2023).

Furthermore, academics and scholars must understand the significance and distinction between SLR and the traditional review based on various parameters. The comparisons are summarized from the prominent papers of Briner and Denyer (2012), Kraus et al. (2020), Hodgkinson and Ford (2015), Lame (2019) and Pittaway et al. (2014), which are represented in Table 1.

Table 1.

Comparison of Different Review Types.

Parameter	SLR	Traditional Reviews
Recognize the importance of a review article	SLRs will only provide a reason if it is required in the articles	Traditional evaluations may include many marginally related studies
Formulation of a review procedure	The SLR method must have an objective and inclusion and exclusion protocol	An objective is not mandatory for this method
Research identification	Systematic, structured, adaptable and transparent in procedure	Has more of a subjective manner of exploration
Assessing and analysing studies	Transparency in the elimination process and objectivity in the decision-making while justifying propositions	The author uses relevant literature to validate their hypotheses
Carrying out retrieval of data	Managed by a standard protocol	The authors’ perceptions serve as the driving force behind this work
Performing data synthesis	The concept-driven core of an SLR looks at literature with a lens for analysis	A summary of present literature reviews
Motives for SLR	Independently generates evidence and addresses a research query	To establish a literature base for a forthcoming empirical undertaking as a separate paper
Time	More time consuming	Less time consuming

Source: Comparison is summarized from Briner and Denyer (2012), Kraus et al. (2020), Hodgkinson and Ford (2015), Lame (2019), and Pittaway et al. (2014).

Literature Review Process

A successful review has three major phases: planning, execution and reporting (Brereton et al., 2007; Kitchenham & Charters, 2007). In the planning phase, researchers determine the reason for a review, formulate the research queries and develop a procedure. The review is then carried out by locating and selecting primary research, as well as collecting, evaluating and synthesizing data to share the results from the literature (Xiao & Watson, 2019). Despite differences in procedures, all types of literature review follow the same eight steps, which are represented in Figure 1 (Xiao & Watson, 2019): (1) formulating the research issue, (2) devising and verifying the review protocol, (3) examining and searching the literature, (4) filtering for inclusion, (5) evaluating quality, (6) data extraction, (7) data analysis and synthesis and (8) summarizing the results in the form of reports. Subsequently, it should be kept in mind that the literature review procedure can be iterative, and unexpected problems may arise during the review process that require changes to the research query or review protocol (Xiao & Watson, 2019). A common problem is that the research questions are too broad; the researchers then narrow the topic and modify the inclusion criteria (Xiao & Watson, 2019). Thus, the review procedure of each stage will next be discussed in detail.

Step 1: Formulating the Research Issue

The objectives of the literature review should be the driving force behind all methodologies and reporting of findings (Xiao & Watson, 2019). Moreover, the selection of articles to be included in the review, the technique used for collecting data, and its analysis and synthesis to develop the reports should be aligned to address the research questions (Snyder, 2019). The process of determining the appropriate research query can be iterative (Xiao & Watson, 2019). Brereton et al. (2007) recommended that utilizing pre-review mapping assists with finding subtopics from the proposed research questions. Further, this pre-review mapping assists researchers in examining whether it is possible to explore a significant amount of material or whether they still need to narrow down the search for a specific research question (Gupta et al., 2018).

Figure 1.

Source: Xiao and Watson (2019), Okoli and Schabram (2010), Snyder (2019), Brereton et al. (2007), and Gupta et al. (2018).

Step 2: Developing and Verifying the Review Protocol

Like a social science research design, the review procedure is a predetermined plan that explains how the evaluation will be conducted (Xiao & Watson, 2019). The review protocol is mandatory for comprehensive systematic evaluations (Okoli & Schabram, 2010). This method is required to enhance the quality of a review as it reduces the possibility of researcher bias in selecting data for analysis and improves the review’s reliability (Kitchenham & Charters, 2007). Additionally, the review protocol must describe every component that involves its purpose, research questions, criteria for inclusion, search techniques, quality assessment standards and screening processes, data extraction, synthesis and reporting strategies (Gomersall et al., 2015). Finally, prior to implementation, it is critical to thoroughly validate the entire process of a review protocol, including its duration (Brereton et al., 2007). In addition, the researcher is capable to explain the method of validating reviews, which is reliant upon specific research objectives. During the review validation phase, two researchers conducted a pilot study by selecting a small number of papers and recording their observations independently. Subsequently, the observations facilitate the review process is authenticated by two reviewers to enhance the clarity and quality of the review procedure (Brereton et al., 2007).

Step 3: Searching the Literature

When a literature review is conducted, it is crucial to use a well-defined search protocol which helps in targeting the findings. This type of systematic review is based on a methodical examination of the existing literature (Hiebl, 2023; Xiao & Watson, 2019). The search protocol depends on several criteria to retrieve the data for assessment:

The identification of search channels for literature

There are three primary sources used to locate literature: electronic databases, the backward search and the forward search (Xiao & Watson, 2019). Electronic databases are now the most common starting point for a literature search, and they represent the core content of publishing literature resources (Petticrew & Roberts, 2006). Examples of electronic databases are WoS, EBSCO, ProQuest, Google Scholar and IEEE Xplore. Examples of grey literature include conference proceedings, theses, reports and white papers (Xiao & Watson, 2019). As a result, to compile a comprehensive list of literature, scholars have to perform a backward search to find important works cited by articles (Webster & Watson, 2002). Using the list of references at the end of the article is an effective method for locating these publications (Webster & Watson, 2002). In addition, a forward search should be conducted to distinguish all articles that have cited the review and thus considered the publication (Webster & Watson, 2002). Search engines such as Google Scholar and ISI Citation Index are forward-article searches (Levy & Ellis, 2006). Finally, employing experts in the field has been suggested as a method to assess and double-check the depth of a search (Petticrew & Roberts, 2006). It is feasible to identify researchers who have made considerable progress in the study by reviewing the list generated by the queries (Okoli & Schabram, 2010). It is preferred to undertake searches in Scopus or WoS which has more formal indexing criteria than using Google Scholar.

The relevant keywords used in the search

The keyword search needs to be drawn from the research questions. The research query can be divided into conceptual domains (Kitchenham & Charters, 2007). For instance, ‘What factors impact company performance using generative artificial intelligence (GAI)?’ is a research question. There are three domains in this example: ‘business’, ‘performance’ and ‘GAI’. For example, a preliminary search utilizing the specified keywords has the potential to retrieve a limited number of documents very quickly. Indexing services such as Scopus or WoS permit the use of Boolean operators, and it is essential to understand how to develop search sequences using ‘AND’ and ‘OR’ (Fink, 2005). ‘AND’ is frequently used to join the primary terms, while ‘OR’ is used to add synonyms (Brereton et al., 2007). Therefore, the possible search strings are (‘business’, ‘organisation’, ‘firm’, ‘enterprise’) AND (‘performance’ OR ‘affect’ OR ‘impact’ OR ‘improve’) AND (‘GAI’ OR ‘large language models’ OR ‘ChatGPT’) if one is exploring the impacts of GAI on firms.

Framing sample strategies for review

All literature enquiries are organized by reviewers using a sample technique and search strategy (Suri & Clarke, 2009). Diverse types of literature reviews employ different approaches for the sample and search procedures (Xiao & Watson, 2019). Depending on the review’s purpose, the inquiry may be exhaustive and detailed or only limited and illustrative (Paré et al., 2015). Also, grey literature such as reports, theses and conference proceedings should be included in the search investigation (Kitchenham & Charters, 2007).

Refining findings with new limitations

Other practical criteria may include publication language, date and founding source (Okoli & Schabram, 2010).

Stopping rules and regulations

When repeated searches find similar references with no additional results, the search can be terminated at any time (Levy & Ellis, 2006).

Step 4: Filtering for Inclusion

This is a two-step process that serves as an effective methodology. First, sort the articles for inclusion according to the abstract reviews, then perform a revised quality evaluation based on a full-text review (Xiao & Watson, 2019). Then, the overall screening procedure is the same for all kinds of literature review. Researchers should construct inclusion and exclusion criteria as per the research topic, and any papers irrelevant to the research queries should be excluded (Kitchenham & Charters, 2007). Inclusion and exclusion criteria must be acceptable and logical in nature (Kitchenham & Charters, 2007; Okoli & Schabram, 2010). Furthermore, filtering criteria should be capable of classifying research that is consistent, interpretable and that offers an acceptable volume of literature for review (Kitchenham & Charters, 2007). Before adopting these criteria, researchers should conduct a pilot study to check reliability (Kitchenham & Charters, 2007). The research design and methodology may serve as inclusion and exclusion criteria, and studies may be limited to specific geographic regions (e.g., developed versus developing countries) (Okoli & Schabram, 2010). Lastly, with regards to the overall screening process, it is recommended that at least two reviewers independently evaluate the studies that meet the review standardization for the inclusion and exclusion process (Gomersall et al., 2015).

Step 5: Quality Evaluation

A quality evaluation which strengthens the full-text articles serves as the final stage of preparing for data extraction and synthesis (Ludvigsen et al., 2016). Before discussing the procedures of comparing and integrating findings, the quality of the studies needs to be ascertained (Ludvigsen et al., 2016). This is often undertaken using the journal’s reputation as a proxy for ascertaining quality. Furthermore, the phrase ‘quality assessment’ refers to the examination of internal validity for systematic reviews and checking it for methodological errors (Petticrew & Roberts, 2006). Some researchers also incorporate studies for ‘external validity’ or ‘generalization in the quality assessment phase’ (Rousseau et al., 2008). Rating studies based on a set of standards appears to be a frequent way to assess quality, and in some cases ranking the research is suggested based on the same methodological criteria used for inclusion/exclusion (Okoli & Schabram, 2010). Like the inclusion screening practice, it is suggested that two or more researchers independently conduct a simultaneous quality assessment (Brereton et al., 2007). The outcome is that reviewers will go through and examine each study against the quality criteria by reading the full text (Noordzij et al., 2009). In the case of a review of academic publications, journal quality must include the inclusion of articles in WoS for indexing and ranks in journal quality from the Australian Business Dean’s Council or the Chartered Association of Business Schools index.

Step 6: Data Extraction

Data extraction often requires coding, particularly for extensive evaluations, and it is critical to identify whether the coding will be inductive or deductive (Suri & Clarke, 2009). The conclusions obtained will directly affect the type of coding used by researchers to optimize the errors (Suri & Clarke, 2009). However, it is advised that at least two researchers separately code the studies. It is mandatory for scholars to evaluate the full publication rather than relying on only the findings of the major interpretation (Gomersall et al., 2015).

Step 7: Data Analysis and Synthesis

The study may organize the data in accordance with the review criteria (Xiao & Watson, 2019) and present it through infographics so that it becomes easy for readers to follow. Typically, this will be a blend of infographics, tables and textual descriptions. However, each review category will have slightly distinct reporting requirements with novel conceptual approaches emerging based on the scope (Dixon-Woods, 2011). The data synthesis from this mixed approach is then utilized to draw conclusions regarding the performance of the research strategy and identify research gaps from the review for future research agendas (Sandelowski et al., 2007).

Step 8: Summarizing the Results in the Form of Reports

Researchers should describe their findings from literature searches, screenings and quality assessments (Noordzij et al., 2009), which is represented in Figure 2. We can also develop the reports based on theory, contexts, characteristics and methodology, which is shown in Figure 3. The literature review should adhere to a concise framework which jointly ties studies with important themes, characteristics and subgroups (Rowley & Slack, 2004). In general, the rigour or adaptability of review methods must ensure that the process is transparent and that conclusions are supported by information to guide future research directions (Okoli & Schabram, 2010). Whittemore and Knafl (2005) further recommended that the findings of an integrated review should be presented graphically (as a table or diagram) to assure readers that their opinions are justified. One way to report this thematically could be using a framework like the TCCM used in Grover et al. (2022) for studying social media influence by reporting the literature based on theory, context, characteristics and methodology. This brings in a better structure and objectivity to the way the findings are presented for the editorial and review process to examine and also for future readers to follow.

Figure 2.

Source: Refer the flowchart from Noordzij et al. (2009) article.

Figure 3.

Source: Grover et al. (2022).

Discussion

The SLR has gained popularity in the management domain in recent years, and an increasing number of publications are publishing review articles as independent research. There is also a rising proportion of published SLRs, which demonstrates a structured literature review conducted before delving deep into the research analysis. This criterion is largely being adapted now by most management journals (Templier & Paré, 2015). There is no clear technique for selecting relevant research topics in SLRs because they are defined by unique research questions and inclusion/exclusion criteria. Hence one of the most prevalent reasons for rejecting a review article for publication is the poor quality of its synthesis. This article illuminates the evolution of SLR and the primary distinctions between a traditional literature review and an SLR process. Further, SLRs reveal that this methodology has evolved significantly in recent years across domains due to technological advances and software support. A variety of research domains such as medical sciences, information systems and computer science discuss the importance of the SLR and are developing protocols for doing high-quality research study. This article emphasizes the research limitations by systematically investigating the methodology of literature analysis. It examines the appropriate stages of carrying out an SLR and explains the various types of literature review.

Conducting literature reviews systematically will strengthen their quality, reproducibility, reliability and validity. This study draws attention to multiple takeaways: First, the author must develop a research query and when conducting the entire literature review process (comprising the literature search, data extraction and analysis and reporting) should modify it to address the research question (Kitchenham & Charters, 2007). The second step is for researchers to uncover and outline the purpose of the review to test a particular hypothesis and extend it from current research to theory development. Third, the formulation of a review protocol is an essential stage for robust SLRs (Brereton et al., 2007; Okoli & Schabram, 2010). The review protocol lowers the risk of researcher bias in choosing data and analysis (Kitchenham & Charters, 2007). Fourth, the researcher must be meticulous while conducting the literature review and be quality conscious. For most reviews, the search for literature should be exhaustive and requires the researcher to investigate multiple databases, perform backward and forward searches, and if necessary, consult with domain experts to understand each study and interpret its results (Ludvigsen et al., 2016). Fifth, research evaluation can be iterative. Researchers must conduct a pilot assessment and determine what is feasible. To some extent, the research query must be refined. During the review process, greater comprehension can be obtained, which requires changes to keywords or analytical approaches. Sixth, the researcher needs to document the review process’s outcomes for the review to be consistent and repeatable by creating documents to maintain the transparency of the procedure. Seventh, the review procedure encourages teamwork and recommends that at least two examiners independently screen literature for inclusion, assess study quality and code studies for analysis (Templier & Paré, 2015). Lastly, we discuss report writing in various forms and suggest suitable opportunities for future research.

In addition to this, we propose to the academic community several guidelines for conducting the SLR procedure:

Identify a Research Topic

Find a research topic of academic interest that contains at least 40 empirical publications after applying selection and elimination criteria, as this suggests that there is a need for literature synthesis. Debates in an area motivate the need for undertaking an SLR.

Database to Conduct Literature Search

Conduct a brief literature search to gain insights into the topic. For this, EBSCO, SCOPUS, WoS, ProQuest, Google Scholar and grey literature can be considered. Find keywords and related terms relevant to the field of study and examine previously published literature reviews. Also note down the research gap in the studies, which helps to frame the research objectives.

Frame the Research Objectives

Determine the purpose of your review article by identifying research questions that can be answered with an SLR and create a literature map. This literature review leads to the development of the conceptual and theoretical model based on research objectives.

Develop Detailed Procedure for Data Evaluation and Extraction

Create a protocol for data evaluation and extraction based on the objective of your SLR. Identify the keyword searches properly, and only shortlist articles from Scopus or WoS. Then preferably categorize the article based on ABDC (A & A*) and ABS (ABS 3 & 4) rankings in Scopus and WoS databases.

Using Keyword Searches in Grey Literature Sources

You can also perform keyword searches in grey literature indexed in ACM library, PROQUEST, EMERALD INSIGHTS, the British Library, TROVE, grey literature reports, etc.

Note Down the Flowchart of Articles Number Inclusion and Exclusion

Maintain transparency throughout the entire SLR process and write down the number of article iterations.

Downloading the Data Files Format

Always extract the data in .csv and Excel formats that contain the title, author, year, abstract, journal and references. Add columns to these which include factors based on your research questions.

Data Filtration Parameters

Synthesize the data in a systematic way using various parameters to develop the tables such as methodology, geography, theory and findings.

Recognize the Present Research Gaps During the Literature Review

Always have a strong direction for future research based on the gaps identified through the review. This could be based on existing research gaps.

Choosing the Right Journals

Target the right journal where SLR articles are published. For example, IJIM Data Insights welcome SLRs which deal with the adoption, usage and impacts of emerging information technologies. Similarly, other journals prefer SLRs as compared to other forms of reviews.

Hence, the effort involved in conducting the SLR process is considerable and a critical part of the research process. SLRs can serve as a base for knowledge creation, set standards for policy and practice, give proof of a particular outcome and can generate cutting-edge concepts and approaches for a specific sector. Therefore, this article serves as an outline for future studies and theories which enhances the quality of study in the global research community.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

References

Adams

R. J.

, Smart

, & Huff

A. S.

(2017). Shades of grey: Guidelines for working with the grey literature in systematic reviews for management and organizational studies. International Journal of Management Reviews, 19(4), 432–454. 10.1111/ijmr.12102

Brereton

, Kitchenham

B. A.

, Budgen

, Turner

, & Khalil

(2007). Lessons from applying the systematic literature review process within the software engineering domain. Journal of Systems and Software, 80(4), 571–583. 10.1016/j.jss.2006.07.009

Briner

R. B.

, & Denyer

(2012). Systematic review and evidence synthesis as a practice and scholarship tool. In Rousseau

Denise M.

(ed.), The Oxford handbook of evidence-based management: Companies, classrooms, and research (pp. 112–129). Oxford Library of Psychology. 10.1093/oxfordhb/9780199763986.013.0007

Canabal

, & White III

G. O.

(2008). Entry mode research: Past and future. International Business Review, 17(3), 267–284. 10.1016/j.ibusrev.2008.01.003

Chakraborty

(2023a). Quantum computing: Are the nations gearing up for it? https://business-frontiers.org/2023/07/06/quantum-computing-are-the-nations-gearing-up-for-it/

Chakraborty

(2023b). How artificial intelligence is transforming business’ supply chains. https://www.business-fundas.com/2021/how-artificial-intelligence-is-transforming-business-supply-chains/

Chakraborty

, & Kar

A. K.

(2017). Swarm intelligence: A review of algorithms. Nature-inspired Computing and Optimization: Theory and Applications, 10, 475–494. 10.1007/978-3-319-50920-4_19

Chalmers

, Hedges

L. V.

, & Cooper

(2002). A brief history of research synthesis. Evaluation and the Health Professions, 25(1), 12–37. 10.1177/0163278702025001003

Chassin

M. R.

, Kosecoff

, Park

R. E.

, Winslow

C. M.

, Kahn

K. L.

, Merrick

N. J.

, Keesey

, Fink

, Solomon

D. H.

, & Brook

R. H.

(1987). Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures. JAMA, 258(18), 2533–2537. https://doi:10.1001/jama.1987.03400180067028

10.

Çizel

, Selçuk

, & Atabay

(2020). A systematic review of common method bias. Anatolia: Turizm Arastirmalari Dergisi, 31(1), 7–18.

11.

Dabić

, Vlačić

, Paul

, Dana

L. P.

, Sahasranamam

, & Glinka

(2020). Immigrant entrepreneurship: A review and research agenda. Journal of Business Research, 113, 25–38. 10.1016/j.jbusres.2020.03.013

12.

Dixon-Woods

(2011). Using framework-based synthesis for conducting reviews of qualitative studies. BMC Medicine, 9(1), 1–2. 10.1186/1741-7015-9-39

13.

Frank

, & Hatak

(2014). Doing a research literature review. In Fayolle

, & Wright

(Eds.), How to get published in the best entrepreneurship journals (pp. 94–117). Edward Elgar Publishing.

14.

Gilal

F. G.

, Zhang

, Paul

, & Gilal

N. G.

(2019). The role of self-determination theory in marketing science: An integrative review and agenda for research. European Management Journal, 37(1), 29–44. https://dois.org/10.1016/j.emj.2018.10.004

15.

Gomersall

J. S.

, Jadotte

Y. T.

, Xue

, Lockwood

, Riddle

, & Preda

(2015). Conducting systematic reviews of economic evaluations. International Journal of Evidence-based Healthcare, 13(3), 170–178. 10.1097/XEB.0000000000000063

16.

Goodman

J. K.

, & Paolacci

(2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196–210. 10.1093/jcr/ucx047

17.

Grover

, Kar

A. K.

, & Dwivedi

(2022). The evolution of social media influence: A literature review and research agenda. International Journal of Information Management Data Insights, 2(2), 100116.

18.

Grover

, Kar

A. K.

, & Janssen

(2019). Diffusion of blockchain technology: Insights from academic literature and social media analytics. Journal of Enterprise Information Management, 32(5), 735–757. 10.1108/JEIM-06-2018-0132

19.

Gupta

, Rajiah

, Middlebrooks

E. H.

, Baruah

, Carter

B. W.

, Burton

K. R.

, Chatterjee

A. R.

, & Miller

M. M.

(2018). Systematic review of the literature: Best practices. Academic Radiology, 25(11), 1481–1490. 10.1016/j.acra.2018.04.025

20.

Hair

J. F.

, Hult

G. T. M.

, Ringle

C. M.

, Sarstedt

, & Thiele

K. O.

(2017). Mirror, mirror on the wall: A comparative evaluation of composite-based structural equation modeling methods. Journal of the Academy of Marketing Science, 45, 616–632. 10.1007/s11747-017-0517-x

21.

Hart

(2018). Doing a literature review: Releasing the research imagination. (2nd ed.). Sage Publications.

22.

Hiebl

M. R.

(2023). Sample selection in systematic literature reviews of management research. Organizational Research Methods, 26(2), 229–261. 10.1177/1094428120986851

23.

Higgins

J. P.

, Thomas

, Chandler

, Cumpston

, Li

, Page

M. J.

, & Welch

V. A.

(Eds.). (2019). Cochrane handbook for systematic reviews of interventions. John Wiley & Sons. 10.1002/9781119536604

24.

Hodgkinson

G. P.

, & Ford

J. K.

(2015). What makes excellent literature reviews excellent? A clarification of some common mistakes and misconceptions. Journal of Organizational Behavior, 36(S1), S1–S5. 10.1002/job.1983

25.

Ilavarasan

P. V.

, Kar

A. K.

, & Aswani

(2017). Employment outcomes of skills Training in South Asian countries: An evidence summary. EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London.

26.

Jeffery

K. G.

(2000). An architecture for grey literature in a R&D context. International Journal on Grey Literature, 1(2), 64–72. 10.1108/14666180010327429

27.

Jeyaraj

, & Dwivedi

Y. K.

(2020). Meta-analysis in information systems research: Review and recommendations. International Journal of Information Management, 55, 102226.

28.

Jones

, & Gatrell

(2014). The future of writing and reviewing for IJMR. International Journal of Management Reviews, 16(3), 249–264. 10.1111/ijmr.12038

29.

Kahiya

E. T.

(2018). Five decades of research on export barriers: Review and future directions. International Business Review, 27(6), 1172–1188. 10.1016/j.ibusrev.2018.04.008

30.

Kar

A. K.

, Choudhary

S. K.

, & Singh

V. K.

(2022). How can artificial intelligence impact sustainability: A systematic literature review. Journal of Cleaner Production, 134120. 10.1016/j.jclepro.2022.134120

31.

Kar

A. K.

, & Navin

(2021). Diffusion of blockchain in insurance industry: An analysis through the review of academic and trade literature. Telematics and Informatics, 58, 101532. 10.1016/j.tele.2020.101532

32.

Kar

A. K.

, Varsha

P. S.

, & Rajan

(2023). Unravelling the impact of generative artificial intelligence (GAI) in industrial applications: A review of scientific and grey literature. Global Journal of Flexible Systems Management, 1–31. 10.1007/s40171-023-00356-x

33.

Kepes

, Banks

G. C.

, McDaniel

, & Whetzel

D. L.

(2012). Publication bias in the organizational sciences. Organizational Research Methods, 15(4), 624–662. 10.1177/1094428112452760

34.

Kitchenham

, & Charters

(2007). Guidelines for performing systematic literature reviews in software engineering version 2.3. Engineering, 45(4ve), 1051.

35.

Knopf

J. W.

(2006). Doing a literature review. PS: Political Science & Politics, 39(1), 127–132. 10.1017/S1049096506060264

36.

Kraus

, Breier

, & Dasí-Rodríguez

(2020). The art of crafting a systematic literature review in entrepreneurship research. International Entrepreneurship and Management Journal, 16, 1023–1042. 10.1007/s11365-020-00635-4

37.

Kumar

, Hollebeek

L. D.

, Kar

A. K.

, & Kukk

(2023). Charting the intellectual structure of customer experience research. Marketing Intelligence & Planning, 41(1), 31–47. 10.1108/MIP-05-2022-0185

38.

Kumar

, Kar

A. K.

, & Ilavarasan

P. V.

(2021). Applications of text mining in services management: A systematic literature review. International Journal of Information Management Data Insights, 1(1), 100008. 10.1016/j.jjimei.2021.100008

39.

Lame

(2019). Revisiones sistemáticas de literatura: una introducción. In Actas de la Conferencia Internacional de Design Society sobre Diseño de Ingeniería (Vol. 1, No. 1, pp. 1633–1642.

40.

Lawrence

, Houghton

, Thomas

, & Weldon

(2014). Where is the evidence? Realising the value of grey literature for public policy and practice: A discussion paper.

41.

Levy

, & Ellis

T. J.

(2006). A systems approach to conduct an effective literature review in support of information systems research. Informing Science, 9. 10.28945/479

42.

Lim

W. M.

(2020). Challenger marketing. Industrial Marketing Management, 84, 342–345. 10.1016/j.indmarman.2019.08.009

43.

Lim

W. M.

, Yap

S. F.

, & Makkar

(2021). Home sharing in marketing and tourism at a tipping point: What do we know, how do we know, and where should we be heading? Journal of Business Research, 122, 534–566. 10.1016/j.jbusres.2020.08.051

44.

Ludvigsen

M. S.

, Hall

E. O.

, Meyer

, Fegran

, Aagaard

, & Uhrenfeldt

(2016). Using Sandelowski and Barroso’s meta-synthesis method in advancing qualitative evidence. Qualitative Health Research, 26(3), 320–329. 10.1177/1049732315576493

45.

Noordzij

, Hooft

, Dekker

F. W.

, Zoccali

, & Jager

K. J.

(2009). Systematic reviews and meta-analyses: When they are useful and when to be careful. Kidney International, 76(11), 1130–1136. 10.1038/ki.2009.339

46.

Okoli

(2015). A guide to conducting a standalone systematic literature review. Communications of the Association for Information Systems, 37. 10.17705/1CAIS.03743

47.

Okoli

, & Schabram

(2010). A guide to conducting a systematic literature review of information systems research. Sprouts: Working Papers on Information Systems, 10(26). http://sprouts.aisnet.org/10-26.

48.

Palmatier

R. W.

, Houston

M. B.

, & Hulland

(2018). Review articles: Purpose, process, and structure. Journal of the Academy of Marketing Science, 46, 1–5. 10.1007/s11747-017-0563-4

49.

Paré

, Trudel

M. C.

, Jaana

, & Kitsiou

(2015). Synthesizing information systems knowledge: A typology of literature reviews. Information & Management, 52(2), 183–199. 10.1016/j.im.2014.08.008

50.

Paul

, & Barari

(2022). Meta-analysis and traditional systematic literature reviews—What, why, when, where, and how? Psychology & Marketing, 39(6), 1099–1115. 10.1002/mar.21657

51.

Paul

, & Benito

G. R.

(2018). A review of research on outward foreign direct investment from emerging countries, including China: What do we know, how do we know and where should we be heading? Asia Pacific Business Review, 24(1), 90–115. 10.1080/13602381.2017.1357316

52.

Paul

, & Criado

A. R.

(2020). The art of writing literature review: What do we know and what do we need to know? International Business Review, 29(4), 101717. 10.1016/j.ibusrev.2020.101717 10.1111/twec.12502

53.

Paul

, Lim

W. M.

, O’Cass

, Hao

A. W.

, & Bresciani

(2021). Scientific procedures and rationales for systematic literature reviews (SPAR-4-SLR). International Journal of Consumer Studies, 45(4), O1–O16. 10.1111/ijcs.12695

54.

Petticrew

, & Roberts

(2006). Why do we need systematic reviews? In Systematic reviews in the social sciences: A practical guide (pp. 1–26). John Wiley & Sons.

55.

Petticrew

, & Roberts

(2008). Systematic reviews in the social sciences: A practical guide. John Wiley & Sons.

56.

Pittaway

, Holt

, & Broad

(2014). Synthesising knowledge in entrepreneurship research: The role of systematic literature. In Handbook of research on small business and entrepreneurship, 6, 83–105. 10.4337/9781849809245.00014

57.

Rana

N. P.

, Kar

A. K.

, Gupta

, Pappas

I. O.

, & Papadopoulos

(2023). Unravelling the dark side of sharing economy: Managing and sustaining B2B relationships on digital platforms. Industrial Marketing Management. 10.1016/j.indmarman.2023.06.002

58.

Rousseau

D. M.

, Manning

, & Denyer

(2008). 11 evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals, 2(1), 475–515. 10.1080/19416520802211651

59.

Rowley

, & Slack

(2004). Conducting a literature review. Management Research News, 27(6), 31–39. 10.1108/01409170410784185

60.

Sackett

D. L.

, Rosenberg

W. M.

, Gray

J. M.

, Haynes

R. B.

, & Richardson

W. S.

(1996). Evidence-based medicine: What it is and what it isn’t. BMJ, 312(7023), 71–72. 10.1136/bmj.312.7023.71

61.

Sandelowski

, Barroso

, & Voils

C. I.

(2007). Using qualitative metasummary to synthesize qualitative and quantitative descriptive findings. Research in Nursing & Health, 30(1), 99–111. 10.1002/nur.20176

62.

Schöpfel

(2010, December). Towards a Prague definition of grey literature. In Twelfth international conference on grey literature: Transparency in grey literature. Grey tech approaches to high tech issues. Prague, 6–7 December 2010 (pp. 11–26).

63.

Shroff

, Kumar

, Martinez

L. M.

, & Pandey

(2023). From clicks to consequences: A multi-method review of online grocery shopping. Electronic Commerce Research, 1–40. 10.1007/s10660-023-09761-x

64.

Snyder

(2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339. 10.1016/j.jbusres.2019.07.039

65.

Suri

, & Clarke

(2009). Advancements in research synthesis methods: From a methodologically inclusive perspective. Review of Educational Research, 79(1), 395–430. 10.3102/0034654308326349

66.

Tavakoli

, & Wijesinghe

S. N.

(2019). The evolution of the web and netnography in tourism: A systematic review. Tourism Management Perspectives, 29, 48–55. 10.1016/j.tmp.2018.10.008

67.

Templier

, & Paré

(2015). A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems, 37(1), 6. 10.17705/1CAIS.03706

68.

Tranfield

, Denyer

, & Smart

(2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. 10.1111/1467-8551.00375

69.

van Dinter

, Tekinerdogan

, & Catal

(2021). Automation of systematic literature reviews: A systematic literature review. Information and Software Technology, 136, 106589. 10.1016/j.infsof.2021.106589

70.

Varsha

P. S.

, Akter

, Kumar

, Gochhait

, & Patagundi

(2021). The impact of artificial intelligence on branding: a bibliometric analysis (1982–2019). Journal of Global Information Management (JGIM), 29(4), 221–246. 10.4018/JGIM.20210701.oa10

71.

Webster

, & Watson

R. T.

(2002). Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26(2), 13–23.

72.

Whittemore

, & Knafl

(2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52(5), 546–553. 10.1111/j.1365-2648.2005.03621.x

73.

Williams

M. D.

, Rana

N. P.

, & Dwivedi

Y. K.

(2015). The unified theory of acceptance and use of technology (UTAUT): A literature review. Journal of Enterprise Information Management, 28(3), 443–488. 10.1108/JEIM-09-2014-0088

74.

Xiao

, & Watson

(2019). Guidance on conducting a systematic literature review. Journal of Planning Education and Research, 39(1), 93–112. 10.1177/0739456X17723971