Searching and reporting in Campbell Collaboration systematic reviews: A systematic assessment of current methods

Types of studies

Systematic reviews of all types and topics were included. Updates to previous systematic reviews were also included when changes to search methods were made. Protocols, methods papers, commentaries, editorials and other types of evidence synthesis (e.g., evidence and gap maps, mega maps) were excluded.

We chose January 2017 as a publication date limit to reflect our interest in examining current search practices. As search methods have evolved with rapid changes in the information landscape, reviews published since 2017 should provide an accurate picture of current practices. That being said, the date of publication of a review is not an accurate representation of when the search for studies was conducted. Reviews published in 2017 or later represent searches conducted in the years before and after 2017. These reviews provide a broader timeframe to look at the change in practices over time.

Types of participants

Not applicable.

Types of interventions

Not applicable.

Types of outcome measures

Included reviews must have reported on an original search strategy developed for that review. For example, in the course of data extraction, we encountered a number of systematic reviews that relied on searches of the Global Policing Database (GPD) (https://gpd.uq.edu.au/s/gpd/page/about), as opposed to designing original search strategies for a range of sources. Since these reviews did not develop or report their own search strategies and instead pointed to the methods used to develop the GPD, we chose to exclude these reviews from this assessment. This was also true of reviews that relied on a search strategy originally developed for a separately published evidence and gap map, or update reviews that did not amend a search strategy for the review published in 2017 or later.

For included reviews, we recorded detailed information about the search-related aspects of the review described in the sections below. For items related to adherence to standards and guidelines, we tried to capture levels of adherence (i.e., complete, partial and no adherence) to gain a better understanding of practices. Depending on the variable, we recorded either Yes/No or one of a set of answers reflecting a range of reporting and conduct. For example, whether authors searched at least two relevant databases was answered with a simple Yes/No response. On the other hand, whether authors reported the websites searched could be a range of answers including: all websites reported (i.e., full adherence to this reporting standard), some websites reported (i.e., partial adherence), none reported (no adherence), or not applicable (did not search websites). More details about how each conduct and reporting item was handled can be found in the data set in File S2.

Electronic searches

Not applicable.

Searching other resources

Not applicable.

Selection of studies

Given the straightforward nature of the selection criteria (i.e., systematic reviews or updates to systematic reviews published in January 2017 or later), a single reviewer conducted the search and selection of studies (S. Y.).

Data extraction and management

A data extraction form was developed based in part on MECCIR reporting standards (Methods Group of the Campbell Collaboration, 2019b), PRISMA 2020 (Page et al., 2021) and the PRISMA-S Extension for reporting literature searches in systematic reviews (Rethlefsen et al., 2021). An initial draft template was piloted by six authors on nine different systematic reviews. After this initial test, the form was modified to include a total of 79 items. The form was again piloted by two authors independently on five systematic reviews, and based on this and discussions with the author team, additional modifications and clarifications were made. Thus, the final data extraction form differs slightly from the form published with the protocol. The final version includes 90 extraction items and can be found in File S1 along with the associated variables names used in the data set.

Data about the following were extracted from each review: 1.

Bibliographic characteristics of the reviews including: title, authors, year of publication, Campbell coordinating group associated with the review, whether the review was an update, and the most recent search date reported in the review.

Data was extracted from all included Campbell systematic reviews using a form developed in Google Sheets. Data extraction was carried out independently by two authors and discrepancies were resolved by a third author. For updated systematic reviews, if the update indicated changes to the search methods, then data was extracted from the updated systematic review for inclusion in subsequent analyses.

Assessment of risk of bias in included studies

We used an adapted version of AMSTAR 2 (Shea et al., 2017), specifically item 4 related to the search, to assess risk of bias in Campbell Collaboration systematic reviews using this widely used critical appraisal tool. Item 4 in the AMSTAR 2 checklist asks ‘Did the review authors use a comprehensive literature search strategy?’. There are eight sub-items required to achieve a Yes or Partial Yes rating for Item 4. Because AMSTAR 2 was developed for medical and health-related systematic reviews, this includes the requirement to search trial/study registries. However, for reviews in the social sciences, such trial or study registries may not exist or be appropriate for searching. Thus, we adapted the AMSTAR 2 checklist by removing the ‘searching trial/study registries’ requirement for a Yes rating.

Measures of treatment effect

Not applicable.

Unit of analysis issues

Not applicable.

Dealing with missing data

Not applicable.

Assessment of heterogeneity

Not applicable.

Assessment of reporting biases

Not applicable.

Data synthesis

The extracted data was pre-processed using a combination of OpenRefine and R/RStudio. Most of the analyses were carried out in R (version 4.2.1) and RStudio (version 2023.12.0 + 369). Descriptive statistics and summaries of the data were calculated for the variables related to conduct and reporting. This includes frequencies and percentages for categorical variables, and mean, median and range for numerical variables. For numerical variables (e.g., time lag in months between search date and publication date), we compared groups using one-way Analysis of Variance (ANOVA).

Some categorical variables were extracted as binary Yes/No answers as to whether an item was conducted or reported. For other variables with a range of answers, answers were grouped into full, partial, or not conducted/reported (or not applicable) and these nuances were taken into consideration in the descriptive analysis and interpretation. For example, for whether authors reported dates of searches, they may have reported a general date range (e.g., ‘searches were run in May 2020’), exact search dates for each search, or no dates at all. In this case, general date range would be considered partial adherence to the standard to report the search dates for each source. To assess adherence to standards, study variables were aligned with the 2019 MECCIR standards and PRISMA 2020 standards. Depending on the variable, degree of compliance (full/partial/none) for each study variable was converted to a number (2/1/0) then assigned a colour (blue/yellow/red) to create a heatmap visualizing study variable adherence to the standards. This was carried out in Excel.

Subgroup analysis and investigation of heterogeneity

We were interested in understanding differences across Campbell coordinating groups as these groups may have variable editorial and discipline-specific processes and practices that could impact search reporting and conduct. We conducted sub-group analyses by coordinating group for a number of variables, including types of grey literature searched, involvement of an information specialist, and time lag between most recent search date and publication date. For the time lag variable, we used one-way Analysis of Variance (ANOVA) to determine whether there were statistical differences between coordinating groups.

We were also interested in whether the involvement of an information specialist as a co-author versus through consulting or not at all was related to differences in search conduct and reporting. Thus, for reviews with an information specialist co-author, that consulted an information specialist but without co-authorship and those not mentioning information specialist involvement, we calculated the percentage adhering to different conduct and reporting best practices and standards.

Sensitivity analysis

Not applicable.

Summary of findings and assessment of the certainty of the evidence

Not applicable.

Results of the search

One-hundred and eleven systematic reviews or systematic review updates were published from January 2017 until the time of the most recent search (March 2024). Of these, 11 were updates to previous systematic reviews. A complete list of included studies, along with the extracted data and Study IDs, which are used for reference throughout the manuscript, can be found in File S2. All of the codes associated with the analyses can be found in Young, 2024.

Included studies

The coordinating groups that published the highest number of systematic reviews were International Development, Education and Social Welfare, each with 29 reviews. The majority of reviews (83) did not relate to any specific geographic region. Twenty-three were related to Low- and Middle-Income Countries only. Three were concerned with the United States only.

Table 1 shows the characteristics of the included reviews.

Excluded studies

Any papers published in Campbell Systematic Reviews since 2017 that were not systematic reviews were excluded upon initial screening of tables of contents. In addition, upon data extraction we identified ten other papers that were ultimately excluded. Two studies were updates that did not change their search methods from the original reviews, which were published prior to 2017. Four studies referred to the search strategies used to assemble the Global Policing Database and did not conduct original searches themselves. Four studies used searches that were previously conducted as part of an evidence and gap map. For a list of excluded studies and their reasons for exclusion, see Table S1 in File S3.

Allocation

Not applicable.

Blinding

Not applicable.

Incomplete outcome data

Not applicable.

Selective reporting

Not applicable.

Other potential sources of bias

Not applicable.

Source selection and reporting

Figure 1 summarizes reporting on sources searched. These reporting items align with mandatory reporting standards in MECCIR (standard R34) and PRISMA (Item #6). Almost all reviews searched at least two relevant databases (98.2%, n = 109). Of the two that did not search at least two relevant databases, one relied heavily on citation searching techniques and an existing repository of relevant literature that uses comprehensive methods to identify studies (3ie Development Evidence Portal, https://developmentevidence.3ieimpact.org/, Study ID 35). The other searched only one multidisciplinary database, and relied more on working paper repositories (Study ID 5). All reviews reported the databases searched, although 37 (33.3%) listed either an umbrella database or a platform name rather than the individual database or sub-database name for at least one source. Platforms can host multiple databases, so listing a platform in lieu of a database makes it impossible to determine which database(s) on that platform was searched. On the other hand, listing both platform and the database name provides increased transparency and reproducibility as databases can often be accessed through multiple platforms impacting how searches are constructed (Rethlefsen et al., 2021). Eighteen reviews (16.2%) did not report any platforms along with the names of the databases. Nineteen (17.1%) reported date ranges of the contents in all of the databases searched, with 15 reporting date ranges for at least some of the databases. A heatmap of all included studies (by ID) and their degree of compliance (full, partial, none) with each data extraction item by conduct and reporting standard is shown in Figure S1 (conduct) and Figure S2 (reporting) in File S3.

OPEN IN VIEWER

Figure 1

Number of reviews fully, partially or not adhering to standards related to reporting databases, grey literature and websites searched (MECCIR R34 and PRISMA Item #6).

With regard to web searching, a little more than half of the reviews (n = 60) searched Google Scholar but none of the other large free, multidisciplinary academic search engines were mentioned in any review (e.g., Dimensions, Microsoft Academic, Lens.org, etc.). A little more than one-third of reviews reported searching Google (n = 42). Bing was searched in three reviews. Six reviews did not search Google, Google Scholar or any government or organizational websites. About 37.1% (n = 39) of reviews that reported searching websites for additional studies did not provide website URLs in either the text or supplementary material.

All reviews searched at least some grey literature (MECCIR C28) and most reviews (n = 91) provided a list of all sources of grey literature searched (MECCIR R34). Figure 2 shows the types of grey literature searched by different Campbell coordinating groups. About half of the reviews searched scholarly databases specializing in regionally specific scholarship (e.g., LILACs, Africa Journals Online, etc.).

OPEN IN VIEWER

Figure 2

The proportion of reviews by coordinating group that intentionally searched sources of different types of grey literature (e.g., searched a dissertations and theses database as opposed to relying on coverage of dissertations and theses in the other bibliographic databases searched).

Search strategy conduct and reporting

Figure 3 summarizes the reporting of search strategies in Campbell systematic reviews. Most reviews reported at least one full database search strategy (90.1%, n = 100), with 70 (63.1%) providing full search strategies for all databases (MECCIR R38, PRISMA Item #7). Exact search dates (MECCIR R35, PRISMA Item #6) for all searches were given in 60 (54.1%) reviews, with 35 (31.5%) reporting general date ranges for search and 16 (14.4%) reporting no dates. Twenty-five (22.5%) reviews reported all exact search strategies for grey literature sources, with 40 (36.0%) reporting some grey literature searches or a general grey literature approach, and 45 (40.5%) reporting no information about grey literature searches (MECCIR R39).

OPEN IN VIEWER

Figure 3

Number of reviews fully, partially or not adhering to standards related to reporting of search strategies (MECCIR R35, R38 and R39 and PRISMA Items #6,7).

With regard to searching Google Scholar, 29 out of the 60 that used it reported a full search strategy, thirteen of which also provided details about how results were selected or screened (e.g., number of pages screened, etc.). Nine reviews provided a list of search terms used in lieu of an exact search strategy for Google Scholar, and 18 gave no details about how the Google Scholar search was conducted.

Most reviews reported search strategies that used Boolean operators (90.0%), database syntax (83.8%), and phrase searching (80.2%) correctly. Most reviews (89.2%) also used appropriate variations on keywords, such as using stemming and spelling variations. Only a little over a half (58.6%) used specific subject headings when searching databases with controlled vocabulary.

For those reviews that indicated using some database limit or filter (such as a limit on publication date range or language), 75.8% provided a justification for the use of those limits (MECCIR C35, R36). No reviews indicated that peer review had been conducted on the search strategies prior to running searches.

The Campbell Collaboration published updated searching and information retrieval guidance in 2017 (Kugley et al., 2017). Of the 111 systematic reviews published since January 2017, 75 of those reviews conducted the searches after the publication of the 2017 guidance, according to the search dates provided. However, very few reviews (n = 12) referenced the Campbell Collaboration's information retrieval guidance document. To assess if conduct or reporting practices have improved over time, or since the publication of the 2017 guidance document, we charted the percent of reviews fully complying with each variable over time (Figure 4). This reveals no clear trends and little apparent impact of the guidance document, which aligns with its lack of citation. Note that searching guidance for Campbell did exist prior to 2017 (Hammerstrøm et al., 2010), but we did not collect data on the citation of this document in the Campbell systematic reviews we assessed.

OPEN IN VIEWER

Figure 4

Percentage of reviews following recommended practices over time. (a) Shows variables related to the conduct of searches and search methods. Dates refer to the date the most recent search was run. Reviews with dates before 2015 and after 2024 were removed due to low numbers. (b) Shows the variables related to the reporting of searches excluding reviews published in 2024 due to low numbers. Dates refer to the date of publication.

Very few reviews explicitly indicated the use of a validated search hedge (8.1%, n = 9), while more (12.6%, n = 14) indicated that they had adapted a search strategy from a previously published review.

Supplementary search methods

In addition to searching sources directly (i.e., database, websites, etc.), additional methods are recommended for finding unpublished or otherwise missed studies (Papaioannou et al., 2010). This includes handsearching of journal tables of contents, contacting experts, forward citation chasing, backward citation chasing, and searching for studies in related reviews (the latter two of which are mandatory according to MECCIR conduct standards, C29 and C30). While almost all reviews used at least one of these methods, over half did not conduct any handsearching (51.4%, n = 57) or forward citation searching (62.2%, n = 69), and about one-quarter did not report searching references of relevant reviews (22.5%, n = 25) or contacting experts (25.2%, n = 28) (Figure 5). Of those that did conduct handsearching, this was mostly of journal tables of contents as opposed to conference proceedings and the date ranges of handsearched journals were reported in 57.4% of those reviews. Only four reviews reported using listservs to solicit additional studies and input from experts, a strategy described by Kugley et al., 2017 as a potential way to identify unpublished studies.

OPEN IN VIEWER

Figure 5

Supplementary searching methods. The number of reviews reporting the conduct of various supplementary methods typically recommended for systematic reviews.

Involvement of information specialists

Information specialists (IS) or librarians trained in evidence synthesis methods can play an important supportive role in Campbell systematic reviews (Kugley et al., 2017). The degree of IS involvement in conducting searches and how they are acknowledged for their contributions varies. We found that about 45% of reviews (n = 50) did not mention any IS involvement. Twenty-eight percent (n = 32) involved an IS as a co-author and 37.8% (n = 42) involved an IS in some sort of consulting capacity (Figure 6). IS involvement was also acknowledged in different ways, for example in the Roles of Authors section (n = 38), methods section (n = 45) and acknowledgements section (n = 15) of reviews.

OPEN IN VIEWER

Figure 6

Involvement of information specialists. The number of reviews with an information specialist as a co-author, in a consulting role only (i.e., mentioned elsewhere in the paper), or with no IS involvement mentioned. For co-authorship, we discerned to the best of our ability whether or not any authors were librarians or information specialists (as opposed to domain experts or other methodologists). Breakdown by coordinating group is shown (using first coordinating group listed in cases where reviews are cross-listed for more than one, for simplicity).

Figure 7 shows the percentage of reviews complying with different conduct and reporting variables by level of IS involvement. The general trends indicate that those reviews with an IS co-author are more likely to comply with different conduct and reporting practices. Among reporting elements in particular, the charts indicate more frequent reporting of grey literature searches, website URLs, methods for forward citation searching, and information management methods in reviews with IS co-authors. For conduct variables, database syntax was used correctly for 100% of the reviews with an IS co-author versus 67.5% of the reviews with no indicated IS involvement. Similarly, database-specific subject headings tend to be used more often in reviews with IS co-authors (75.0% vs. 42.5%). For the numbers and percentages underlying Figure 7, see Table S3 and S4 in File S3.

OPEN IN VIEWER

Figure 7

Quality of search conduct and reporting with and without information specialist involvement. Percent of reviews with no IS involvement, with indications of consultations only, and with co-authorship in relation to compliance with (a) conduct and (b) reporting best practices. For actual numbers and percentages of adherence to search conduct and reporting standards by IS involvement, see Tables S3 and S4 in File S3.

Other considerations

Timeliness of search

About one-third of reviews noted that the search was updated during the course of the review before publication (35.1%, n = 39). Figure 8 summarizes the data about the time lag between the most recent search date and the publication date. The median number of months between the running of the most recent search and review publication was 20 (min = 1, max = 90). We used a one-way analysis of variance (ANOVA) to compare time lag from running the search to publication date between coordinating groups and found no statistical difference between groups (F(4, 112) = [0.7961], p = 0.53; Figure 8b). MECCIR Conduct standard C37 suggests that searches should be updated within 12 months before publication of the review.

OPEN IN VIEWER

Figure 8

Time lag between search and publication date. (a) Histogram of the number of months elapsing between the most recent search and the date of review publication. (b) Box plot of the time lag between search and publication date by coordinating group. Coordinating groups with only one review have been excluded from the figure.

Source selection and reporting

We found that Campbell reviews search a wide array of sources including multiple bibliographic databases as well as specific sources of grey literature. Lists of databases were routinely reported and database platforms were reported in most cases, which is an important element of reproducibility (Rethlefsen et al., 2021). One element not well reported in Campbell reviews is database dates of coverage when available, which was also found by the assessment by Ramirez et al., 2022 of Education coordinating group reviews. While this is a mandatory reporting item in both the 2019 MECCIR and 2020 PRISMA standards, this item is not included in PRISMA-S. Date ranges of some databases can vary by library subscription, and for reviews with no date restrictions, this information can be helpful to a reader in determining the date range of the search.

Searching grey literature is important for identifying unpublished or ongoing work (Paez, 2017). For reviews in the social sciences, grey literature can be an important source of relevant studies conducted by organizations and government agencies. We found that all Campbell reviews search for grey literature to some degree. However, reporting of the sources of grey literature searched was less consistent in its adherence to best practices and standards than database searches. While grey literature sources were listed in most reviews, URLs of websites were often unreported. Reporting of URLs is a relatively low hanging fruit that could improve transparency of grey literature searching, and could be remedied by providing author templates for search reporting.

Search strategy conduct and reporting

The reporting of exact search strategies is critical to transparency and reproducibility of systematic reviews. Full search strategy reporting for all sources is a mandatory reporting item for both the 2019 MECCIR and PRISMA 2020 standards. Nonetheless, we found that less than 65% of Campbell reviews fully reported database searches or exact search dates. Exact search strategies for grey literature sources were far less common, with only 22.5% of reviews reporting all complete grey literature searches. Reporting of exact grey literature searches is an onerous task due to the poor search capability of many grey literature sources and the need for iteration and multiple search strategies. It requires a well-thought out approach and a plan for documentation. Guidance and training for Campbell authors could be provided to improve grey literature search methods, documentation and reporting.

Google and Google Scholar were used in many reviews, but similarly to grey literature, search strategies and approaches for screening results were underreported. There are many challenges associated with using these platforms for systematic reviews (Gusenbauer and Haddaway, 2020). Yet, there are methods that can sufficiently capture some level of transparency in how they are used (see Briscoe and Rogers, 2021 for approaches to searching Google and a recent update by Briscoe et al., 2024). Again, more explicit guidance for Campbell authors, reporting templates and training can help address this shortcoming.

While we did not assess search quality, we did examine at a high level the use of database syntax, keywords, phrase searching and database subject headings. In general, reported searches in Campbell reviews showed sufficient use of Boolean operators, keyword variations, phrase searching, and database syntax. Subject headings in databases with controlled vocabulary were used less frequently. Understanding the nuances of different database platforms and how to most effectively combine keyword searching with controlled vocabulary is a specialty of information specialists and librarians, and thus their involvement is critical. When an information specialist is not available to collaborate with an author team, for example in resource-limited settings without libraries and library staff, review by trained information specialists can help mitigate errors and problems in the search before searches are run. For example, a 2018 study by Spry and Mierzwinski-Urban, 2018 found that search strategy peer review can result in additionally retrieved studies in rapid reviews. While Campbell reviews do go through a routine peer review process by an information specialist, no Campbell reviews reported peer review of search strategies outside of that standard process. Yet, this additional step can be a useful way for review teams to strengthen their protocols prior to submission and identify significant biases in their search early in the process. Moreover, after protocol review, which only requires a search strategy for the main database, the search must be adapted across many sources prior to running it. This is an entry point for potential errors, and a search peer review after search adaptation but prior to running the search can mitigate such problems. Frameworks like PRESS (McGowan et al., 2016) can be a useful tool for this added layer of peer review. That said, peer review is not a replacement for information specialist involvement in designing and running searches, as described in more detail below.

It should be noted that the search guidance document originally published in 2017 by Kugley et al., 2017 is currently, as of the writing of this manuscript, undergoing an update. We found a surprising lack of reference to this 2017 document in the reviews we looked at. This document contains detailed guidance for the conduct and reporting of searches, as well as useful lists of sources for different topic areas covered by Campbell, and yet has apparently been underutilized by Campbell authors. With its update, we recommend efforts to increase visibility to authors and other outreach to improve uptake and knowledge translation of this guidance. This could go hand in hand with promotion of the new Campbell training course for systematic reviews and meta-analysis (Valentine et al., 2022). This online self-paced course includes a detailed unit on searching, and could help improve search conduct and reporting particularly when author teams lack information specialist co-authorship.

Supplementary search methods

The use of additional methods beyond database and grey literature searching can help surface unpublished or ongoing work, or studies not found by other means for various reasons. A recent scoping review has assessed citation chasing practices in systematic reviews. They found that of the 47 studies assessing citation chasing in evidence synthesis, 96% found added value for this approach (Hirt et al., 2023). Backward citation chaining (i.e., looking through references of included studies) is required by MECCIR, and commonly conducted in Campbell reviews. Forward citation chaining may also be useful for identifying new studies but is far less common in Campbell reviews. More research is needed to determine the added value of forward citation chaining for Campbell reviews.

Handsearching is another method typically recommended for systematic reviews, and yet conducted by only about half of the included Campbell reviews published since 2017. Handsearching can be a time-consuming and labour-intensive task (Cooper et al., 2020), however, new tools like Paperfetcher can make this task more efficient (Pallath and Zhang, 2023). In addition, clear guidelines about how to conduct handsearching (i.e., how many back issues to review, etc.) can be difficult to find. Moreover, the added value of handsearching for Campbell reviews is not clear.

Role of information specialists

Studies have demonstrated that the involvement of information specialists in systematic reviews improves search reporting (Aamodt et al., 2019; Meert et al., 2016; Pawliuk et al., 2024; Rethlefsen et al., 2015). Moreover, a recent study by Ramirez et al., 2022 examined adherence to published protocols of Campbell reviews from the Education Coordinating Group, and found that while overall adherence was good, it was better when reviews reported librarian involvement. We found similar trends across most conduct and reporting variables assessed as indicated by Figure 7 showing a general increase in adherence to many of these standards when an information specialist is a co-author compared to when an information specialist is not mentioned or only consulted. In particular, there was better reporting of sources and searches in information specialist-coauthored Campbell reviews.

While involving an information specialist in Campbell reviews is not required, it is highly recommended across Campbell's guidance and training (Kugley et al., 2017; Valentine et al., 2022; Wilson et al., 2023). Moreover, all Campbell protocols and reviews are peer reviewed by an information specialist prior to publication. Not all author teams have access to trained librarians and information specialists. Support from Campbell editorial teams could help increase information specialist involvement, in terms of consultations or pre-submission peer reviews. Figure 6 indicates some differences across coordinating groups with respect to information specialist involvement. Working with individual coordinating groups on their workflows and author support approaches could help to develop group-specific information specialist support to address access and use of information specialist expertise. This in turn could improve the quality of conduct and reporting of search methods.

Other considerations

In addition to general conduct and reporting of searches, we were interested in determining the degree of time lag between the most updated search for a review and its publication. Systematic reviews can quickly become outdated with the pace of research publishing ever increasing, and the lack of an updated search strategy during the review process can exacerbate this problem. In fact, across all coordinating groups, we found a median time lag of nearly 2 years (20 months) between the most recent search and publication date, although this is within the 24 months recommended by the AMSTAR critical appraisal tool (Shea et al., 2017). The 2019 MECCIR standards highly recommend that searches be rerun within 12 months of the publication of the reviews (Methods Group of the Campbell Collaboration, 2019a), and this is a mandatory item for Cochrane reviews (Higgins et al., 2023). The length of time for peer review of Campbell Collaboration systematic reviews may contribute to this outdatedness, however we do not have data to support this idea. The current challenges in academic publishing broadly related to finding peer reviewers may exacerbate this problem (Flaherty, 2022). Workflows or guidance that encourage authors to work on updating searches during the peer review process may help to mitigate these time lags.