Abstract
Bridget Pratt and Adrian Harper ( 2021) conducted a comparison of articles identified through six electronic literature searches. Their aim was to “identify ethics literature… that discussed combatting neo-colonial models of research”. They used manifest content analysis to compare the conceptual content of articles from the fields of global health (GH) and Australian Aboriginal health (AH). This innovative application of a literature review approach from literary and media studies, to health sciences in which literature reviews have traditionally focused on synthesizing evidence about intervention effectiveness, should be commended. It has potential to advance theoretical understandings of ethics in health research. However, I argue here that Pratt and Harper’s (2021) search strategy has several weaknesses, which suggests that their results must be interpreted with caution.
Peer review of electronic search strategy (PRESS) guidelines
This commentary discussed the search strategy Bridget Pratt and Adrian Harper used in a comparative review of Aboriginal and Global health literature, with reference to the peer review of electronic search strategy (PRESS) guidelines (McGowan et al., 2016; Sampson et al., 2009). The PRESS guidelines are designed to optimize recall (ensure no relevant articles are missed) and precision (ensure that only relevant articles are included) in literature sets identified through electronic searches (Sampson et al., 2009). They recommend that researchers work with a librarian to, inter alia: translate key concepts in the research question/aim into groups of search terms (parameters), identify all relevant synonyms (terms) to include in each parameter; and select appropriate operators and truncations to combine terms and parameters.
Authors’ Search Strategy
Pratt and Harper (2021) initially developed four search strategies working with a librarian (two each for Australian Aboriginal health (AH) and global health (GH)). In response to my peer review comments, and as an attempt to ensure that “relevant… articles had not been missed due to [using] different search terms”, they later conducted two “supplementary searches”, which differed from the four original searches and identified a significant number of missed articles (Pratt and Harper, 2021). However, the supplementary searches did not align the (a) databases, (b) parameters or (c) terms used to identify literature in each field (Table 1), nor optimize recall and precision.
* Core terms are presented without required truncations and Boolean operators for specific databases searched. Bold text indicates term is expected but does not represent the concept in which it was included.
Databases
The databases searched to identify literature were EMBASE and Medline (Ovid) (AH and GH), Scopus (GH only), and Informit (Aboriginal & Torres Strait Islander combined) (AH only). Unlike Scopus, The Informit collection includes ‘grey’ literature (e.g. government reports) (National Library of Australia, undated). Thus, grey literature was returned by, and needed to be manually screen out of, the AH, but not the GH article set.
Parameters
I expected the search strategy would involve four parameters, each containing a set of synonyms representing a concept related to the research aim (Table 1- Expected Parameters). As the authors did not define their parameters conceptually (they were simply numbered 1–4), I identified the concepts represented within each of their parameters based on the search terms they used (presented in Web Supplement from Pratt and Harper 2021). This revealed that the authors’ searches:
collectively included terms related to each of the four expected concepts included terms representing a fifth concept “strategies” for combatting neo-colonialism; included terms related to two or more concepts in a single parameter; used conceptually different parameters for the AH and GH searches.
Eliding a parameter representing health instructed the search engine to include articles from non-health disciplines, which would need to be manually screened-out, in the AH (but not the GH) searches. Including terms related to several concepts within a single parameter, meant that studies that included a term for any one of the concepts were returned in one, but not the other, literature set. For example, in the GH (but not the AH) search, Parameter 1 included terms representing neo-colonialism, health research and population (Table 1). This reduced the precision of the search, because it instructed the search engine to return articles including a term related to any one of concepts rather than all three concepts. It again increased the manual work of screening out irrelevant articles, which increased the risk of screening errors (Sampson et al., 2009). Using conceptually different parameters to structure the GH and AH searches meant that two conceptually different sets of literature were identified. For example, a parameter to capture the concept “health” was included in the GH but not the AH search. In the AH search, terms representing “strategies” were combined in a single parameter with “neo-colonialism” terms, whereas in the GH search these terms were split into separate parameters (Table 1).
Terms
Terms within each parameter should include all relevant (a) synonyms, (b) spelling variants and (c) truncations. Truncations are operators which instruct the search engine to return articles including variations of the same base word (e.g. participat* returns participate and participation but not participant) (McGowan et al., 2016).
Pratt and Harper (2021) did not include terms to identify United States spelling (e.g. they searched for decolonisation but not decolonization). The truncations and terms they used at times seemed too specific (e.g. research ethics instead of simply ethics). Important synonyms were not included. For example, although the authors defined GH as a field focused on people in “low-and middle-income countries”, they did not include this phrase, nor terms for low-and middle-income country names, in their search strategies. They did not include “neo-colonialism”, a key concept for their review, in any of their search parameters.
Different combinations of synonyms were included in each search strategy. For example, the initial Informit search (AH) included a single “ethics” term in each parameter (i.e. did not include any terms for neo-colonialism, health or population concepts). The “neo-colonial” terms used to search the AH and GH literature differed.
Different search strategies included (different combinations of) terms representing “strategies” for combatting neo-colonialism, such as “participate” and “collaborate”. However, the aim of the study was to identify strategies to combat neo-colonialism. This implies that the strategies are not yet known and cannot be included as search terms.
Implications and Conclusion
The examples I have highlighted show that Pratt and Harper’ (2021) search strategies were weakened by sub-optimal recall and precision. As a result, they (a) likely missed relevant literature (e.g. GH articles that referred to the population using a country name or low- and middle- income country) and (b) actually included irrelevant literature (e.g. studies from non-health disciplines such as Anderson, 2007; Hemming & Others, 2007; Kwaymullina, 2016) in the AH search. Including synonyms for a subset of strategies to combat neo-colonialism potentiated missing articles that used other terms to refer to strategies for combatting neo-colonialism (e.g. empowerment, policy advocacy).
Collectively these weaknesses of the search strategy suggest that the findings of Pratt and Harper's (2021) comparative analysis (e.g. that AH and GH differently emphasis strategies) should be interpreted cautiously. The differences might have been found because the authors compared an incomplete set of GH literature, with an overly inclusive set of literature about research with Aboriginal Australians, that included articles from beyond the field of AH. Establishing true differences would only be possible if all (or a representative sample of) relevant literature and only relevant literature were included in the AH and GH sets.
The PRESS guidelines were developed with the aim of optimizing the recall and precision of electronic search strategies in health and biomedical literature reviews. These are typically focused on synthesizing evidence about intervention effectiveness. My experience suggests that the PRESS guidelines are also a useful resource, and the principle of optimizing recall and precision is an essential foundation, for conceptual reviews such as Pratt and Harper’ (2021) comparison of strategies for combatting neo-colonialism used in AH and GH.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
