Ethical considerations for sharing aggregate results from pragmatic clinical trials

Abstract

A growing literature has explored the ethical obligations and current practices related to sharing aggregate results with research participants. However, no prior work has examined these issues in the context of pragmatic clinical trials. Several characteristics of pragmatic clinical trials may complicate both the ethics and the logistics of sharing aggregate results. Among these characteristics include that pragmatic clinical trials may affect the rights, welfare, and interests of not only patient-subjects but also clinicians, meaning that results may be owed to a broader range of groups than typically considered in other research contexts. In addition, some pragmatic clinical trials are conducted under a waiver of informed consent, meaning sharing results may alert participants that they were enrolled without their consent. This article explores the ethical dimensions that can inform decision-making about sharing aggregate results from pragmatic clinical trials, and provides recommendations for that sharing. A central insight is that healthcare institutions—as key partners for the conduct of pragmatic clinical trials—must also be key partners in decision-making about sharing aggregate pragmatic clinical trial results. We conclude with insights for future research.

Keywords

Pragmatic clinical trials ethics aggregate results

Introduction

There is a rising expectation for sharing aggregate research results with research participants and that doing so is an ethical obligation. By aggregate research results, we mean lay-friendly summaries of overall study findings, as opposed to either individual research results or scholarly manuscripts written for a scientific or other specialty audience. The Declaration of Helsinki instructs that “[a]ll medical research subjects should be given the option of being informed about the general outcome and results of the study.”¹ Similarly, a recent report from the National Academy of Sciences, Engineering, and Medicine advises sharing lay summaries of aggregate research results no later than 12 months following study completion.² Sharing aggregate results with participants is supported by the ethical principle of respect for persons, recognizing participants’ contributions to research.³ These guidances and arguments notwithstanding, sharing aggregate results with participants remains fairly uncommon. A 2018 article found merely 2% of clinical trials completed or terminated within the prior 3 years provided lay-friendly aggregate results to study participants.⁴

A growing literature has explored the ethical obligations and current practices related to sharing aggregate results.^5–7 However, no prior work has examined these issues in the context of pragmatic clinical trials (PCTs). Pragmatic trials, as distinguished from explanatory trials, are “designed for the primary purpose of informing decision-makers regarding the comparative balance of benefits, burdens, and risks of a biomedical or behavioral health intervention at the individual or population level,”⁸ and generally are conducted with participants and in settings that aim to reflect real-world conditions.

Several features common to many PCTs may complicate both the ethics and the logistics of sharing aggregate results. Among these features include their potentially large scale (with some involving over a hundred thousand patient-participants),^9–11 their inclusion of clinicians as research participants (as opposed or in addition to patients), their potential use of waivers or alterations of consent, the nature of the interventions under study, and the key role of healthcare systems for their successful conduct.

In this article, we explore the ethics and logistics of sharing aggregate results from PCTs. We begin by describing arguments for and against sharing that have been offered in the context of explanatory research and research with biobanks, and examine how the features of PCTs may complicate these arguments. We then note the important role for institutions in decision-making about sharing aggregate results from PCTs and suggest recommendations to facilitate appropriate decision-making about results sharing. We conclude by outlining an agenda for future research. While we focus our analysis on the United States (US), the underlying ethical considerations can inform decision-making in other contexts.

Ethical arguments for sharing aggregate research results and their application to PCTs

Respect and reciprocity

Sharing aggregate research results is supported by the ethical principles of respect and reciprocity.^5,12 Providing participants with information about what was learned from a study is consistent with the principle that individuals be treated as ends in themselves and not as mere means, while recognizing the integral role of research participants in the development of new knowledge.^3,6,13 Research participants invest time and assume risks and burdens to facilitate the generation of new knowledge. Correspondingly, researchers should ensure that participants are informed about the insights gained from their participation. Relatedly, sharing aggregate results can demonstrate appreciation for the service that participants provide to advance research.⁶ As many study participants are motivated by the goal of helping others, sharing results informs them of the ends to which they contributed. Sharing aggregate results also appears consistent with participants’ expressed preferences, as studies consistently find a majority of research participants indicate that they want to be provided with summary results about studies with which they were involved.^14–16

How might these considerations inform decision-making about sharing aggregate results from PCTs? For many PCTs, respect-based arguments similarly support sharing with participants, serving as a means to demonstrate appreciation for participants’ contributions to research. However, at least two features of PCTs may complicate the analysis of respect- and reciprocity-based obligations. First, PCTs often involve comparing validated interventions that are part of usual care. There is debate over the extent to which these studies pose research risks.^17,18 Nonetheless, at least some of these studies pose substantially lower added research risks and burdens as compared to early-phase explanatory trials. This fact may lessen the extent to which reciprocity grounds an obligation to share aggregate results. Second, PCTs may broaden the types of groups to whom aggregate results might be owed. Previous scholarship and guidance on sharing aggregate results have generally focused on sharing to patient-participants.^5,7,16,19 Yet, many PCTs explore interventions targeted at clinicians, who themselves might be considered research participants—and therefore who may be owed information about what was learned as a result of their contributions.²⁰ An additional argument in favor of sharing aggregate results with clinicians—and, importantly, the leadership of the health systems in which they practice—is that doing so can support the social value of the research, as sharing aggregate results with stakeholders who can facilitate adoption of the results into practice may improve implementation of findings.

Promoting participant trust and research(er) trustworthiness

Participant trust refers to the extent to which participants believe researchers act appropriately. Researcher trustworthiness refers to the extent to which researchers in fact act appropriately and, therefore, merit participant trust. It is obviously important for researchers to act appropriately and, therefore, be trustworthy. In addition, assuming they are, it is valuable for participants to trust them. Trusting researchers who are trustworthy involves having an accurate impression of them. Moreover, having trust in research and researchers likely increases individuals’ support for and willingness to participate in research. To the extent the research is ethical and socially valuable, that trust is a good thing.

Sharing aggregate research results with participants has been proposed as a means to increase public trust in researchers and in the research process,^5,13,21 and, relatedly to raise public awareness of the importance of research (which may in turn improve trial accrual).^13,22 These arguments apply to many PCTs as well, providing additional reasons to share aggregate results for these trials. Adoption of a general practice of disclosing aggregate results increases the transparency of the research and helps to assure the public that researchers are acting appropriately. This effect can be realized even when the public does not access the information that is disclosed. Simply knowing that researchers consistently make their results available to participants provides some assurance that they are not hiding anything, which may improve trust.

When PCTs are conducted under a waiver of consent, the impact of sharing aggregate results on public trust in and support for research may be more complicated. Sharing aggregate results might be the means by which individuals first learn they were enrolled in a study. This is particularly a concern for trials that waive consent and do not provide other means to prospectively notify potential participants about the research activity. In that case, individuals’ trust in research and the participating healthcare system(s) might be undermined.²³ In particular, they might assume they were not informed about the research because it was inappropriate in some way or other.

In our view, this risk, while important, does not suggest that researchers should refrain from sharing aggregate results for studies conducted under a waiver of informed consent. Instead, it provides a reason to notify individuals that they are participating in research, in which case the receipt of aggregate results would not be the first time they are provided this information. Arguably, even when there are compelling reasons not to notify participants prospectively, participants still have a claim to be informed about the aggregate results, despite the potential for undermining trust. Further empirical research is needed to evaluate the impact of sharing aggregate results from studies conducted under a waiver of consent and to develop strategies by which to promote trustworthiness and (appropriate) trust.

Countervailing considerations for sharing aggregate results

Consequentialist-based considerations

Several concerns have been raised about sharing aggregate results with participants. One set of concerns relates to the potential resultant consequences, including harm to participants (and/or their families) from sharing.⁵ Among the concerns raised about disclosure include fear or anxiety about the implications for an individual’s own health status (including when individuals are unaware of the trial arm to which they were assigned), “survivor guilt” for those assigned to a trial arm ultimately demonstrated as superior, and emotional distress among family members if the research participant has died.^5,13,19 Others have noted potential challenges related to health literacy, including that individuals may not understand the implications of aggregate results, or, even more concerningly, that they may use them as the basis for incorrect or harmful medical decisions.^7,13 Additional concerns relate to the relative value of sharing when trials do not meet their enrollment targets, are stopped early for futility, or otherwise fail to produce statistically meaningful results, and the potential for participants to misunderstand the implications of incomplete results and become less willing to support the conduct of future studies. Further concerns have been raised in related contexts regarding the downstream impacts of disclosure on other stakeholders, particularly clinicians, who may be tasked by their patients with providing additional interpretation and consultation about study results (yet receive minimal if any compensation, training, or other support to help them in doing so).⁵

The aforementioned concerns have largely been raised in the context of explanatory trials involving novel therapeutics. However, at least two consequentialist-based considerations seem especially relevant for PCTs. The first relates to the relative value of sharing, including whether participants do, in fact, desire aggregate results from PCTs, or whether there are some types of PCTs from which patient-participants are less likely to desire aggregate results. Consider, for example, that some PCTs test interventions for which patients would not generally be consulted outside the research context, such as comparing staffing ratios or testing whether one clinical decision support tool better promotes the uptake of evidence-based practice than another. While it is a testable empirical question, it is not clear that participants do, in fact, desire to receive this information, nor that it would be valuable to them.

Second, the large scale of many PCTs raises the potential for substantial downstream impacts of disclosure for clinicians, health systems, and payors, if patient-participants seek assistance from their clinicians to support interpretation of study results or desire additional clinical management as a result of study findings.^24,25

Logistical and practical barriers to sharing

Even researchers who express support for sharing aggregate study results may face logistical and practical barriers when trying to do so. For example, researchers describe barriers navigating Institutional Review Boards (IRBs) and related privacy policies, with several empirical reports indicating that IRBs may be an obstacle to sharing aggregate results, especially if such sharing was not originally specified in the protocol nor communicated to participants within informed consent forms.^7,26 Other barriers described by researchers as undermining sharing include the lack of funding support (as grants often do not pay for these activities), researcher inexperience or discomfort in writing for lay audiences, and competing academic incentives (in that promotion and tenure criteria generally emphasize dissemination to academic audiences rather than to research participants or the general public).^6,7,26

Sharing aggregate results from PCTs may exacerbate some of these logistical challenges. Many PCTs have large sample sizes distributed across multiple health systems. The size of these studies may render some notification strategies impracticable. For example, a recent study assessing various approaches to sharing aggregate results from an ovarian cancer study found that mailing printed summaries had a mean cost of £14 (US$17.6) per participant—a cost the study authors said was small relative to the average per participant cost per trial.²⁷ This cost looks quite different, however, when applied to PCTs involving tens or even hundreds of thousands of individuals, particularly when using mailed letters or other individualized approaches, given substantially larger postage costs and associated staff time for preparation.

Large sample sizes characteristic of many PCTs may require consideration of less resource-intensive disclosure strategies, including institutional newsletters, flyers in clinics where the research was conducted, and patient portal messages. While these disclosure strategies may be less personalized than other approaches, the opportunity for embedded PCTs to share aggregate results through pre-existing dissemination mechanisms within health systems may actually lessen barriers to sharing for some PCTs. However, reliance on clinical systems may create other challenges, particularly when considering that PCTs often rely on extant data,^28,29 meaning that there may be several years from when data are collected to when aggregate results would be available for dissemination. During this time, contact information for at least some participants will have become out of date, and others will no longer be seeking care from the health system into which the PCT was embedded (meaning they will not be reachable through patient portals or system-wide communications). These logistical considerations need not necessarily undermine arguments for sharing aggregate results in PCTs but suggest operational challenges in doing so.

Privacy and confidentiality considerations

Finally, contacting participants to share study results may present threats to privacy and confidentiality, as sharing aggregate results may require researchers to collect additional personal data from participants, beyond that which would otherwise be required for the conduct of the research study, such as name, phone number, or other contact information.¹⁹ This concern is particularly salient for trials for which the only reason to collect additional identifying information would be for the purpose of disclosing results through personalized approaches, such as mailed letters or patient portal messages. Analysis based on these considerations will likely be similar for PCTs as for explanatory trials or other study types. However, these considerations would not preclude the ability to share information through less personalized approaches, such as via institutional newsletters.

Role for healthcare institutions

Existing literature on sharing aggregate research results largely emphasizes the role of researchers, and, to a more limited extent, funders, insofar as they can support the associated costs. Yet, as we and others have noted elsewhere, the healthcare institutions in which PCTs are embedded are key stakeholders for PCTs,^30–32 and, correspondingly, should be directly involved in their conduct, including in decisions about whether and how to return aggregate results. While researchers will still play an important role in sharing aggregate research results, institutions must be active partners in these efforts.

One set of reasons that healthcare institutions are key stakeholders for decision-making about sharing aggregate research results from PCTs is based on feasibility. Investigators may not have access to contact information for all individuals enrolled in PCTs and may require institutional involvement to reach the appropriate population. In addition, institutions often control several potential means by which to communicate results to participants, including patient portals, patient-facing newsletters, and signage or other notifications within clinics, and clinician-facing communications, such as grand rounds and emailed newsletters.

Institutional involvement in sharing aggregate research results is also supported from considerations of efficiency. Study teams often lack appropriate expertise in developing lay-friendly communications, including about research results.^7,33 Institutions could mitigate this barrier by creating a communication core, akin to other service cores offered by many research institutions (e.g. biostatistics, community engagement) to support investigators in disseminating lay-friendly general summaries.³⁴ Other strategies include developing standardized content for sharing aggregate results,²⁶ with introductory general descriptions about clinical trials, and a template structure designed to capture the types of information that research participants generally request (e.g. what the study is about, who was included, what the study found, and where the participants can go for more information).²¹ Similar standardized approaches to dissemination could be adopted for other groups, such as clinicians.

Finally, several studies have identified the fact that existing incentive structures overwhelmingly prioritize academic publication for promotion and tenure over efforts to share research findings with study participants.^7,26 Institutions might correct this disincentive by restructuring promotion criteria so as to better recognize researchers who share aggregate study results with research participants, such as by considering these efforts within promotion packages as service contributions.³³

Given the diversity of healthcare institutions engaged in PCTs and the organizational structures within them, the stakeholders responsible for decision-making may vary across institutions but may include those charged with oversight of clinical and health systems research, such as a Vice Dean for Research within an academic medical center.

Recommendations

This analysis suggests several insights for PCTs (Table 1). First, the importance of demonstrating respect and appreciation for those (patients and clinicians) whose contributions made the research possible suggests that sharing aggregate results should be the presumptive default. For PCTs targeting clinician behavior, for which clinicians may be both the subjects of the research and critical partners to facilitate implementation of study results, the importance of dissemination to align practice with evidence provides an additional important reason in favor of sharing aggregate results. Researchers and institutions may be justified in not sharing aggregate results with all affected stakeholder groups for some PCTs, but they bear the burden to justify why sharing should not occur.

Table 1.

Recommendations for sharing aggregate results from PCTs.

Recommendations
(1) Sharing aggregate results from PCTs should be the default; decisions to NOT share must be justified.
(2) Planning for sharing aggregate results should begin at the earliest phases of the research study.
(3) The health care institutions into which PCTs are embedded should be key partners in sharing aggregate results, including decisions both about what and how to share.
(4) Investigators and health care systems should take particular care when disclosing aggregate results from PCTs conducted under waivers or alterations of consent. Proactive disclosure of research without consent, and explanations for why consent was not obtained, may promote trustworthiness of investigators and health care institutions.

Second, planning for aggregate results sharing should begin at the earliest phases of research and should be incorporated into study protocols, funding applications, IRB applications, and, where appropriate, informed consent processes.²⁶ While early planning is recommended for all trials, it may be particularly important for PCTs, given the likely need for coordination with partners outside of the research team members, including healthcare system operational leaders. Research funders, institutions, and IRBs can support this planning through establishing and maintaining expectations that aggregate results be shared. For example, funders could provide support within study budgets to facilitate sharing aggregate results. At initial review, IRBs should review the process and plan for disclosure of aggregate results, including which stakeholder groups will receive results, and should prompt investigators to justify any decision not to disclose. IRBs and others responsible for regulating research conduct should work with investigators to clarify privacy policies and how they relate to sharing aggregate results with PCT participants and other stakeholders.³³ Of note, prior research has indicated a gap between approved researcher plans and actual practices for disclosure,²⁶ hence, IRBs should take advantage of continuing review (if there is one) and study close out to promote compliance with the initially approved plan. Finally, for PCTs that may not have included language about aggregate sharing within consent forms, IRBs should not consider this as a categorical reason to prohibit such sharing.

Third, as noted previously, institutions are key partners in the conduct of PCTs, and therefore also must play a key role in both the decision-making about and ultimate implementation of sharing aggregate results from these trials. In addition to the specific strategies described earlier, institutions should work with research teams to identify the relevant groups to whom aggregate results should be shared and to strategize accordingly about the most appropriate means by which to share. Institutional resources, including standardized templates or consultation services, may also support these efforts.

Fourth, particular consideration must be given when sharing aggregate results from PCTs conducted with waivers of consent. As described earlier, if not done thoughtfully, disclosure of results from unconsented research may undermine, rather than enhance, trust. Researchers and institutions should work cooperatively to develop dissemination strategies that demonstrate appropriate transparency about PCTs. Communications about aggregate results could include a description of the research, why it occurred without individual-level consent, and the recognized importance of communicating about ongoing minimal risk efforts to improve care delivery. Engaging patient representatives may support the ability to tailor messages accordingly.³⁵

The challenges presented by sharing aggregate results from PCTs conducted under a waiver of consent provide further reason that institutions and researchers should, whenever possible, consider ways to promote transparency about research activities—even, or perhaps especially, for studies for which individuals are not offered a choice about study enrollment. If part of the challenge is that aggregate results disclosure may be the first notification an individual has that they were enrolled in a research study, researchers and institutions might mitigate confusion or distrust by seeking opportunities to notify individuals about PCTs underway within the health system, including general notifications about system-level commitments to embedding research into ongoing clinical care as part of a broader commitment to continuous care improvement, and specific notifications about particular research studies in which individual patients or clinicians may have been enrolled.

Future research questions

Future research is needed to guide ethical decision-making and practice for sharing aggregate results from PCTs. Particular questions of importance include understanding PCT patient-participant preferences for disclosure, including the types of PCTs for which results are more or less valued, and whether and how to handle sharing for research that takes place without specific consent. Additional research might explore the impact of receiving results, including the consequentialist-based concerns that are commonly offered as reasons not to disclose. Finally, descriptive studies of current practices can promote knowledge sharing across investigators of potential strategies for sharing. These descriptions, along with evaluations of the impact of various disclosure strategies, can support the development of best practices for sharing results and for sharing with different types of stakeholders.

Conclusion

Sharing aggregate study results from PCTs presents distinct ethical considerations as compared to sharing results from explanatory trials, though important lessons can be drawn from the latter. PCTs broaden the groups to whom results may be owed, based on ethical considerations of respect and reciprocity. Their size may exacerbate logistical challenges related to disclosure, yet their embedded nature may also offer additional opportunities for sharing results through existing clinical communication systems. In addition, PCTs conducted under a waiver of consent require careful decision-making about how to share in a way that promotes, rather than undermines, trust. In all aspects of these decisions, health care institutions are critical partners for ethical conduct when deciding whether and how to share aggregate research results. Study teams should therefore collaborate with their health system partners from the earliest stages of trial planning to identify the stakeholders to whom results should be shared, and the appropriate mechanisms for doing so.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported within the National Institutes of Health (NIH) Pragmatic Trials Collaboratory through cooperative agreement U24AT009676 from the National Center for Complementary and Integrative Health (NCCIH), the National Institute of Allergy and Infectious Diseases (NIAID), the National Cancer Institute (NCI), the National Institute on Aging (NIA), the National Heart, Lung, and Blood Institute (NHLBI), the National Institute of Nursing Research (NINR), the National Institute of Minority Health and Health Disparities (NIMHD), the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), the NIH Office of Behavioral and Social Sciences Research (OBSSR), and the NIH Office of Disease Prevention (ODP). This work was also supported by the NIH through the NIH HEAL Initiative (award no. U24AT010961). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NCCIH, NIAID, NCI, NIA, NHLBI, NINR, NIMHD, NIAMS, OBSSR, or ODP, or the NIH or its HEAL Initiative. Dr Morain and Ms Brickler also received support for this work through a Making a Difference grant from The Greenwall Foundation.

ORCID iDs

Stephanie R Morain

Abigail Brickler

Patricia Pearl O’Rourke

Caleigh Propes

References

World Medical Association. Declaration of Helsinki–ethical principles for medical research involving human subjects, 2013, https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/

Committee on the Return of Individual-Specific Research Results Generated in Research Laboratories, Board on Health Sciences Policy, Health and Medicine Division, National Academies of Sciences, Engineering, and Medicine. Returning individual research results to participants: guidance for a new research paradigm. Washington, DC: National Academies Press, 2018.

Shalowitz

Miller

FG.

Disclosing individual results of clinical research: implications of respect for participants. JAMA 2005; 294(6): 737–740.

Getz

Farides-Mitchell

Assessing the adoption of clinical trial results summary disclosure to patients and the public. Expert Rev Clin Pharmacol 2019; 12(7): 573–578.

Beskow

Burke

Fullerton

, et al. Offering aggregate results to participants in genomic research: opportunities and challenges. Genet Med 2012; 14(4): 490–496.

Sgro

Maurer

Nguyen

, et al. Return of aggregate results to study participants: facilitators, barriers, and recommendations. Contemp Clin Trials Commun 2023; 33: 101136.

Long

Purvis

Flood-Grady

, et al. Health researchers’ experiences, perceptions and barriers related to sharing study results with participants. Health Res Policy Sys 2019; 17(1): 25.

Califf

Sugarman

Exploring the ethical and regulatory issues in pragmatic clinical trials. Clin Trials 2015; 12(5): 436–441.

Jarvik

Meier

James

, et al. The effect of including benchmark prevalence data of common imaging findings in spine image reports on health care utilization among adults undergoing spine imaging: a stepped-wedge randomized clinical trial. JAMA Netw Open 2020; 3(9): e2015713.

10.

Huang

Septimus

Kleinman

, et al. Chlorhexidine versus routine bathing to prevent multidrug-resistant organisms and all-cause bloodstream infections in general medical and surgical units (ABATE infection trial): a cluster-randomised trial. Lancet 2019; 393(10177): 1205–1215.

11.

Huang

Septimus

Kleinman

, et al. Targeted versus universal decolonization to prevent ICU infection. N Engl J Med 2013; 368(24): 2255–2265.

12.

Emanuel

EJ.

What makes clinical research ethical?

JAMA 2000; 283(20): 2701–2711.

13.

Fernandez

Kodish

Weijer

Informing study participants of research results: an ethical imperative. IRB 2003; 25(3): 12–19.

14.

Shalowitz

Miller

FG.

Communicating the results of clinical research to participants: attitudes, practices, and future directions. PLoS Med 2008; 5(5): e91.

15.

Long

Stewart

Cunningham

, et al. Health research participants’ preferences for receiving research results. Clin Trials 2016; 13(6): 582–591.

16.

Taylor

Manov

Crew

, et al. Preferences for updates on general research results: a survey of participants in genomic research from two institutions. J Pers Med 2021; 11(5): 399.

17.

Kim

Kimmelman

Practical steps to identifying the research risk of pragmatic trials. Clin Trials 2022; 19(2): 211–216.

18.

Asch

Ziolek

Mehta

SJ.

Misdirections in informed consent — impediments to health care innovation. N Engl J Med 2017; 377(15): 1412–1414.

19.

Purvis

Long

Eisenberg

, et al. First do no harm: ethical concerns of health researchers that discourage the sharing of results with research participants. AJOB Empir Bioeth 2020; 11(2): 104–113.

20.

Smalley

Merritt

Al-Khatib

, et al. Ethical responsibilities toward indirect and collateral participants in pragmatic clinical trials. Clin Trials 2015; 12(5): 476–484.

21.

Zimmerman

Perry

Hanlen-Rosado

, et al. Developing lay summaries and thank you notes in paediatric pragmatic clinical trials. Health Expect 2022; 25(3): 1029–1037.

22.

Partridge

Winer

EP.

Informing clinical trial participants about study results. JAMA 2002; 288(3): 363–365.

23.

Nicholls

Carroll

Goldstein

, et al. Patient partner perspectives regarding ethically and clinically important aspects of trial design in pragmatic cluster randomized trials for hemodialysis. Can J Kidney Health Dis 2021; 8: 205435812110328.

24.

Morain

Mathews

DJH

Weinfurt

, et al. Stakeholder perspectives regarding pragmatic clinical trial collateral findings. Learn Health Sys 2020; 5(4): e10245.

25.

Morain

Mathews

DJH

Geller

, et al. Identification and management of pragmatic clinical trial collateral findings: a current understanding and directions for future research. Healthcare 2021; 9(4): 100586.

26.

Schroter

Price

Malički

, et al. Frequency and format of clinical trial results dissemination to patients: a survey of authors of trials indexed in PubMed. BMJ Open 2019; 9(10): e032701.

27.

South

Bailey

Bierer

, et al. Site staff perspectives on communicating trial results to participants: cost and feasibility results from the Show RESPECT cluster randomised, factorial, mixed-methods trial. Clin Trials 2023; 20(6): 649–660.

28.

Simon

Shortreed

Rossom

, et al. Principles and procedures for data and safety monitoring in pragmatic clinical trials. Trials 2019; 20(1): 690.

29.

Richesson

Green

Laws

, et al. Pragmatic (trial) informatics: a perspective from the NIH Health Care Systems Research Collaboratory. J Am Med Inform Assoc 2017; 24(5): 996–1001.

30.

Morain

Largent

Think pragmatically: investigators’ obligations to patient-subjects when research is embedded in care. Am J Bioeth 2023; 23(8): 10–21.

31.

Larson

Tachibana

Thompson

, et al. Trials without tribulations: minimizing the burden of pragmatic research on healthcare systems. Healthcare 2016; 4(3): 138–141.

32.

Weinfurt

Hernandez

Coronado

, et al. Pragmatic clinical trials embedded in healthcare systems: generalizable lessons from the NIH Collaboratory. BMC Med Res Methodol 2017; 17(1): 144.

33.

McElfish

Long

James

, et al. Characterizing health researcher barriers to sharing results with study participants. J Clin Trans Sci 2019; 3(6): 295–301.

34.

Cook

Mayers

Goggins

, et al. Assessing research participant preferences for receiving study results. J Clin Trans Sci 2020; 4(3): 243–249.

35.

Bruhn

Campbell

Entwistle

, et al. What, how, when and who of trial results summaries for trial participants: stakeholder-informed guidance from the RECAP project. BMJ Open 2022; 12(3): e057019.