Challenges in Use of Practice-based Research Networks for a Medical Device Trial to Detect SARS-CoV-2

Abstract

Introduction/Objectives:

Primary care practice-based research networks (PBRNs) participated in a point of care (POC) device study funded by by the National Institutes of Health and led by the University of Massachusetts Chan Medical School (UMass) to speed the development, validation, and commercialization of POC tests to detect SARS-CoV-2. The purposes of this study were to describe the characteristics of participating PBRNs and their respective collaborators in this device trial and describe complications challenging its execution.

Methods:

Semi-structured interviews were conducted with lead personnel from participating PBRNs and UMass.

Results:

Four PBRNs and UMass were invited to participate and 3 PBRNs and UMass participated. This device trial recruited 321 subjects in 6 months; 65 subjects from PBRNs. Each PBRN and the academic medical center site enrolled and recruited subjects differently. Main challenges identified were having adequate clinic personnel to enroll and aid in consent and questionnaire completion, frequently changing inclusion/exclusion criteria, use of the digital electronic data collection platform, and having access to a −80°C freezer to store supplies.

Discussion:

This trial involved numerous researchers, primary care clinic leaders and staff, and academic center sponsored program staff and attorneys resulting in a resource-intensive endeavor to enroll 65 subjects in the real-world clinical setting of primary care PBRNs with the academic medical center enrolling the rest. Multiple obstacles to standing up the study were encountered by the PBRNS.

Conclusions:

Primary care PBRNs rely largely on the goodwill established between academic health centers and participating practices. For future investigations involving device studies, collaborating PBRN leaders should assess whether recruitment criteria may change, obtain detailed lists of equipment needed, and/or know if the study is likely to be halted suddenly to appropriately prepare their member practices.

Keywords

device trial primary care practice-based research network SARs-CoV-2 COVID-19

Introduction

Primary care practice-based research networks (PBRNs) “are groups of primary care clinicians and practices working together to answer community-based health care questions and translate research findings into practice.”¹ PBRNs are an extension of investigation from large academic medical centers into the community, medical care systems, and other coalitions, which allow late translational research to be executed on populations seeing primary care providers with the goal of transforming rigorously studied clinical innovation into practice.² The advent of PBRNs in primary care 5 decades ago has led to an extensive body of research, derived from community-based practices. PBRNs enable pooling of data across practices and expediate larger sample sizes, broader generalization of results, and shared resources.³ In 2018, physician office visits totaled 860.4 million with 51% of those visits being made to primary care physicians.^4,5 On average that is 2.6 visits per patient per year providing a rich, yet underutilized environment for research.⁵ PBRNs exist in every state with 56 primary care physician PBRNs in the Agency for Healthcare Research and Quality (AHRQ) PBRN Registry.^6,7

In June 2020, a project officer from the Center for Translation Research and Implementation Science at the National Institutes of Health contacted primary care PBRN directors from the AHRQ PBRN Registry about a prospective clinical trial to conduct efficacy testing of various point of care (POC) devices for SARS-CoV-2 diagnostic tests. This initiative was developed by the National Institutes of Health (NIH), National Institute of Biomedical Imaging and Bioengineering, and the Point of Care Technology Research Network, who together launched the Rapid Acceleration of Diagnostics (RADx^SM) Tech program to speed the development, validation, and commercialization of POC and home-based tests to detect SARS-CoV-2.^8-11 The RADx Tech Clinical Studies Core was created by the Center for Advancing Point of Care Technologies in Heart, Lung, Blood, and Sleep Diseases, the Massachusetts Medical Device Development Center at the University of Massachusetts (UMass) Lowell and Medical School with existing infrastructure and multidisciplinary expertise from leading academic healthcare centers across the US through the Point-of-Care Technologies Research Network which is designed to implement diagnostic device studies with diverse populations and settings.¹⁰ The goal was to make SARS-CoV-2 testing available to every American. For this program, researchers, scientists, and/or inventors with rapid testing technology competed for the highly coveted opportunity to be matched with business and manufacturing experts to increase their odds of success and funding for rapid investigation with the goal of submitting Emergency Use Authorization (EUA) to the US Food and Drug Administration (FDA) for their devices.¹² Through the RADx Tech Clinical Studies Core, NIH’s goal was to use multiple clinical research sites across the country to evaluate the variety of testing methods in a community-based setting.

United in a commitment to engage in rigorous scientific research regarding testing for SARS-CoV-2, interested PBRNs offered the opportunity to engage a diverse array of subjects from different geographic locations, healthcare systems, and patient populations. While there have been related historical efforts to harness PBRNS for the investigation of infectious disease epidemiology and management, this opportunity may have been the first time for a group of primary care PBRNs to be involved with a clinical trial for a medical device.^13,14

UMass received the main award for the RADx Tech Clinical Studies Core in July 2020 from the National Heart, Lung, and Blood Institute. Prospective PBRNs submitted proposals to the UMass Chan Medical School by September 2020. Successful applicants received subawards in December and were then onboarded to RADx with support from the UMass Chan Medical School Clinical Research Corps. Subsequently, they participated in a POC device study, which was given the alias of 2322, with a goal of recruiting approximately 350 subjects (See Appendix A for 2322 Trial).

The planning phase for the PBRNs to participate in the 2322 device trial took approximately 6 months, which is about the same as other device trials with the usual challenges of identifying suitable sites and waiting for approval by signing authorities.¹⁵ Extensive training began in March 2021, and subject enrollment began in May with the study ending in September 2021. It took only 9 months to enroll 321 subjects in total with the majority, 256, enrolled at UMass. In total, 65 subjects were recruited by the 3 PBRNs who were compensated per subject for participation and were not involved in the study design, which was dictated by the sponsor.

In this investigation, the University of Iowa (UI) research team sought to describe how participating PBRNs implemented this unique device trial. The purposes of this study were to (1) describe the characteristics of participating PBRNs and their respective collaborators in the 2322 trial, and (2) describe complications challenging its execution. Scrutinized domains included the operation of the investigational device, navigating study procedures, engaging with the Eureka digital research platform,¹⁶ and working with a remote clinical laboratory.

Methods

This was a narrative interview study using semi-structured interviews of each of the 3 research teams at the participating PBRNs and academic medical center for the 2322 device trial. The UI research team, consisting of the principal investigator, project manager, and research assistant (BL, JD, KP) conducted the interviews for each PBRN and the academic center separately. Institutional Review Board (IRB) approval was obtained for this study (UI, IRB ID # 202204667) that did not require written informed consent. Subjects included principal investigators, co-investigators, project managers, and research coordinators/assistants who participated in the trial.

Subject Recruitment

Potential subjects were invited to participate in this study with an exempt information/consent sheet via email that explained the purpose of study, which included evaluating the 2322 device trial implementation, recruitment of subjects, and problems incurred. A week later, the e-mail was resent to non-responders. Invited participants were from 3 PBRNs and an academic medical center (See Table 1). Subjects were compensated $50 for their participation. One invited PBRN did not participate in this study as they were unsuccessful in getting the necessary agreements executed for the trial.

Table 1.

Participating PBRNs, Sites Recruited, Location, Catchment, and 2322 Execution Strategies.

	Network	Sites recruited for study by PBRN	Recruitment strategies	Research staff	Support/community based staff	Subjects enrolled
PBRN
Iowa Research Network (IRENE), Department of Family Medicine, University of Iowa, Iowa City, IA	300 clinicians in 183 practices are members of IRENE serving more than 400 000 patients	Five family medicine offices from Council Bluffs, Davenport, Iowa City, Rockwell, and West Burlington Actual recruitment sites: Council Bluffs, Davenport, and Iowa City	Clinic-based events advertised to established patients and community Car-line testing events advertised to local community In-person recruitment and advertising at community COVID-19 testing sites Cold calling subjects who met criteria via medical record review at the University of Iowa Hospitals and Clinics Advertising to local educational institutions	Three present at all events performing consent, sample collection, processing samples, and running device	Community physicians and nurses completed human subjects, good clinical practice, and protocol training to support studies and collect samples. At 2 offices, staff aided in recruitment and sample collection.	38
Kansas Patients and Providers Engaged in Prevention Research (KPPEPR), Department of Family Medicine & Community Health, University of Kansas, Kansas City, KS	50 clinicians from 50 practices are members of KPPEPR serving approximately 200 000 patients	Single clinic in Fredonia, KS Actual recruitment site: Fredonia	Recruitment of clinic patients during visits	A single research assistant on site consenting and running device; research coordinators and PI supporting recruitment remotely	Office physicians and laboratory staff performed sample collection and PCR testing	7
Oregon Rural Practice-based Research Network (ORPRN), Oregon Health & Science University, Portland, OR	600 clinicians from 400 practices are members of ORPRN serving a large part of the state of Oregon	Federally Qualified Health Center in Hood River, a public health department in The Dalles, urgent care clinic with four locations in Medford (2 locations), Ashland, and Grants Pass, OR Actual recruitment site: Hood River and The Dalles	Clinic-based events advertised to established patients and community	Four present at all events performing consent, sample collection, processing samples, and running device	Provided space and setup within their facilities, access to WiFi, key date/timeframes for access, and outreach to their patients. Site staff did not participate in recruitment or sample collection	20
Non-PBRN
UMass Chan Medical School (UMCMS), Clinical Research Corps of the UMass Chan Medical School, Worcester, MA	Non-PBRN	Non-PBRN	In person recruitment and advertising at community COVID-19 testing sites Advertisements in public places such as community organizations, farmers markets, colleges, and health centers encouraging subjects to come to community events or institutional clinical research center Outreach to patients who tested positive for COVID-19 at UMass-sponsored community events or in ambulatory, emergency department, or inpatient settings throughout the UMass Memorial medical system Supported all PBRN sites from regulatory, operational, and logistical standpoints	9-10 present at all events performing consent, sample collection, processing samples, and running device	Community partner provided space within their facilities, access to WiFi, key date/timeframes for access, and outreach to their patients. Academic medical center staff recruited subject and performed sample collection	256

The interviews were a blend of structured and unstructured questions developed by the UI team. Each participant received the planned content of the interview prior to the interviews conducted and recorded on Zoom Meetings and saved to a secure shared drive accessible only by research team members. The questions for the interviews included key dates, including study start dates and dates of executed contracts, descriptions of sites where study enrollment/activities were conducted, contract agreements at participating PBRN member offices, travel arrangements for research members to recruitment sites, roles and duties of team members participating in the study, use of the 2322 SARS-CoV-2 saliva test equipment, the Eureka digital data collection platform, data entry into Quest Diagnostics, and issues incurred during the trial.

Data Analysis

Structured questions, such as date contracts signed and participating offices/testing lines, were answered by a single participating team member from the PBRNs and the academic medical center and forwarded to our research team. During the interviews, participants described the implementation of the 2322 device trial at their respective sites. This format offered insights from the participant’s view of their implementation issues. Questions asked of the participants included format and process of formal agreement with respective PBRN participating members, description of the clinical setting of recruitment and enrollment, and role of the research team and employees at the respective participating sites. The participants also discussed perception of pros and cons of the 2322 device, the Eureka platform, Quest Diagnostics, and UMass’s communication structure. Specific information from each PBRN and the academic medical center were described.

Results

All principal investigators at each enrolled PBRN and the UMass academic center participated in separate interviews with 3 UI research team members. At KU, the principal investigator (PI) and central project coordinator were interviewed. At OHSU, the PI, central project manager, and coordinator were interviewed. At UMass, the investigator managing the PBRNs, clinical research device logistics program manager, clinical research device associate, and project director were interviewed. Overall, 9 persons participated in the interviews, 3 of whom were male.

Each PBRN’s division of sponsored program’s lawyers reviewed and signed 5 distinct contracts with UMass, Quest, and the 2322 manufacturer prior to initiation of the trial (See Table 2). The contracts were signed over a 5-month time-period for UI, 7-month time-period for KU, and 6-month time-period for OHSU. KU did not sign a Research Testing Agreement as their gold standard PCR specimens were analyzed at their participating clinic’s laboratory.

Table 2.

Contracts Signed to Participate in 2322 Device Trial With UMass.

Name of contract	Type of contract
Federal Demonstration Partnership (FDP) Fixed Rate Clinical Research Subaward	Subaward from UMass to each site to participate in the trial
Standard Material Transfer Agreement for the Transfer of Human Specimens	Legal contract to send collected specimens from each site to UMass
Confidential Disclosure Agreement	Legal agreement outlining information UMass and the PBRNs will share with each other
Multi-Party Equipment Transfer and Collaborative Research Agreement	Legal agreement between UMass, the company owning the 2322 test device, and the participating PBRN
Research Testing Agreement	Legal agreement between the PBRN and Quest Diagnostics for testing specimens and reporting results

Each PBRN recruited and enrolled a convenience sample of subjects. Implementation varied by site. Each PBRN’s recruitment strategies, staffing models, and community involvement are summarized in Table 1. KU chose to implement in a community-based practice and placed a 2322 device in an office, left it there without the research team being onsite, and completed enrollment in a traditional clinical setting. In contrast, OHSU and UI research teams went to their respective sites and conducted the study, with a difference that UI had staff at the clinics assist with the trial and OHSU did not. UI streamlined the process to have a research team member assigned to a specific task, for example, 1 person completed all Quest Diagnostic laboratory orders, another worked with each study participant to complete the eConsent and surveys on Eureka prior to their appointment, and another collected the specimens. In addition, UI research staff had each potential subject’s paperwork and specimen labels printed ahead of time, which facilitated efficient completion of the study appointment. OHSU had a single study team member complete all tasks for each participant.

PBRN member offices faced multiple challenges in study start-up and protocol execution. In particular, subject inclusion criteria changed multiple times, challenging the PBRNs and leading to canceled recruitment events, and straining relationships between academic medical centers and the potential participating PBRN member offices (see Tables 3 and 4).

Table 3.

Changing Inclusion/Exclusion Criteria for Subjects Participating in 2322 Device Trial.

Date	Inclusion/exclusion 2322 subject criteria
March 2021	Asymptomatic or Symptomatic: • 18 years of age or older • able to speak and understand English • willing to answer surveys about health • willing and able to provide 2 types of samples (nasal swab and saliva)
April 2021	Asymptomatic: • no symptoms attributable to COVID-19 within the 48-h period before screening • no known exposure to COVID-19 within the 14-day period before screening Symptomatic: • symptoms attributable to COVID-19 present within the 48-h period before screening • symptom onset within the 7-day period before screening or must report exposure to SARS-CoV-2 within the 14-day period before screening
End of April 2021	Symptomatic only: • symptoms consistent with COVID-19 in the 48 h prior to enrollment and/or exposure to someone diagnosed with COVID-19 in the proceeding 14 days
June 2021	Symptomatic only: • two or more symptoms attributable to COVID-19 with at least 1 symptom present at enrollment and onset of symptoms within 7 days before screening • received no doses of COVID-19 vaccine
July 2021	Symptomatic only: • at least 2 symptoms consistent with COVID-19 in the past 48-h that developed within the prior 7 days, and at least 1 symptom persisting at the time of enrollment

Table 4.

Research Factors Pertaining to Device Trial: Challenges, Consequences, and Solutions.

Research factor	Challenges	Consequences	Solutions
Device Investigator Study Plan (ISP)
Frequent ISP changes	• During the trial, the inclusion criteria for subjects with regard to symptom requirements, time since onset of symptoms and/or vaccination status were changed frequently and with little notice.	• Fewer subjects than anticipated were eligible at recruitment events, leading to low enrollment or event cancelation. • Some subsites were no longer viable options for recruitment due to modified inclusion criteria. • Delayed or canceled recruitment events strained study team relationships with clinic sites.	• Advocate for early involvement and integration of the PBRNs in project planning when feasible • Establish early communication with sponsor and affiliate sites about any potential for protocol changes • Establish minimum notice period with sponsor for protocol changes • Consider contract with sponsor for compensation for any events rendered unfeasible by protocol changes to mitigate loss of site productivity • Consider running multiple protocols when feasible so that an alternate experiment is available if one is paused or not feasible at a particular site
Pauses	• The trial endured multiple unanticipated pauses for protocol and/or ISP changes or DSMB review.	• Delayed or canceled recruitment events strained study team relationships with clinic sites.	• Make sure that clinics know that the DSMB could pause studies at any point, and recruitment events could be canceled at the last minute. • May be difficult to avoid as dependent on DSMB; consider planning events with partners that can flex to other purposes if study is paused • Consider contract with sponsor for compensation for any events rendered unfeasible by pauses to mitigate loss of site productivity • Consider running multiple protocols at a time when feasible so that an alternate experiment is available if one is paused or not feasible at a particular site
Device (2322)
Device size	• Some planned recruitment sites could not accommodate the large size of the device.	• Delayed or canceled recruitment events strained study team relationships with clinic sites.	• Establish logistical feasibility of transporting and operating materials in advance of committing to partnerships or event types • Consider having PBRN staff observe events at sponsor/home institution prior to setting up community sites in order to prompt anticipation of logistical barriers • Have participating entities understand PBRN research
Control solution requirements	• Control solutions (positive/negative) needed storage in −80°C freezers. Controls needed to be run frequently on newly installed machines, for a new lot or shipment of cartridges, when a new operator was managing the machine, or when there was a suspected problem with the machine.	• Controls could not be stored at recruitment sites; research staff needed to drive large distances to retrieve control solutions. • Controls needed to be transported on dry ice which required coordination and training beyond what was initially expected. • Significant time needed to run controls	• Establish logistical feasibility of transporting and storing materials in advance of committing to partnerships or event types • Storage and transportation considerations must be written into standard protocols for remote or community sites not affiliated or proximal to larger laboratories • Consider having PBRN staff observe events at sponsor/home institution prior to setting up community sites in order to prompt anticipation of logistical barriers
Sample storage requirements	• Dry ice was a necessity both for storing control solution in the field, and for storing and mailing second saliva samples to the 2322 device company.	• This was an added complication for some PBRN sites. Dry ice was not easily accessible at rural sites, and a limited number of FedEx offices allow shipment of dangerous goods on dry ice. Accessibility of dry ice and an appropriate FedEx office had to be considered when determining recruitment start and end times.	• Establish logistical feasibility of transporting and operating materials in advance of committing to partnerships or event types • Consider extensive “tabletop” rehearsal of protocol for each site in advance of committing to event or location to anticipate unique logistical obstacles. • Consider that “work arounds” (such as using dry ice to transport specimens) may create additional complications to protocol adherence that require advanced planning
Control cartridge failures	• There were multiple control cartridge failures while attempting to run the controls upon receipt of new cartridges, training of new operators of the device, or errors during testing.	• Control cartridge failures took devices out of use at events already on-going in the field, which led to fewer subjects being recruited at planned event and delayed testing for those who desired testing.	• May be hard to anticipate; consider early troubleshooting with sponsor and requesting adequate backup equipment
Device failures	• There were multiple device failures while attempting to operate the device during events.	• Other device failures took devices out of use at events already on-going in the field, which led to fewer subjects being recruited at planned events.	• May be hard to anticipate; consider early troubleshooting with sponsor and requesting adequate backup equipment • Consider having PBRN staff observe events at sponsor/home institution prior to setting up community sites in order to prompt anticipation of logistical barriers
Data collection platform (Eureka)
Study identifier confusion	• The Eureka digital research platform was built for use by many studies with some study sites having similar URLs (e.g., testus.eurekaplatform.org/ and testusbank.eurekaplatform.org/). • Some subjects self-enrolled in the wrong study causing difficulty with assigned Eureka study identification numbers.	• Subjects entered data in the wrong platform, leading to confusion with enrollment, frequent corrections, and lengthening of research visits as subjects had to redo surveys. • Incorrect study information had to be canceled.	• Consider extensive “tabletop” rehearsal of protocol for each site in advance of committing to event or location • Only engage in supervised registration of subjects • Consider building in multiple confirmations for subjects to ensure they are in correct study • Insist that subjects access initial research platform from QR code or personalized link
WiFi connectivity	• Successfully using Eureka required wireless internet access at all times. WiFi hotspots were purchased and connected to study telephone plans so subjects could access the Eureka app.	• Hotspot reception problems occurred depending on the distance from the hotspot. When a subject was enrolling in their car via a tablet or phone, it was difficult to position study staff over the car window in full personal protective equipment while maintaining a distance from someone with COVID-19 symptoms.	• Consider extensive “tabletop” rehearsal of protocol for each site in advance of committing to event or location to ensure logistical feasibility including informational technology considerations • Consider having PBRN staff observe events at sponsor/home institution prior to setting up community sites in order to prompt anticipation of logistical barriers
Accuracy Eureka	• As the inclusion criteria for the 2322 trial changed frequently, the eligibility criteria in Eureka did not match changes in real time.	• Coordinators had to fill out an additional form correcting any inaccurate data entered into Eureka.	• Early communication with sponsor and affiliate sites about potential for protocol changes • Establish and train on paper/manual enrollment methods early
Rigid data input parameters	• Patients completing questionnaires could not change their answers after submission. • Coordinators could not easily view or change coordinator data entry after submission.	• There was no straightforward mechanism for correcting information entered in the digital platform. • All changes for data entry on participant questionnaires had to be emailed to the Eureka support team to be corrected. • If coordinators were concerned an error had been made in coordinator data entry, they had to go into a separate correction form that included every data point, find the correct data point, and make a correction.	• Consider embedding more streamlined correction mechanism into data collection forms* • Consider supervising subjects while conducting questionnaires*
Laboratory services (Quest Diagnostics)
Delays in testing	• The Quest laboratory was overwhelmed during the study period resulting in analysis delays for some PCR tests.	• PCR results were delayed in being returned to subjects.	• Unique situation precipitated by the pandemic, may be difficult to anticipate* • Include in informed consent results may be delayed
Assay downtimes	• Occasionally and without warning, Quest Roche assay would go down and samples from the various sites were run on the laboratory developed test (LDT) assay.	• Study staff had to work unplanned days/weekends to notify subjects as per protocol requirements. Online form correction for the assay type had to be corrected for every impacted subject.	• Unique situation precipitated by the pandemic, may be difficult to anticipate* • Consider standardizing backup assay when applicable
Study logistics
Shipping time	• Delays in device shipping to site. • Delays in sample return to lab.	• Postponed and canceled subject recruitment events. • Some samples were unacceptable for testing due to prolonged time from collection to lab receipt resulting in loss of subjects and inability to report results to subjects.	• Build room for delays into all shipping timelines and storage protocols • Establish shipping network with proximity to study sites prior to protocol initiation and ensure that shipping sites can manage special needs for specimens (ie, shipping on dry ice)
Incomplete supplies	• Some shipped test kits were missing either cups or cartridges, rendering them unusable. • Supplies were shipped in small batches, resulting in sites running out of necessary equipment.	• Delayed or canceled recruitment events, fewer subjects could be recruited.	• Require all shipped materials to arrive well in advance of planned events • Standardize packing and unpacking protocols for all shipped materials • Consider shipping crucial supplies in bulk • Consider allowing sites to source supplies locally when applicable
Shipping centers	• Only some shipping center were authorized for use for shipment of study samples and only a subset of those centers allowed shipments on dry ice, causing study teams extensive travel times.	• Accessibility of dry ice and an appropriate FedEx office had to be considered when selecting recruitment sites and event start and end times.	• Establish shipping network with proximity to study sites prior to protocol initiation and ensure that shipping sites can manage and special needs for specimens (ie, shipping on dry ice)
Communication with UMass	• Meetings were all on the phone or zoom and not in person. • There were numerous meetings scheduled with siloed teams and not all PBRN personnel could attend all meetings.	• It took some time for the sponsor to identify problems with the device protocol, which resulted in analyzer failures because sponsor staff were not able to be physically present at sites. • There was a delay in crucial information regarding protocol changes or regulatory matters if a change was discussed at a meeting but not announced in writing.	• Consider onsite meetings with sponsors, especially when training on devices and protocols • Consolidate and standardize meetings • Create clear real-time written record of all meetings, especially any regarding protocol updates • Consider having designated site study staff member to track and attend all essential meetings • Request that sponsor had dedicated PBRN project manager to serve as central resources for all PRBN sites
Staffing	• Some community clinics lost staff who were planning to aid in recruitment during the pandemic.	• Some partner clinics had to cancel participation due to the complex workload and training required for this study.	• Unique situation precipitated by the pandemic, may be difficult to anticipate* • Establish minimum staffing needs with community sites early • Consider flexing study staff out to community sites to support onsite recruitment

May not be PRBN specific obstacles.

Additional challenges were encountered in operation of the analyzer, the Eureka digital data collection platform, Quest laboratory services, and study logistics. Many of these challenges were related to the nature of PRBN-based device research and the need to perform this study remotely from central academic research centers, often in rural, community settings. These challenges and their consequences are summarized in Table 4.

Discussion

Planning for this PBRN 2322 device trial involved diverse research teams at each PBRN and academic center; physicians, nurses, laboratory staff, and medical assistants from 10 offices; and UMass community testing sites and clinical research centers across 4 states. Lawyers and sponsored program personnel facilitated the numerous contracts for the project and IRB applications and modifications were completed at each PBRN academic center which then ceded to the UMass IRB. Planning arrangements for recruitment sites at some PBRNs by necessity started before awards were made from UMass. Not all offices/recruitment sites were used because initial plans for implementation changed rapidly based on the 2322 device specifications, specimen transport requirements, and inclusion/exclusion subject criteria. The commitment and efforts of the participating researchers and PBRN members is admirable.

None of the PBRN offices had a −80° C freezer. KU found a resolution for storage of the control solution at another facility nearby. The other PBRNs’ arrangements involved finding dry ice supplies, transporting the 2322 devices to the office, setting up each day, and transporting the control solution on dry ice. The original plan to leave the testing devices at each office was not feasible because of the necessity for a −80°C freezer. Even though PBRN office personnel were trained in humans subjects and good clinical practice education, the steps involved in enrolling subjects and collecting samples required them to be very adept at using Eureka, Quest, and other requirements, which was very onerous for participating primary care offices, which were also focusing on primary care delivery during a pandemic. If devices are intended to be used in rural, community-based practices, provision of adequate resources to community settings should be taken into consideration during device development and the necessary resources should not be assumed.

PBRN leaders expend significant time and effort to build relationships with offices who may be interested in engaging in research. Many clinics have a positive experience participating in trials with PBRNs and choose to participate in future studies. Such relationships are crucial to the continued success of PBRN work. In this trial, several PBRNs leaders expressed concern that they had damaged relationships with their partner offices due to changes in recruitment inclusion criteria necessitating repeated last-minute recruitment cancelations.

A recommendation that evolved from the interviews was that for future studies, PBRNs talk frequently during proposal and onboarding time-periods to facilitate collaboration with each other, as the PBRNs encountered challenges and needs that differed from those of larger academic centers. UMass held weekly meetings with each PBRN and shared their best practices, example workflows, and recruitment strategies. However, these meetings did not provide an opportunity to hear details about the successes or challenges at other PBRNs. Additionally, when possible, earlier involvement of all participating PRBNs in project planning with the sponsor from the inception of the study may permit them to provide valuable input in protocol preparation, communication strategies, and regulatory and financial structures to prevent downstream complications.

The PBRNs didn’t realize they would be competing against each other to recruit subjects in 1 single device trial to attempt to recoup costs incurred for the long start up and training period and relatively short recruitment period. Initially it was understood the each PBRN would have different devices and could easily give them to PBRN collaborators to use in a healthcare setting, but the 2322 device did not facilitate this plan. The implementation phase of including PBRNs in RADx occurred during the COVID-19 second East Coast surge which limited the need for testing many new devices. PBRN participants in the 2322 trial are to be commended for being out in the field collecting specimens from persons with COVID-19 symptoms. They put their own health and wellbeing in jeopardy to assist with this trial.

This report highlights some of the challenges in implementation of a device trial. A relational approach with the manufacturer through UMass to the PBRNs was not used for the trial, but instead the PBRNs created opportunities for offices and patients in community-based practices to test the device. PBRN researchers were not involved in protocol conception (written by UMass), but in a collaborative manner facilitated problem-solving techniques based on issues they experienced using the device, ultimately improving the accuracy of the device PCR results.

In Table 4 challenges and consequences were noted for the protocol, 2322, data collection platform, laboratory services, and study logistics. Potential workarounds or solutions are suggested. The overall protocol for RADx Tech COVID-19 Test Us Study’s purpose was to conduct clinical studies on novel testing devices in diverse population and settings in the United States using a platform trial design allowing for many different devices to be subsumed under 1 protocol. The Device Investigator Study Plan (ISP) guided use of individual devices. It is difficult to know if PBRN researchers would opt out of a trial depending on the type of device given to test. The 2322 device provided to PBRNs had its own unique challenges and the resourceful PBRN researchers worked through those problems.

Conclusion

Conducting the trial in community-based practices caused unique challenges for PBRN researchers and office personnel, resulting in a learning curve for everyone using the new device. The device trial was shorter than most PBRN clinical trials as subject accrual was dependent on obtaining valid results to garner FDA approval. For future investigations, PBRN leaders are encouraged to query study sponsors to find out whether recruitment criteria may change and/or if the study is likely to be halted suddenly to appropriately prepare their member clinics. A detailed list of equipment needed would be appropriate if primary care offices are to participate.

Footnotes

Appendix A: 2322 Trial

The 2322 device was a saliva test for SARS-CoV-2 using a polymerase chain reaction (PCR) at the POC where test results can be provided immediately. The PCR test is the gold standard test for diagnosing COVID-19. Rapid tests used in POC settings for detecting COVID-19 are usually antigen tests, which use antibodies to portions of the COVID-19 virus, with less sensitivity than PCR tests. The 2322 device was designed to be an optimally sensitive POC PCR device.

Power calculations and number of subjects needed for 2322 were not shared with the PBRNs. Study enrollment for the 2322 device began at UMass with 5 analyzers on 3/10/2021 and was paused on 3/22/2021 by the NIH Data and Safety Monitoring Board (DSMB) for 2 main concerns: (1) discrepancy in the number of mid-turbinate nasal swab PCR tests Roche assay that were positive compared with subjects who had negative results on the 2322 device, and (2) the number of invalid quality control samples on the 2322 device. The trial restarted on 4/14/2021 at UMass, and devices were shipped to PBRNs on 4/21/2021. The KU received 1 analyzer, OHSU received 4 analyzers, and UI received 3 analyzers. Upon receipt, positive/negative controls were run on each analyzer. On 5/17/2021 the DSMB paused the study again because the data collected and presented did not meet the goal for positive and negative percent sample agreement. On 6/4/2021 the study was relaunched, and on 9/10/2021 enrollment ended as the company was ready to apply for an EUA from the FDA. Throughout the trial, inclusion/exclusion criteria changed numerous times because of DSMB monitoring and company improvement changes (See Table 3).

In April 2022, the 2322 device was granted EUA by the FDA as a test for the qualitative detection of nucleic acid from SARS-CoV-2 in saliva specimens from individuals suspected of COVID-19 by a healthcare provider. It was the first EUA for a PCR test of COVID-19 for SARS-CoV-2 to be used in the POC setting. The value in testing saliva specimens for COVID-19 is that invasive nasopharyngeal collection is not needed, removing an uncomfortable procedure for patients, and allowing frontline staff to stay at a distance.¹⁷

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Prime Sponsor: National Institutes of Health 3U54HL143541-02S2 to University of Massachusetts, Worcester with subaward no. OSP/WPM33765-IOWA to the University of Iowa, subaward PI Barcey T. Levy. National Institutes of Health National Center For Advancing Translational Sciences UL1TR002537, and the University of Iowa Department of Family Medicine. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. “The project described was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant KL2TR001454. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.”

ORCID iDs

Jeanette M. Daly

Megan E. Schmidt

Kim Parang

References

Agency for Healthcare Research and Quality. Practice-based research networks (PBRNs). 2022. Accessed 28 March, 2022. https://www.ahrq.gov/cpi/about/otherwebsites/PBRN/pbrn.html

Tierney

Oppenheimer

Hudson

, et al. A national survey of primary care practice-based research networks. Ann Fam Med. 2007;5(3):242-250. doi:10.1370/afm.699

Westfall

Mold

Fagnan

Practice-based research—“Blue Highways” on the NIH roadmap. JAMA. 2007;297(4): 403-406. doi:10.1001/jama.297.4.403

Ashman

Santo

Okeyode

Characteristics of office-based physician visits, 2018. NCHS Data Brief. 2021;408:1-8.

Centers for Disease Control and Prevention. Physician office visits. 2022. Accessed June 21, 2022. https://www.cdc.gov/nchs/fastats/physician-visits.htm

Westfall

Roper

Gaglioti

Nease

Jr.

Practice-based research networks: strategic opportunities to advance implementation research for health equity. Ethn Dis. 2019; 29(Suppl 1):113-118. doi:10.18865/ed.29.S1.113

Agency for Healthcare Research and Quality. PBRN registry. 2022. Accessed March 28, 2022. https://www.ahrq.gov/ncepcr/communities/pbrn-registry/index.html

Hafer

Buchholz

Dunlap

, et al. A multi-institutional partnership catalyzing the commercialization of medical devices and biotechnology products. J Clin Transl Sci. 2021; 5(1):e119. doi:10.1017/cts.2021.779

Tromberg

Schwetz

Pérez-Stable

, et al. Rapid scaling up of Covid-19 diagnostic testing in the United States—The NIH RADx Initiative. New Engl J Med. 2020; 383(11):1071-1077. doi:10.1056/NEJMsr2022263

10.

Gibson

Fahey

Hafer

, et al. The RADx Tech Clinical Studies Core: a model for academic based clinical studies. IEEE Open J Eng Med Biol. 2021;2:152-157. doi:10.1109/ojemb.2021.3070830

11.

Robinson

Gaydos

Van Der Pol

, et al. The Clinical Review Committee: impact of the development of in vitro diagnostic tests for SARS-CoV-2 within RADx Tech. IEEE Open J Eng Med Biol. 2021;2:138-141. doi:10.1109/ojemb.2021.3070818

12.

Saviola

The FDA’s role in medical device clinical studies of human subjects. J Neural Eng. 2005;2(1):S1-S4. doi:10.1088/1741-2560/2/1/001

13.

Fowlkes

Giorgi

Erdman

, et al. Viruses associated with acute respiratory infections and influenza-like illness among outpatients from the Influenza Incidence Surveillance Project, 2010–2011. J Infect Dis. 2014;209(11):1715-1725. doi:10.1093/infdis/jit806

14.

Nagykaldi

Calmbach

Dealleaume

Temte

Mold

Ryan

JG.

Facilitating patient self-management through telephony and web technologies in seasonal influenza. Inform Prim Care. 2010;18(1):9-16. doi:10.14236/jhi.v18i1.749

15.

Higginson

Schweighart

Radloff

. Challenges and opportunities in medical device clinical trials in Europe. Regulatory Focus. Accessed December 8, 2022. https://www.raps.org/RAPS/media/news-images/Feature%20PDF%20Files/21-9_Higginson-et-al.pdf

16.

Peyser

Marcus

Beatty

Olgin

Pletcher

MJ.

Digital platforms for clinical trials: the Eureka experience. Contemp Clin Trials. 2022;115:106710. doi:10.1016/j.cct.2022.106710

17.

Caixeta

Oliveira

Cardoso-Sousa

, et al. One-year update on salivary diagnostic of COVID-19. Front Public Health. 2021;9:589564. doi:10.3389/fpubh.2021.589564