Leveraging the functionality of Research Electronic Data Capture (REDCap) to enhance data collection and quality in the Opioid Analgesic Reduction Study

Abstract

Background

The Opioid Analgesic Reduction Study is a double-blind, prospective, clinical trial investigating analgesic effectiveness in the management of acute post-surgical pain after impacted third molar extraction across five clinical sites. Specifically, Opioid Analgesic Reduction Study examines a commonly prescribed opioid combination (hydrocodone/acetaminophen) against a non-opioid combination (ibuprofen/acetaminophen). The Opioid Analgesic Reduction Study employs a novel, electronic infrastructure, leveraging the functionality of its data management system, Research Electronic Data Capture, to not only serve as its data reservoir but also provide the framework for its quality management program.

Methods

Within the Opioid Analgesic Reduction Study, Research Electronic Data Capture is expanded into a multi-function management tool, serving as the hub for its clinical data management, project management and credentialing, materials management, and quality management. Research Electronic Data Capture effectively captures data, displays/tracks study progress, triggers follow-up, and supports quality management processes.

Results

At 72% study completion, over 12,000 subject data forms have been executed in Research Electronic Data Capture with minimal missing (0.15%) or incomplete or erroneous forms (0.06%). Five hundred, twenty-three queries were initiated to request clarifications and/or address missing data and data discrepancies.

Conclusion

Research Electronic Data Capture is an effective digital health technology that can be maximized to contribute to the success of a clinical trial. The Research Electronic Data Capture infrastructure and enhanced functionality used in Opioid Analgesic Reduction Study provides the framework and the logic that ensures complete, accurate, data while guiding an effective, efficient workflow that can be followed by team members across sites. This enhanced data reliability and comprehensive quality management processes allow for better preparedness and readiness for clinical monitoring and regulatory reporting.

Keywords

Electronic data capture quality improvement Research Electronic Data Capture clinical data management clinical trial

Background

Advances in technology have significantly transformed the research landscape. Contemporary clinical research integrates technology into current protocols and research infrastructures to improve quality, efficiency, and productivity.^1–3 From laboratory processes, such as automated specimen processing and analysis, to patient-facing digital health technologies and eConsents, technology is being incorporated into clinical research to optimize operations and improve the clinical trial enterprise.^4–7 Electronic data capture (EDC) has gained traction replacing conventional data collection methods in clinical trials. With evidence suggesting that paper-based data collection is more prone to errors than direct data entry, EDC systems have the potential to improve data quality, increase efficiency, and reduce costs.^8,9 In a systematic review of 53 unique pain studies conducted, Jibb et al.¹⁰ demonstrated that EDC is not only comparable to conventional methods, but participants indicated electronic modalities were easy to use and more often preferred. The Opioid Analgesic Reduction Study (OARS) incorporated EDC into its study design as a means enhance data quality. OARS leveraged the functionality of a secure EDC software called Research Electronic Data Capture (REDCap) to not only function as its data reservoir but also provide the framework for its comprehensive quality management (QM) program across sites to ensure complete, accurate, data while guiding an effective, efficient workflow that can be followed by all team members.¹¹

Early in study development, the OARS team agreed on EDC as an alternative to conducting its multi-site clinical trial using paper. In considering its research infrastructure, the study team recognized the need to balance the integration of multiple digital research technologies with the effective collection of high-quality data.¹² The study teamed turned to REDCap (Version 11.1.0; published by Vanderbilt University) to fulfill these requirements and effectively execute study protocols. Created in 2004 and sustained by the REDCap Consortium, REDCap is a secure web application that is “specifically geared to support online and offline data capture for research studies and operations,”¹¹ with license and support offered free to non-profit organization members of the REDCap Consortium. Used by researchers worldwide, REDCap has proven to be an effective, accessible option for data collection.^13–18 Kianersi et al.¹⁹ is one of the few studies which report on the methodological uses of REDCap within a rigorous clinical trial. In their COVID-19 randomized controlled trial, REDCap features were incorporated to efficiently capture data and electronically interface with participants through automation.

REDCap was selected for its versatility, reliability, functionality, and available institutional expertise. Our university currently maintains REDCap infrastructure, offering experience and technical support. As described in Figure 1, REDCap was transformed from a traditional data capture/storage vessel to an expanded function study management tool that directly supports OARS continuous quality improvement process.

OARS study design and methods

OARS is a double-blind, prospective clinical trial investigating analgesic effectiveness to manage acute post-surgical pain after impacted third molar extraction.²⁰ The main study objective is to determine whether a non-opioid combination of over-the-counter analgesics (ibuprofen/acetaminophen) is non-inferior to the most commonly prescribed opioid (hydrocodone/acetaminophen) combination product, with respect to patient-perceived pain levels and patient satisfaction observed for 7 days following extraction. Stratified by gender, 800 participants will be enrolled across five clinical sites. Rutgers University serves as the single Institutional Review Board (IRB) of record (Study ID: Pro2020002299), and in June 2020, OARS was registered in ClinicalTrials.gov (#NCT04452344). Enrollment began in January 2021.

Using the consent form, research staff review study protocols, risks, and benefits with interested individuals. Written consent is obtained and documented for all participants by completing and signing the eConsent form in REDCap, which is FDA 21 CFR Part 11 compliant. A printed consent copy is given to each participant upon execution. Next, participants are screened for eligibility. Adult males and non-pregnant females (age > 18 years) who require extraction of at least one partial/full bony impacted mandibular third molar, in good health, able to take ibuprofen, acetaminophen, and hydrocodone and own a smartphone are eligible. Participants are evaluated for exclusion criteria related to comorbidities, concomitant medications, and social history that would preclude the use of hydrocodone, acetaminophen, and ibuprofen.

Eligible participants are actively engaged in study activities from the surgical visit through the post-operative period. Prior to surgery, participants complete a pre-visit questionnaire that records their current dental disposition, including baseline pain levels, sleep, activity, and pain tolerance. Through random assignment, participants are provided a subject kit which contains over-encapsulated, blinded study analgesic and an activity monitor, for use during the post-operative period. Study medication comes in two electronic bottles, each bottle containing 20 capsules. The OPIOID group receives 5 mg hydrocodone/300 mg acetaminophen and a placebo, while the non-opioid group receives 400 mg ibuprofen and 500 mg acetaminophen. Participants are instructed to take one dose (one capsule from each bottle) immediately after surgery followed by q4–6 h as needed for pain. Twice daily during post-operative period, participants record pain, activity, sleep, and medication usage using an eDiary while continuously wearing an activity monitor. At the post-operative visit, participants return the electronic bottles with unused study medications and the activity monitor undergo a clinical exam and complete a questionnaire.

Methods

Figure 2 highlights the nine REDCap projects that support OARS. In addition to patient data, operational, and logistical aspects of the study are built into REDCap, including credentialing/delegation of duties, inventory control, and QM.

User rights for data access and security

Individual user rights within REDCap are assigned following the principal of least privilege, granting access to the minimum level that is required. Each site is assigned a Data Access Group, which limits access of site-specific identifiable data to the site director and research coordinator(s). In addition, the biostatisticians, the chief clinical officer, and the data quality analysts have access to identifiable data across sites. The REDCap logging function, which uses secure, computer-generated, time-stamped audit trails, independently records the operator, the date and time of operator entries, and actions that create, modify, or delete electronic records.

The REDCap application is managed by the Rutgers School of Dental Medicine Office of Information Technology and directly integrates with Rutgers University’s Identity Management system, responsible for enforcing password complexity and providing a centralized mechanism for revocation of user access rights. Data are recorded and accessed using University’s encrypted, virus and password protected computers behind firewalls.

Clinical data management

Data repository

Participant’s data are captured in the OARS EDC REDCap project utilizing 19 IRB-approved REDCap instruments, 10 of which are repeatable. When adding a new participant record, REDCap automatically assigns a Subject Study ID. Study team members record clinical data directly into the REDCap interface on study devices while logged into their REDCap accounts. Electronic consents are completed as surveys to ensure a PDF automatically appears in the file repository.

Data integration

For OARS, REDCap integrates with several other data platforms. Twilio (https://www.twilio.com/en-us), a third party web-service, allows survey link invitations to be sent via URL in an SMS text message to the participant’s mobile phone. Twice daily, participants receive a text that contains the link for their eDiary entries. In addition, data collected via the SMRxT electronic medication bottles are transferred via cellular technology from the SMRxT bottle (https://www.smrxt.com/) to the SMRxT companion software (NOMI) (https://nomiadherence.com). A cellular communication application programming interface allows dosing data to be linked into REDCap, recording each instance (date, time, and quantity) a capsule(s) is removed/added from each bottle.²¹ With each dose taken, the NOMI platform automatically generates a new REDCap record for that participant, identifying type of activity (pill removed or added) and time the action was taken.

User notifications

Various REDCap alerts contribute to data completeness. For example, when saving a form, an alert is generated when a required data element is missing, redirecting the end-user to the missing data field. REDCap reportable event forms also utilize alerts. For example, when a protocol deviation form is initiated by study staff, an alert is automatically generated and sent to appropriate site personnel, the protocol chief, and the PI. As the review of protocol deviations required multi-level approval, the alert remains active until all responsible parties have reviewed.

Project management

Credentialing and delegation of duties

The OARS study team is comprised of approximately 60 individuals with varied skills and expertise, including clinicians, biostatisticians, REDCap administrators, information technology specialists, and pharmacy technicians. As the study engages personnel across institutions, OARS necessitated a comprehensive electronic system to manage personnel records. REDCap functions as the main integrated personnel system for the project related to on-boarding, continuing review, and decommissioning. System access and privileges are clearly delineated and signature functions are utilized to capture attestations, as well as, multi-level approvals (Table 1). Each study personnel is given a unique record ID in REDCap which can be accessed using the Record Status Dashboard.

Table 1.

Personnel management.

Form	Data elements
On-Boarding Checklist	General info: role, site, start date Contact info Credentials (including applicable license numbers) Required trainings and completion dates DocuSign^® ability and verification of credentials
Delegation of Duties	Administrative duties Subject-facing duties Reportable events • Adverse events • Serious adverse events • Protocol deviations/violation • Unanticipated problem Quality management Training attestation signature PI signature
On-Boarding	System access System privileges
Off-Boarding	Removal of IT privileges Decommissioning notes or comments

PI: principal investigator; IT: information technology.

Materials management

Inventory tracking

There are multiple projects within REDCap that support inventory management. Several inventory projects were required to track separate entities (i.e. investigational product bottles, actigraphs,²² SMRxT bottles) with discrete data elements and different life cycles. Investigational product was tracked from receipt from the manufacturer through destruction. The tracking is paramount not only for trial success but also compliance with institutional, state, and federal regulations. Figure 3 illustrates the comprehensive inventory management REDCap projects.

Quality Management

Data QM features utilized in REDCap function simultaneously to support the OARS comprehensive QM program.

Data validation

REDCap instruments ensure data validity through univariate, multivariate, and cross-module alerts. Whenever possible, univariate validation rules are applied to data fields when items need to be entered. Structured data responses require the entry of valid responses only (i.e. check a yes or no box, select from a list of possible answers such as a Likert-type scale). Furthermore, use of programming validations in data fields limit entry format and prevent entry of out-of-range fields. The REDCap system utilizes drop-down menus to minimize errors. Data elements have been designed to provide the description and the data element coding. When a single answer is required from a multiple choice set, the data element is programmed to only allow one answer. Greater than 95% of data fields are programmed as required.

In addition, multiple fields have been programmed to include branching logic, field-based stopping rules and survey logic. Branching logic is used when respondents must provide answer to follow-up questions, eliminating the need to skip non-applicable questions. Calculation aids have been integrated into surveys and questionnaires to facilitate decision tree logic. For example, in order to assist the research coordinators in determining whether visits are occurring in the allowable windows, the EDC has been programmed to calculate the number of days between enrollment and surgery and between surgery and post-op.

Prior to production-level study initiation and following any REDCap version upgrades, comprehensive user-acceptance testing was carried out on all study instruments to ensure the collection of high-quality data. A development version of the database, which perfectly mirrored the current production version, was configured to enable end-to-end testing of each study instrument and instrument flow process against each of the user roles. Furthermore, all study instruments that required post-production changes underwent rigorous testing in the mirrored development database prior to any implementation.

Dashboards

Functionally, the Record Status Dashboard in REDCap allows for real-time, high-level data monitoring. This feature provides an overview of the current state of the data and trial progress at a glance. In programming REDCap, the study team incorporated the dashboard into its QM program by combining dashboard use with another prominent REDCap feature, Form Status. When saving a form, REDCap automatically offers three status choices that correspond to three colors of the radial buttons on the dashboard: Incomplete (Red), Unverified (Yellow), and Complete (Green). During data collection, OARS has opted to require all forms to be saved as “Unverified.” As part of the QM program, a data quality analyst reviews the data for accuracy and completeness. Once reviewed, the data quality analyst changes the form status to either “Complete,” indicating the data is valid or “Incomplete” to signal the data are either missing, incomplete, or requires correction.

QM forms

QM checklists were incorporated directly into REDCap projects to support quality data review. Specifically, for the OARS EDC, each participant record contains five quality checklists utilized by data quality analysts at designated intervals in the QM process to ensure the accuracy and completeness of data, as well as, monitor whether key project activities are performed as outlined in the protocol and manual of procedures. Similarly, inventory projects employ QM checklists to support verification that data elements/forms have been captured appropriately and that regulatory processes have been completed as appropriate. Using color-coded form status designations in the REDCap dashboards, end-users can note the quality review status and whether a form requires attention.

Queries

REDCap queries are the main communication tool utilized to rectify both participant and inventory data. Functioning as an internal email system, queries allow targeted emails to request review and adjudication of known data discrepancies and/or provide clarifications. To support the query process, all REDCap forms are designed to require the name or initials of the individual performing the data entry. This provides an efficient mechanism to identify the form’s author, eliminating the need to review audit logs. In addition, to ensure closure of the feedback loop, queries can only be closed by the initial author.

Reportable events and regulatory reporting

All reportable events that occurred during the conduct of the trial are to be reported using the REDCap Adverse Event, Serious Adverse Event, Unanticipated Problem, and/or Protocol Deviation form. Each form follows a hierarchical approval process to ensure that situations are reviewed and action plans developed as appropriate. At the principal investigator (PI)/Clinical Chief level, all reportable events are evaluated for regulatory reporting. If deemed appropriate, reporting details are documented directly on the REDCap form. Site-specific alerts signal an event and the need for form completion. Alerts continue until the final form approval is obtained. Form completion status visible on the dashboard serves to indicate the progress of this review.

Data export

Throughout this trial, separate reports track participant flow, side effects, adverse events, efficacy assessment, etc. In order to export the data from the REDCap server, the option to export into “SAS Statistical Software” is used for each individual report. This option generates two files: a *.csv file that contains the raw data and an SAS syntax file that is used to import the *.csv file and apply proper formatting for each specific variable. After using the SAS syntax file to import the data into SAS, either new SAS code is drafted or changes are applied to existing SAS code to link the exported data by participant ID or study treatment kit ID (as needed), and then perform the necessary calculations for tracking enrollment, retention (participant flow), side effects, adverse events, etc., as well as the statistical data analysis for efficacy assessment.

Results

For user-acceptance testing for the REDCap OARS EDC project, 402 scripts were identified as relevant toward the build, 380 scripts were executed and passed on the first attempt, establishing a 94.5% first-time pass rate and 22 scripts failed upon first attempt. For each of these scripts, modifications were made to the system configuration, and they subsequently passed. The user-acceptance testing was deemed successful with a 100% pass rate. In addition, at 72% study enrollment completion, there are 76 production revisions logged in REDCap, which range in scope from field label corrections to the addition or revision of study forms as required by the study sponsors. Of note, only four of these revisions required REDCap administrator approval.

OARS EDC (participant data forms excluding the eDiaries and the reportable event forms)

At 72% study completion, 1536 participants have been consented, 1299 participants have been enrolled (randomized and received investigational product), and 1272 participants have completed assigned study procedures in the OARS clinical trial.

Form completion

At 72% completion, 12,328 REDCap forms have been completed. Table 2 illustrates the number of missing forms and forms which are missing one or more data elements. Of note, the Post-Op Case Report and the Post-Op Subject Questionnaire were both completed during the post-op visit. There are 11 missing Post-OP Case Reports and 7 missing Post-Op Subject Questionnaires. Seven participants did not return for any portion of their post-op visit and after repeated failed attempts to bring them back, these participants were deemed lost to follow-up. In addition, four participants completed only a partial post-op visit and chose not to complete their post-operative exam due to time delays. There are no additional missing forms in any other form category.

Table 2.

REDCap forms which are missing and either incomplete or with errors.

REDCap form	Number of forms completed	Missing forms	Missing forms (%)	Number of incomplete/erroneous forms	Forms incomplete/erroneous (%)
Consent	1542^a	0	0.00	0	0.00
Eligibility	1535	0	0.00	0	0.00
Gender and Pregnancy Status	1414	0	0.00	1	0.07
Pre-Op Subject Questionnaire	1375	0	0.00	2	0.15
ID Link and SMS Test	1305	0	0.00	2	0.15
SMS Test and Training	1304	1	0.08	1	0.08
Surgical Case Report	1305	0	0.00	0	0.00
Post-Op Case Report Questionnaire	1272	11	0.86	0	0.00
Post-Op Subject Questionnaire	1276	7	0.55	2	0.16
Total	12,328	19	0.15	8	0.06

REDCap: Research Electronic Data Capture.

Six participants were consented twice.

Data resolution for the EDC

From the 12,328 REDCap forms, 523 queries were initiated to request clarifications and/or address missing data and data discrepancies. Twenty-five remain open, and 498 have been closed.

Reportable events

Table 3 illustrates the events documented by study personnel. Study personnel were encouraged to report all events that occurred throughout the conduct of the trial. All events were reviewed and necessity to report determined by the PI and Chief Clinical Officer. This workflow allowed for a comprehensive evaluation of each unexpected event or deviation from the protocol and maintained consistency in reporting.

Table 3.

Events documented throughout the study period.

Events	Total (n)	Mean (%)
Adverse Events (AE)
Participant Unanticipated Clinic or ED Visits (# events)	64	4.9
Serious Adverse Events (SAE)
Participant Hospital Admissions	3	0.2
Participant Fatalities	0	0.0
Protocol Deviations/Violations (PD/PV)
Total Protocol Deviations/Violations (completed forms)	149	11.5
Total Protocol Deviations/Violations (reportable)	44	3.4
Unanticipated Events (UE)
Total Unanticipated Events (completed forms)	44	3.4
Total Unanticipated Events (reportable)	0	0.0

ED: Emergency Department.

OARS inventory projects (inventory forms)

In addition to the OARS EDC project, three REDCap inventory projects monitor the movement of investigational product within the study packages using a number of data collection and QM forms:

OARS Subject Kit Ordering, Shipping, and Receiving (6 forms = 5 data + 1 QM).

OARS Rutgers CPCC Packing and Unpacking Participant Kit (4 forms = 3 data + 1 QM).

OARS Returning Participant Kits to the Rutgers Core (6 forms = 5 data + 1 QM).

At 72% study completion, there are 7486 completed inventory forms within these 3 projects.

Discussion

REDCap has been instrumental in the effectiveness of OARS. Not only has REDCap provided the tools for easy data entry at multiple sites, but it also has provided very powerful tools to aid in data quality, enrollment tracking, and centralized clinical trials management. Except for required DEA 222 forms, the study is able to be conducted in a paperless environment. The use of dashboards has been woven into the study framework and provides the study team with a real-time, comprehensive view of study progress on the micro-(participant) and macro-(site) level. At a glance for a given participant, site personnel can view forms completed during study visits and between study visits (eDiaries). Sites can also visualize their complete portfolio of participants and status related to target enrollment.

As illustrated in Table 2, dashboards, coupled with the use of data validation and queries, result in accurate, effective capture of participant data with minimal missing (0.15%) or incomplete or erroneous forms (0.06%). The various data validation tools and methods that have been implemented within the forms/surveys have not only made forms easier to complete and analyze (structured responses eliminate free text fields) but support workflows and decision-making. For example, while each research coordinator has been trained to determine eligibility, the OARS Eligibility Determination form supports the decision-making process. Branching logic allows eligibility data to be collected systematically. If a data element triggers ineligibility, the next eligibility question does not appear, but rather a red alert appears on the screen indicating the participant is ineligible. The use of REDCap queries also comes with several advantages: data security (the data remains within the secure REDCap system) and the ease of retrieval through REDCap Messenger (study messages are only within this system; no lengthy searches through institutional email inboxes).

The OARS team has successfully integrated the REDCap reportable events forms into its reporting workflow. The initiation of a reportable event form triggers an alert to site and study leadership to relay that an event has occurred that requires attention. Study staff document any “extra-ordinary” situation using a reportable event form (adverse event, serious adverse event, protocol deviation/violation, or unanticipated event). This workflow encourages reporting without being punitive and stimulates thorough review, discussion, development of action plans, and regulatory reporting as appropriate. Action plans are often site-specific but serve as training tools when discussed across sites with the full study team.

REDCap inventory projects not only effectively track investigational product from manufacture to destruction, but store data essential in the regulatory review process conducted by study staff as well as clinical monitors. A major advantage to having several inventory projects is the ability to maintain masking for those who must be blinded (the clinical site personnel).

Other clinical trials management software systems were considered; however, Rutgers University information technology (IT) had limited experience using them. Many systems were more expensive and would have required hiring personnel specifically to manage the clinical trials software. For example, the OnCore was a viable option; the team assessed its more rigid structure would have prevented our ability to easily manage study inventory. As a BOX file system had already been developed by the study team to manage the Essential Document “binder,” much of the OnCore capabilities were not required. The OARS study did not call for linking the clinical management system with the school’s electronic medical record, protocol timeline management easy to perform without the aid of software due to the short study protocol, and the billing module did not accommodate the nuances of billing for both medical and dental services.

While REDCap is an effective tool, there are some limitations. First, REDCap is not a relational database which resulted in the need to create multiple REDCap projects. As the REDCap projects are discrete, data review across the projects can be cumbersome and requires more steps for the statistical team. Second, REDCap’s reporting tools are extremely limited. Thus, data need to be exported into either Excel or a statistical program in order to calculate quality metrics. Third, while the development of surveys and other study instruments is intuitive and can be learned quickly, utilizing specialized functions, such as targeted, site-specific alerts, goes beyond normal programming and requires personnel with advanced experience and expertise in the program. The degree of REDCap knowledge rules and logic dictates the ease with which one can operate within the system. OARS benefits from having REDCap experts as part of the core study team for project development, monitoring, and support. Fourth, to distribute the workload, some REDCap form/surveys were developed by lesser experienced members and there were inconsistent conventions used in naming variables. A standard variable naming convention should be established prior to instrument development. Fifth, there are also limitations related to time-dependent data such as when data elements cannot be rectified. If a data element requires a time-dependent response, querying a participant after the time interval would yield an invalid response. Another instance is when a participant leaves prior to completing a form. In this situation, real-time data collection is missed. Finally, the effectiveness of the QM program does rely in part on the consistency among data quality analysts. Standard workflows must be developed related to REDCap form and dashboard use for reproducible results.

Conclusion

For OARS, leveraging REDCap system features has proven beneficial and contributed to its success. The REDCap infrastructure and enhanced functionality provides the framework and the logic that guides an effective, efficient workflow that can be followed by team members across sites and allows for better preparedness and readiness for clinical monitoring and regulatory reporting.

REDCap is an effective digital health technology that can be especially beneficial for multi-site clinical trials. As a data collection vessel, REDCap allows for direct data entry and can be programmed to incorporate data validation rules, ensuring the accuracy and completeness of the data collected. For consistency across sites, the digital interface allows for remote access, while maintaining data integrity and security through privileging and data access groups. As a data QM tool, REDCap functionality can support an effective QM process. Deliberate use of dashboards can provide a real-time snapshot of the current disposition of the data and signal the need for follow-up. Targeted queries can play a vital role in data resolution, especially across clinical sites. Identified differences between sites allows for root cause analysis and comprehensive process improvement of the clinical trial.

Footnotes

Acknowledgements

The authors wish to thank Julie Chapman-Greene for her steadfast commitment to this project.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by National Institute of Dental and Craniofacial of the National Institutes of Health awards under award nos UG3DE028860 and UH3DE028860. The opinions and assertions in this article are those of the authors and are not to be construed as necessarily representing the views of the organizations the authors are affiliated with, the organizations that participate as network regions or the National Institutes of Health.

ORCID iD

Janine Fredericks-Younger

References

Forum on Neuroscience and Nervous System Disorders, Board on Health Sciences Policy, Health and Medicine Division, et al. Transforming clinical trials with technology. In: Stroud

Bain

Norris

SMP

(eds) Neuroscience trials of the future: proceedings of a workshop. Washington, DC: National Academies Press, 2016, pp. 37–42.

Ngayua

Agyei-Boahene

. Applying advanced technologies to improve clinical trials: a systematic mapping study. Scientometrics 2021; 126(2): 1217–1238.

Chettri

Wang

Balkin

, et al. Development of a mobile app for clinical research: challenges and implications for investigators. JMIR Mhealth Uhealth 2022; 10(4): e32244.

Sharma

Harrington

McClellan

, et al. Using digital health technology to better generate evidence and deliver evidence-based care. J Am Coll Cardiol 2018; 71(23): 2680–2690.

Lawrence

Dunkel

McEver

, et al. A REDCap-based model for electronic consent (eConsent): moving toward a more personalized consent. J Clin Transl Sci 2020; 4(4): 345–353.

Chen

Turner

Sholle

, et al. Evaluation of a REDCap-based workflow for supporting federal guidance for electronic informed consent. AMIA Jt Summits Transl Sci Proc 2019; 2019: 163–172.

McKenna

Geoghegan

Swezey

, et al. Investigator experiences using mobile technologies in clinical research: qualitative descriptive study. JMIR Mhealth Uhealth 2021; 9(2): e19242.

Mitchel

Kim

Choi

, et al. Evaluation of data entry errors and data changes to an electronic data capture clinical trial Database. Drug Inf J 2011; 45(4): 421–430.

Gewandter

Dworkin

Turk

, et al. Improving study conduct and data quality in clinical trials of chronic pain treatments: IMMPACT recommendations. J Pain 2020; 21(9–10): 931–942.

10.

Jibb

Khan

Seth

, et al. Electronic data capture versus conventional data collection methods in clinical pain studies: systematic review and meta-analysis. J Med Internet Res 2020; 22(6): e16480.

11.

Callison

. Redcap. Sutton: Severn House, 2007.

12.

Chettri

Wang

Balkin

, et al. Development of a mobile app for clinical research: challenges and implications for investigators. JMIR Mhealth Uhealth 2022; 10(4): e32244.

13.

Using REDCap to enhance data quality in clinical trials IADR abstract archives, https://iadr.abstractarchives.com/abstract/51am-3664582/using-redcap-to-enhance-data-quality-in-clinical-trials (accessed 13 October 2022).

14.

Crane

Comer

Arenson

, et al. Using REDCap to facilitate web-based therapeutic intervention research. Nurs Res 2019; 68(6): 483–487.

15.

Tomko

Gray

Oppenheimer

, et al. Using REDCap for ambulatory assessment: implementation in a clinical trial for smoking cessation to augment in-person data collection. Am J Drug Alcohol Abuse 2019; 45(1): 26–41.

16.

Harris

Taylor

Minor

, et al. The REDCap consortium: building an international community of software platform partners. J Biomed Inform 2019; 95: 103208.

17.

Harris

Taylor

Thielke

, et al. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42(2): 377–381.

18.

Harvey

. REDCap: web-based software for all types of data storage and collection. Spinal Cord 2018; 56(7): 625.

19.

Kianersi

Luetke

Ludema

, et al. Use of research electronic data capture (REDCap) in a COVID-19 randomized controlled trial: a practical example. BMC Med Res Methodol 2021; 21(1): 175.

20.

Feldman

Fredericks-Younger

, et al. The Opioid Analgesic Reduction Study (OARS)-a comparison of opioid vs. non-opioid combination analgesics for management of post-surgical pain: a double-blind randomized clinical trial. Trials 2022; 23(1): 160.

21.

Nomi system, https://nomiadherence.com/system/ (accessed 13 October 2022).

22.

Culiolo

Shaw

. ActiLife, https://actigraphcorp.com/actilife/ (accessed 13 October 2022).