Abstract
The clinical research landscape has changed dramatically in recent years in terms of both volume and complexity. This poses new challenges for Institutional Review Boards’ (IRBs) review efficiency and quality, especially at large academic medical centers. This article discusses the technical facets of IRB modernization. We analyzed the information technology used by IRBs in large academic institutions across the United States. We found that large academic medical centers have a high electronic IRB adoption rate; however, the capabilities of electronic IRB systems vary greatly. We discuss potential use-cases of a fully exploited electronic IRB system that promise to streamline the clinical research work flow. The key to that approach utilizes a structured and standardized information model for the IRB application.
Keywords
Remarkable growth has occurred in biomedical research over the past several decades. Total biomedical research funding by principal sponsors based in the United States nearly doubled from US$47.8 billion in 1994 to US$94.3 billion in 2003 (Moses, Dorsey, Matheson, & Thier, 2005). Although the rate of that increase may be slowing (Dorsey et al., 2010), the sheer amount of research now being conducted has resulted in an unprecedented increase in Institutional Review Boards’ (IRBs) workload. This places a severe burden on IRBs that have always operated under significant resource constraints (Catania et al., 2008).
The changing context of biomedical research poses new challenges for IRBs. There is a growing trend toward multisite clinical research, complicating the traditional, institution-oriented oversight system. A substantial literature discusses the problems faced by IRBs reviewing multisite studies including redundancy, inconsistency, and inefficiency in communications between IRBs from a central site to participating sites (Burman, Reves, Cohn, & Schooley, 2001; Greene & Geiger, 2006; McWilliams et al., 2003).
With this increase in research volume, the number of IRBs in the United States has also increased substantially. Catania et al. (2008) showed that the number of IRBs increased by 41% from 2004 to 2008. According to recent (March 2013) data from the Office for Human Research Protections (OHRP), there are 2,937 Institutional Review Board Organizations (IORGs), which include 3,589 active individual IRBs. Clinical research professionals spend a great deal of time preparing IRB applications, amendments, renewals, and reports. Studies show that investigators complain that the IRB application process is burdensome and, in some instances, the wait to obtain IRB approval has delayed project initiation (Catania et al., 2008; Ness, 2007; Silberman & Kahn, 2011; Straight, 2009).
The etiology of IRB inefficiency is complex. Much of the published work in this area has focused on proposed regulatory reforms, alternative review models, and enhanced education strategies for researchers and IRB personnel (Greene & Geiger, 2006). To the best of our knowledge, there is no existing research addressing the technical aspects of the issue. In this article, we analyze the adoption of information technology by IRBs across the nation and discuss the considerable potential of electronic IRB systems.
Background
Significant effort has been focused on designing information systems that support the efficient conduct and tracking of clinical studies. These efforts led to the emergence of a domain referred to as Clinical Research Informatics (CRI; Kahn & Weng, 2012). In the human subject protection subdomain, many institutions are employing some form of information technology to support IRB application submission, tracking, and review. In this article, the term e-IRB (electronic IRB) will be used to refer collectively to those information systems that support, at minimum, a user interface for online IRB application submission. According to the 2012 metrics report from the Association for the Accreditation of Human Research Protection Programs, Inc. (AAHRPP; 2014), shown in Figure 1, more than 90% of their accredited institutions use a database to track IRB applications. Fewer organizations use an online system for IRB application or review, but the trend shows a steady increase over the past few years.
The 2013 metrics report from the Association for the Accreditation of Human Research Protection Programs, Inc.
The AAHRPP metrics report gives a general idea of the self-reported information technology deployment across IORGs. However, it is not clear what specific types of “online systems” are in use and what system capabilities are provided. To gain a deeper insight into the technological sophistication of IRBs in the United States, we analyzed e-IRB use at major academic institutions across the United States.
Method
The goal of our project is to analyze the current IRB submission approaches used by large academic medical centers across the United States. For institutions that employ an e-IRB, we further categorize the information system based on the system’s source (e.g., in-house developed software, commercial software, software as a service, etc.). The purpose of our analysis is to characterize system variation across different institutions, identify the gaps in supporting more sophisticated IRB review tasks, and propose a research agenda to address those gaps.
We chose to study IRB application systems used by all of the Clinical and Translational Science Award (CTSA) centers (CTSA Institutions, n.d.). By 2013, the CTSA program supported by the National Institute of Health (NIH) had expanded to 61 academic medical institutions since its launch in 2006. We chose CTSA institutions as our target sample for three reasons:
CTSA institutions are leading academic medical centers in the United States. As such, they are research intensive and have a heavier workload compared with small institutions (Catania et al., 2008).
The CTSA program aims to build a collaborative environment across institutions and this will have important implications for IRB reviews of multisite studies.
CTSA institutions have a strong focus on the adoption of information technology to increase the efficiency and speed of clinical and translational research.
We conducted our analysis by reviewing the official IRB websites at each CTSA center and related participating institutions over the period of September to December 2012. Most of the IRB websites post detailed IRB submission guidelines, training documents, or videos for investigators. We reviewed all available materials to determine the type of submission system in use and to compare the content of the application forms across institutions.
In those cases where we were unable to determine these details from the website alone, we called or emailed the IRB office to collect information about their e-IRB. If we called the IRB office 3 times and were still unable to obtain the relevant information, then we counted it as unknown and excluded it from the final analysis. Compared with the more normative approach of using surveys, our use of online resources avoided the standard problems of low response rate and long response periods (Archer, 2008).
Results
All 61 CTSA centers were included in the analysis. Each CTSA center could have had one or more partner institutions. For those CTSA centers, we chose two to three partners at random to include in the study. We ended up analyzing 121 individual institutions in the 61 CTSA centers. Among the 121 individual institutions, there are 103 IORGs in total, because some institutions relied on an external IORG at a partner institution. Among the 103 IORGs we analyzed, one IORG used two different submission systems for their different IRB panels. Therefore, 104 submission systems were analyzed in total. Eight of the 104 systems were excluded because we neither could find relevant information on their website nor reach the relevant personnel by phone or email. The 96 remaining systems were categorized into three top categories: Hard Copy, Email (flash drive or CD), and Online Submission as shown in Table 1.
Analysis Results of IRB Application Systems Used in CTSA Institutions.
Note. IRB = Institutional Review Boards; CTSA = Clinical and Translational Science Award; e-IRB = electronic IRB; SaaS = software as a service; IORGs = Institutional Review Board Organizations.
One hundred three IORGs are analyzed in total. One IORG used two different submission systems for their different IRB panels. Of the 104 submission systems analyzed, 8 were excluded because we neither could find relevant information on their website nor reach the relevant personnel by phone or email.
Online submission systems were further categorized into five subcategories based on the data collected:
Commercial e-IRBs: e-IRB systems that were purchased from commercial vendors.
e-IRBs developed in-house: e-IRB systems that were developed internally by the institution’s technology team.
Software as a service (SaaS) e-IRBs: e-IRB systems where submitted IRB applications were hosted by a vendor and made available to customers as services, essentially a cloud-based service. SaaS e-IRBs are accessed by users using a Web browser.
Community source e-IRBs: A special type of open-source software with institutional contributions by colleges, universities, and some commercial firms rather than from individuals.
Other online e-IRBs with unspecified system types.
This categorization identifies the diversity of existing e-IRB systems. Commercial e-IRBs from the same vendor are built upon a similar hardware and software platform but can be configured, to some degree, to match the needs of a given IRB. Each e-IRB developed in-house will be built on a distinct hardware and software platform, and the IRB can define its own forms and work flow. Such an approach maximizes the utility of the e-IRB but does so at the cost of a considerable investment in technical staff. The SaaS approach permits the least degree of flexibility but has the advantage of being the least expensive. Community source e-IRBs allow an IRB to benefit from best practice insights from other institutions but also require a substantial investment in technical staffing. A comprehensive review of the most desirable features of an e-IRB system is provided in He et al. (2014).
To analyze the comprehensive list of features supported by different IRBs, we need to get access to the system or detailed system documentation. As most IRB systems are only accessible to its internal users, it will be difficult for us to evaluate the individual systems’ features. This can be studied by distributing questionnaires IRB administrators as future work. About 20% of the IORGs used an independent commercial IRB service as a complementary IRB review method for industry-sponsored clinical studies. From the analysis, we obtained a deeper insight into the IRB application systems used across the nation. Our major findings are listed below:
As shown in Table 1, 72% of IORGs used some type of online submission, which is higher than the sample reported by the AAHRPP metrics for the same year (60% in 2012).
Slightly more than 72% (50 out of 69) of CTSA institutions using an online submission purchased commercial software or developed their own e-IRB systems rather than using SaaS e-IRBs.
The capabilities of the online submission IRB systems vary greatly. Some online submission systems simply allow investigators to upload application documents in Word or PDF format. Others have “smart form” features that can guide investigators through relevant Web forms dynamically based on user input.
The design of each institution’s application templates or forms varies widely. Most commercial e-IRB systems support customizable forms, which allow flexible customization. However, this also means that two institutions using the same e-IRB product may have totally different application form designs.
Many of the commercial e-IRB systems used by CTSA centers are part of a complete electronic research administration solution from vendors. A single vendor may provide a suite of software products for multiple research administration activities such as grant proposal preparation/submission, award management, publication management, and so on.
The IRB application systems used in one institution can be diverse. A given university can have multiple IORGs for different departments or units. Each of them may use a different IRB application system.
Discussion
We analyzed the information technology used by IRBs in major academic medical centers across the United States. Referring again to Figure 1, to the extent that the IORGs tracked by AAHRPP are representative of average national adoption rates in the United States, CTSA centers show above average e-IRB adoption. However, the capabilities of CTSA online submission IRB systems vary greatly. About a quarter of CTSA e-IRB systems (28%) only accept hardcopy applications or applications submitted through email or flash drive. Of the rest, about 20% choose to build the IRB technology themselves in-house. That diversity, coupled with a lack of information exchange standards, complicates networking IRB sites. The diversity between e-IRBs arises from different interpretations of federal regulations, organizational policy and priority variations, the increasing complexity of human subject studies, and a lack of standardization and harmonization efforts. The IRB domain analysis model we presented elsewhere (He et al., 2014) is an attempt to promote best practice in e-IRB design by standardizing information collection and work flow.
The content design of each institution’s application templates or forms also varies widely, even for institutions using the same commercial product. We relied on our abstraction of online materials at the institutions we studied, which has the potential to introduce a systematic bias in our sampling. However, we abstracted from the same online materials that researchers themselves would be consulting and found that the data we were collecting were straightforward to collect. Current e-IRB systems primarily focus on managing internal administrative tasks. To the best of our knowledge, none of the systems we studied provided content-based review support for IRB members.
We discuss below the considerable potential of e-IRB systems in enhancing review efficiency and quality, and in streamlining the clinical research work flow. The institutions that can benefit most from fully exploited e-IRBs are those that have a high volume of clinical research and have a significant investment in clinical research information infrastructure. To realize the promise of e-IRBs, a structured and standardized IRB application model approach is essential (He et al., 2014).
Given the critical role that research ethics committees play in human subjects research, we find it surprising that the technological support for IRB’s lags that found in other aspects of research such as data analytics, statistical analysis, and CRI applications (Kahn & Weng, 2012). IRBs, and the researchers who depend on them, clearly have an interest in being as efficient and as well-integrated with the rest of the research information flow pipeline as possible. In the next section, we suggest several possible ways to realize better e-IRBs.
We argue that a fully developed e-IRB has the potential to go beyond basic administrative tasks. It can contribute to enhanced IRB review efficiency and quality, as well as streamline the clinical research work flow.
First, the IRB application repository is a valuable resource for archiving all human subject research conducted at an institution. It holds the potential to help a variety of stakeholders such as investigators, reviewers, policy makers, and accreditation institutions for future analysis, performance measurement, reporting, and auditing. Review criteria based on best practices can be established by referencing previously reviewed study protocols and sharing these across institutions. For example, Brown and Merritt (2013) have proposed organizing information about incentives offered to participants with reference to a commonly recognized set of ethically salient parameters. Information buried in scanned images or unstructured documents will limit the ability to search and analyze such information across studies efficiently, hindering IRBs from developing and applying consistent policies and comparable practices.
Second, e-IRB systems can be designed to enhance review quality and efficiency via automated review decision support enabled by structured study protocol information and pre-defined rules based on regulations and best practice. For example, applications could be automatically assigned to IRB members with corresponding experience or expertise based on the area of study.
Third, e-IRB systems can be integrated with each other to streamline the clinical research work flow. For multisite studies, sharing application information among IRBs in participating institutions can facilitate joint review by relieving duplicative burdens on both IRBs and investigators.
From a technical standpoint, all the use-scenarios discussed above necessarily rely on structured and standardized IRB application information. The content of the scanned paper IRB applications, or Word or PDF files, is extremely hard to search efficiently across documents. Even for those systems that support Web forms, most information is stored as unstructured free text. This hinders collaborative review among different IRBs by limiting the exchange and sharing of IRB application information. A domain analysis model for e-IRB systems to address these issues, by defining standardized and structured application information elements, is discussed in He et al. (2014).
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by a National Center for Advancing Translational Sciences (NCATS) award 8TL1TR000103-05 (Former Number: 1U54RR023426-01A2).