Development of a Computer-Assisted Adverse Drug Events Alarm and Assessment System for Hospital Inpatients in China

Abstract

Background:

Computerized detection is a promising method for monitoring adverse drug events (ADEs); however, this method is currently in its infancy and is a new area of exploration in China. This study aimed to develop a computerized ADE alarm and assessment system to help pharmacists effectively detect, assess, and analyze possible ADEs in patients in China.

Methods:

Based on the clinical characteristics of these adverse drug events, we designed combined multiparameters as ADE alert rules to be assembled into detection configurations. We also developed system function modules by extracting data from the People’s Liberation Army (PLA) general hospital information system (electronic medical records). Positive predictive values were calculated for the alert.

Results:

Five function modules were created in this platform: automatic screening, assisted evaluation, risk characteristic analysis, report generation into SRS (spontaneous reporting system), and a dictionary database. Four ADE alert configurations were set in our ADE alarm and assessment system: drug-related thrombocytopenia, anemia, liver injury, and kidney injury. The positive predictive values of the 4 monitored ADEs were approximately 44.4% to 95.8%.

Conclusions:

An automatic ADE screening system was established for hospitalized patients in Chinese medical institutions. Compared with previous studies, combined drug-event alerts and a system-assisted assessment interface performed better than alerts based only on laboratory values. Furthermore, this platform’s assisted-layered evaluation and risk factor analysis functions could save considerable time for professionals and improve early prevention of potentially serious ADEs. To date, this system has been applied in 10 large-scale medical institutions.

Keywords

adverse drug events (ADEs)clinical pharmacists computerized alarm automatic screening risk characteristics

Introduction

Adverse drug events (ADEs) refer to patient harm resulting from a drug. ADEs have been further defined as harm resulting from a medication dose that may or may not be “normally used in humans.”^1,2 ADEs are the most common category of iatrogenic injuries experienced by hospitalized patients. Recent studies have shown that in adult patients, the reported incidence of ADEs requiring admission to an intensive care unit (ICU) ranged from 0.37% to 27.4%, and the associated mortality rate ranged from 2% to 28.1%, with a mean duration of 2.3 to 6.4 days.^3,4 Thus, ADEs are not only associated with increased morbidity and mortality but are also related to prolonged hospitalization and higher care costs.

Previous ADE detection systems developed several event-detection methods, such as spontaneous reporting models, direct observations, chart reviews, and trigger tools; however, no tools exist that can display these ADEs to physicians without a time lag.⁵ Compared with traditional single spontaneous reporting methods, an integrated approach that involves multiple detection methods would be more effective. Electronic patient records (EPRs), along with the development of powerful computers and new statistical algorithms, hold great potential for detecting unknown ADEs.⁶ Computerized detection may be a promising method for integrated ADE surveillance because it requires less time for large-scale data monitoring and reporting.⁷ Multiple detection principles can be easily integrated into one computerized method that includes direct observation results, chart review, and trigger tools to realize well-designed data collection and real-time monitoring and processing.

The well-accepted computer-assisted identification methods work well in global medical institutions for detecting and reporting ADEs^1,8,9; however, recent research has shown the limitations of computerized ADE detection, which mainly results from a lack of uniform diagnostic standards and an international consensus for various ADEs.⁸ Diagnostic standards and the positive predictive values (PPVs) of the triggers have rarely been discussed, thus limiting their practical utility and efficiency.^10

–15 An international consensus should follow the local medical terminology and drug use habits.^16

–19 Unfortunately, the related research from China has not been included in the database until now.⁸ As a developing country with a large population, much of China’s data for detecting and reporting ADEs have not been fully applied. However, computerized surveillance for ADEs in China remains in its infancy.

In this study, we aimed to develop and investigate a computer-assisted ADE monitoring and assessment system. This platform should help pharmacists effectively assess and quickly analyze ADEs to reduce the harm induced by these ADEs for timely hospital interventions. This system was developed based on a hospital information system from China and is applied to fill the gaps of the common spontaneous reporting system. Previous passive surveillance can be transferred to current active surveillance and real-time monitoring.

Methods

Environmental Setting

The General Hospital of the People’s Liberation Army (PLAGH) is a modern general hospital in China. More than 3.8 million outpatients are seen annually, with over 110,000 admissions and more than 65,000 operations. This study was designed and developed based on the hospital’s sufficient case load and well-run information system.

Designs

System construction

In the ADE automatic screening system, clinical ADE characteristics and combined multiple indicators include medical advice, diagnosis, and laboratory information. The 2 main parts of the ADE screening system are ADE identification and further assessment. Automatic screening was performed first, and the identified results were then validated by clinical pharmacists. In addition, all possible critical clinical features are collected in various diagnoses from target ADEs. The study’s overall design concept is demonstrated in Figure 1.

Figure 1.

The system’s overall design concept.

System-aided screening and classification based on ADE-related confounding factors

ADEs are generally caused by many complicated factors, such as disease progress, concomitant drug use, and underlying diseases. In our system, patients with latent confounding factors were classified into 3 levels based on the case assessment difficulty. Level 1 was set at the baseline value of key trigger indicators that were abnormal before drug administration. Level 2 included confounding underlying diseases (wards) (with greater influence on assessing target events to be monitored). Level 3 was confounding concomitant drug use. Level 1 to 3 cases were more difficult to assess because they were accompanied by confounding factors and required repeated screening, re-evaluation, and case discussion.

We also set exclusion standards. The first was a lack of key trigger indicators, and the second was temporary drug administration within 24 h based on medical advice. These were difficult to identify, even after a chart review. After excluding these cases as well as cases that were difficult to assess from the aforementioned levels, the remaining cases could be easily identified and were defined as conventional cases.

ADE alert configuration design on clinical characteristics and screening motivation

In this study, the 4 most common ADEs in clinical practice were selected, which were drug-induced thrombocytopenia, anemia, liver injury, and kidney injury. The monitoring program should be designed with different monitoring principles, based on the various events’ characteristics. Some critical medical events were selected as key points for building the technical route and system function architecture.

Based on these characteristics, clinical confounding factors, diagnostic criteria, and international ADE consensus, we first designed key indicators (core event indicators), reference indicators (secondary event indicators and indicators of confounding factors), trigger definition, severity classification, exclusion criteria, and preclassification criteria. Second, the suspected drugs catalog for each ADE module, the rescue medications catalog, and the antitumor drugs catalog were constructed. Third, identification rules were set for single ADEs, considering multiple indicators, such as medical advice, diagnosis, and laboratory information. Finally, to improve the automatic monitoring accuracy, the parent version of each ADE alert configurator was designed to adjust to the monitoring aims and target drug characteristics (Table 1).

Table 1.

ADRs alert Configurations of the Platform.

ADRs Alert Configurations	Core Indicators	Reference Indicators	Trigger Definition	Severity Classification	Exclusion Criteria	Preclassification Criteria
Drug-related thrombocytopenia	PLT and its variation range	surgery record, blood loss, myelogram, PT, INR, APTT, CRP, PCT, WBC, NEUT, EOS, RBC, IgE, ALB	PLT was lower than the normal limit (100×10⁹/L) or the 75% of the normal limit after drug administration	Level I: (76-100)×10⁹ /L Level II: (50-75) ×10⁹ /L Level III: (26-49) ×10⁹ /L Level IV: (0-25)×10⁹ /L	Temporary medical advice Lack of PLT indicator (lack of detections before and after target drug administration)	PLT was abnormally decreased (<100×10⁹/L) or increased (>400×10⁹/L) before target drug administration Hematology, oncology related wards The patients with severe acute pancreatitis (SAP), system lupus, hepatoblastoma, severe liver cirrhosis, and blood system disease Combined use of heparin and chemotherapeutic agents
Drug-related anemia	Hemoglobin and its variation range	RBC, PCV, U-RB, U-RBC-C, EOS, IgE	Hb values of 2 consecutive detections after the drug use, male <120 g/L, female <110 g/L	Level I: 95-110 g/L Level II: 80-95 g/L Level III: 65-80 g/L Level IV: 0-65 g/L	Temporary medical advice Lack of Hb indicator (lack of detections before and after target drug administration)	In blood system diseases (such as acute non-lymphocytic leukemia, acute lymphoblastic leukemia, aplastic anemia, and acute myeloid leukemia) Abnormal hemoglobin count before administration Hematology (myeloproliferative disorders)
Drug-related liver injury	ALT, AST, TB	ALP, γ-GT, DB, PT, INR, PTA, ALB, U-BIL, UBG, CRP, HBsAg, HBsAb, HBeAg, HBeAb, HBcAb	ALT&TB values of 2 consecutive detections after the drug use, ALT >40 U/L and/or TB >21 μmol/L, or single detection ALT >80 U/L or TB >42 μmol/L	Per the CFDA standard, Mild: 10 × ULN > ALT > 2 × ULN “or” 1 × ULN <TB ≤ 5 × ULN Severe: ALT ≥ 10 × ULN “or” 5 × ULN < TB ≤ 10 × ULN	Temporary medical advice Lack of ALT/AST/TB indicator (lack of detections before and after target drug administration)	Major/discharge diagnosis included the diseases of “liver” or “gallbladder” or “jaundice” Cases of abnormalities in the most recent key indicator and the baseline value prior to administration The gastrointestinal cancer ward
Drug-related kidney injury	SCr and its variation range	UN, U-RBC, U-RBC-C, U-PRO, U-P.CAST, U-Volume, Pro-BNP, EOS, PTA, ALB	For the changing serum creatinine range, the SCr increase administering the drug was higher than the baseline value by more than 26.5 μmol/L or more than 50% of the baseline value.	Per the CFDA standard, SCr was used as the main indicator supplemented by the prothrombin time activity, albumin, and other indicators. Acute kidney injury: The highest SCr value was 50% higher than the baseline after administration, or the highest SCr value was ≥354 µmol/L and the acute increase was 44.2 µmol/L after administration	Temporary medical advice Lack of SCr indicator (lack of detections before and after target drug administration)	Hospital admission/discharge diagnosis included the diseases of “kidney” or “heart failure” Cases of abnormal SCr baseline before administration (UN was only used as a reference index) Department of Nephrology ward.

Abbreviations: ALB, serum albumin; ALP, alkaline phosphatase; ALT, alanine transaminase; APTT, activated partial thromboplastin time; AST, aspartate transaminase; CFDA, China Food and Drug Administration; CRP, C-reaction protein; DB, direct bilirubin; EOS, eosinophil; γ-GT, γ-glutamyl transferase; HB, hemoglobin; IgE, immunoglobulin E; INR, international normalization ratio; NEUT, neutrophil; PCT, procalcitonin; PCV, packed cell volume; PLT, platelet count; Pro-BNP, pro-brain natriuretic peptide; PT, prothrombin time; PTA, prothrombin activity; RBC, red blood cell; TB, total bilirubin; SCr, serum creatinine; UBG, urobilinogen; U-BIL, urine bilirubin; UN, urea nitrogen; U-PRO, urine protein; U-P.CAST, urine pathological cast count; U-RBC, urine RBC; U-RBC-C, urine RBC count; U-Volume, urine volume; ULN, upper limit of normal value; WBC, white blood cell.

System Effect Assessment

To ensure the reliability of the assessment results by manual confirmation, 4 experienced clinical pharmacists and two physicians were invited to assess the results. For the positive cases screened by a computer, we evaluated the relevance with an “assisted evaluation” module function in the system. The causality was marked as “certain,” “probable,” “possible,” “unlikely,” or “unassessable.” We referred to the WHO’s causality criteria. The staff received relevant training to improve their evaluation accuracy. Next, the retrospective monitoring cases that were manually evaluated as true positives by the automatic alarm system were input into the risk characteristics analysis module to analyze the occurrence characteristics. The true-positive cases evaluated by the real-time monitoring program were subjected to intervention by clinical pharmacists in combination with actual conditions. Furthermore, these true-positive cases could be transmitted to the hospital’s Individual Case Safety Reports (ICSR) Reporting System, which could then be reported to the Chinese National ADR Monitoring System (NADRMS) by the “report generation into SRS” module. The causes of the false-positive cases in the automatic screening will be analyzed and used to improve the system’s automatic screening rules.

Participants and Data Collection

The study population was selected from inpatients at PLA General Hospital from January 1, 2015, to September 30, 2016. Selection criteria included all patients over age 18 who were administered linezolid, vancomycin, gemcitabine, or levofloxacin. Data were collected using the “Automatic Monitoring and Evaluation Alert System for Adverse Drug Events in Hospitalized Patients.” Ethical approval for the study was received from the Ethics Committee of General Hospital of the People’s Liberation Army.

Statistical Analysis

All calculations of descriptive statistics were performed using SAS, version 9.1.3 (SAS Institute, Cary, NC). Groups were compared using a t test (parameters), Mann-Whitney U test (nonparameter), or chi-square test. Data are presented as the mean ± SEM, with significance at P < .05.

Results

System Function Modules

Five function modules were created (Table 2): automatic monitoring, assisted evaluation, characteristics analysis, risk warning, and a computable knowledge base. The Automatic Monitoring Module is the study’s basic module, which transfers drug-event monitoring plans into directives based on combination alert rules. The warning result was reviewed by clinical pharmacists or professionals during working hours the next day. The automatic monitoring result would then be presented in the second monitor module, the Review and Evaluation Module, which was created to provide clinical pharmacists and physicians with supporting information that could be used to confirm ADEs. The Characteristics Analysis Module is the third module, which was obtained by statistically analyzing previous true-positive cases. These cases were re-evaluated to confirm their characteristics. The Risk Warning Module includes ICSR generation, intervention suggestions, and monitoring reports. The ICSR generation module takes charge of positive case reporting, including system warnings, confirmed by clinical pharmacists. The Dictionary Database Module comprises event configuration and basic information maintenance. Basic information maintenance includes user management, indicator maintenance, coding control, the antitumor directory, medication review rules, a special indicators dictionary, HIS interface, and data synchronization.

Table 2.

Function Modules of the Platform.

Module	Function	Items (parameters)
Automatic screening	To develop and implement the monitoring plan based on the diagnosis, doctor’s advice, test indicators, and medical record	Monitoring events (single or multiple selection by the ADE module), drugs (specifications/unit/manufacturer), wards, patients (age/sex), methods (real-time/retrospective), and time (start/end) are selected
Assisted evaluation	To establish a case assessment table for the alarm case, stratified cases, and excluded cases assessed by automatic monitoring for reassessment by clinical pharmacists to determine causality (causality assessment)	Tables for each case to be assessed are established, including patient demographics in 19 items; patient’s physiological state (liver and kidney function 1 day before using the monitoring target drug / surgical condition / transfusion situation / reference indicators of each event); doctor’s advice (auto-labeled suspicious auxiliary drugs); adverse reaction information (automatic calculation of the onset time, latency, recovery time, extreme value, variability, chart for changing trends of the sensitive indicators); correlation of the assessment factors with causality assessment (expectedness / reasonableness of the time frame / de-challenge / re-challenge, underlying disease/concomitant drug use); evaluation criteria of the correlational WHO causality criteria (certain/probable/possible/unlikely/unassessable)
Risk characteristics analysis	To statistically analyze the positive cases after assessment (the level above “may”) and fitting the characteristic curve	Incidence, severity, time of onset, latency, recovery time, extreme value, variability
Risk warning/link to SRS	To generate warning reports based on the risk characteristics, alert the clinical pharmacists to intervene, and connect with the hospital’s internal ADE spontaneous reporting system to generate an ICSR (individual case safety report)	CFDA standard (NADRMS)²⁰ report generation into SRS
Dictionary database	To maintain the database (computable knowledge base)	ADE event configurator, basic drug information, code comparison, suspicious drug library and corresponding rescue drug library prepared by the ADE module, anticancer drug library, rights management

Demonstration of the Systematic Monitoring Efficacy of Linezolid-Related Thrombocytopenia

Linezolid-related thrombocytopenia was used as an example to demonstrate the system’s efficacy.¹⁸ A total of 746 cases and 899 medication instances were monitored, screened, and re-evaluated by the system automatically. The time taken was 5 minutes 41 seconds. One hundred seventy-five ADE cases were reported, and 41 cases were warned, which were divided into 6 levels according to the previous standards. No thrombocytopenia was found in 134 cases. The warning cases were re-evaluated and confirmed by clinical pharmacists and physicians in the respiratory department (qualified control). Finally, 25 cases were confirmed as true positives. The positive predictive value was approximately 60.98%, and the incidence was 15.72%. Among these, 13 cases were confirmed to be between grade III and grade IV thrombocytopenia. In these cases, platelet counts lower than 75% of the baseline value were observed in 8 cases (4.57%).

In the system’s automatic preclassification steps, 150 cases of “lack of indicators” and 50 cases of “temporary medical advice” were first excluded, and 524 cases labeled as “unable to evaluate” or “hard to evaluate,” which were classified as levels 1 to 3, were layered displays. The obtained results were as follows (Table 3): 244 cases of abnormal base value (59 warnings of platelet counts below 75% of the base value, 24%), 187 cases of combined use of heparin (112 warnings, 60%), and 93 cases of mixed diseases (30 warnings, 32%). Because the mixed diseases were tumors, hemopathy, and rheumatic immunologic diseases, the positive cases were re-evaluated and confirmed by oncologic clinical pharmacists and physicians. Four cases were positive, 15 may have been irrelevant, and 11 cases could not be assessed. The total positive predictive value was 13.3%.

Table 3.

Automatic Systematic Monitoring of Linezolid-Related Thrombocytopenia and Reassessment of the Prestratified Cases Based on Confounding Factors.

Case type	Total	Warning	Normal	Simulated incidence	Confirmation	Positive predictive value
Conventional cases	175	41	134	23%	25	60.98%
Lack of indicators	150	ND	ND	ND	ND	ND
Temporary orders	50	ND	ND	ND	ND	ND
Abnormal basic value	244	59*	ND	24%*	ND	ND
Combined use of heparin	187	112	75	60%	ND	ND
Mixed diseases	93	30	63	32%	4	13.3%

* platelet count below 75% of basic value.

For the function “Characteristics Analysis,” 4 typical cases with causality marked as “certain” were obtained, and the characteristics of linezolid-related thrombocytopenia could be obtained (median time). The platelets were reduced to the lower limit of normal after drug administration for 7 days (median), the values were reduced to a minimum value 2 days after drug withdrawal (median), and the recovered normal value was 5.5 days after drug withdrawal (median). Platelet transfusion was performed in one case. A relevant factors analysis was performed for 25 true-positive cases, in which causality was marked as “certain,” “probable,” or “possible.” Intergroup analysis was performed on basic information from patients with or without linezolid-related thrombocytopenia (Table 4). The results showed a significant difference between the groups for age, body mass index, weight, and critical condition (P < .01).

Table 4.

Demographic and Clinical Data Analysis of the Relevant Factors of Linezolid-Associated Thrombocytopenia.

Indicator		Yes	No	P Value
Sex	Male, n (%)	15 (60.00)	92 (68.66)	.3970
Sex	Female, n (%)	10 (40.00)	42 (31.34)	.3970
ICU	Yes, n (%)	15 (60)	33 (24.63)	.0010
ICU	No, n (%)	10 (40)	101 (75.37)	.0010
Age	N (missing)	25 (0)	134 (0)	.0026
Age	Mean (SD)	64.20 (20.02)	52.31 (17.42)	.0026
Height	N (missing)	25 (0)	131 (3)	.0381
Height	Mean (SD)	164.08 (7.06)	167.51 (7.60)	.0381
Weight	N (missing)	25 (0)	129 (5)	.0000
Weight	Mean (SD)	57.54 (11.65)	67.93 (10.92)	.0000
BMI	N (missing)	25 (0)	129 (5)	.0003
BMI	Mean (SD)	21.30 (3.75)	24.22 (3.56)	.0003

Abbreviation: BMI, body mass index; ICU, intensive care unit.

Validation of 4 Adverse Events and Targeted Drugs

If the ADE is a drug-induced disease, the ADE’s automatic monitoring and assessment system works as a screening and reporting tool. Thus, the selected 4 target drugs are known to correlate with the corresponding ADEs, including linezolid-associated thrombocytopenia, vancomycin-associated kidney injury, levofloxacin-associated liver injury, and gemcitabine-associated anemia. Finally, the system efficacy was confirmed by clinical pharmacists and physicians from respiratory, anti-infection, and oncology departments (Table 5).

Table 5.

Validation of 4 Adverse Events and the Targeted Drugs.

Event Types^a	Target Drug	Routine Cases	Warning	Confirmation	Positive Predictive Value, %	Incidence, %
Thrombocytopenia	Linezolid injection	175	41	25	60.98	15.72
Kidney injury	Vancomycin injection	619	9	4	44.4	0.65
Liver injury	Levofloxacin injection	1763	71	34	47.89	2.66
Anemia	Gemcitabine hydrochloride	136	24	23	95.8	17.04

^a Evaluated by clinical pharmacists and physicians from respiratory, anti-infection, cardiology, and oncology.

Discussion

Multifunctional Automatic Monitoring System

This system combined multiple functions instead of a single trigger tool. An efficient ADE computer-aided monitoring system should be established based on effective human-machine cooperation. The trigger principle of the ADE automatic monitoring system is easy to implement, and the keys to automatic identification are the rule algorithm and the system’s sensitivity and specificity; however, identifying the signals is only the first step. The second step is to confirm, evaluate, and intervene the signals, which must be performed by clinical pharmacists and specialty clinicians. Because of the complexity of the clinical background, in practical applications, this stage is the key to confirming the true positive rate of the automatic alarm cases. Assisting medical professionals in quickly assessing, effectively analyzing, and providing intervening warnings is the important manifestation of system performance and practicality.

In this study, based on the discriminatory features of each ADE and the principle of causality, the confounding factors in actual clinical practice were comprehensively considered in designing the individual assessment table for the automatic alarm cases and unalarmed cases in the “assisted evaluation” module with the preset auxiliary evaluation indicators. Using the drug-related thrombocytopenia event as an example, because infection and surgery may be predisposing triggers of thrombocytopenia, more than 10 disease progression indicators that may affect the evaluation of the ADE correlation were automatically listed, including C-reactive protein, procalcitonin, white blood cell counts, neutrophil counts, surgical records, and blood loss. In addition, the assessment table can automatically reveal the patient’s abnormal physiological status before using the drug, review medication problems, mark suspicious drugs and corresponding rescue drugs, illustrate the graph to visually reflect the changing trend of the sensitive indicators, and calculate the adverse event characteristics, such as the changing trend, onset time, and duration, thus helping medical professionals quickly assess the cases and minimize the workload of searching original medical records. During testing, the system performance was commended by the clinical pharmacists and doctors.

The system’s functional characteristics were also reflected in the risk characteristic assessment of the target drug to improve risk prevention. Generally, to assess the drug risk, the severity, incidence, duration, target population, and intervention efficacy for the adverse events should be considered. Only when the risk characteristics are discovered and understood can the risk be better managed. This system can provide information on the incidence, severity, risk factors, onset time, and duration of adverse reactions and can analyze the auxiliary drugs that may increase the risk by analyzing the true-positive case characteristics. These are important for new drugs and special drug users

Drug Adverse Event Combination Alert and Confounding Factor Control Can Improve the System’s Monitoring Performance

The ADE automatic monitoring system simulates artificial screening and diagnosing drug-borne diseases. This study’s first purpose was identification, so the technical program was designed to correspondingly monitor the target drug-key events. The judging times were both before and after the medication, rather than mass comparing and screening to infer the suspected drugs based on the exposed event, thus providing better accuracy and higher sensitivity. For example, the positive predictive value of the cases for performance verification was higher. However, this does not mean that the system can only monitor a single target drug. The system can also monitor a group of target drugs or a group of adverse drug events, such as for combinations of single drug–multiple ADEs, multiple drug–single ADEs, and multiple drug–multiple ADEs by monitoring event and drug selection. In clinical practice, we will make a list of the drugs used in the clinical ward or those drugs that are frequently used. We then import them into the monitoring program of the system, establish the target drugs group, and select alert configurations to match all the ADEs. In this way, the detected signals may include unanticipated risks that are not mentioned in the drug instructions or that have a significantly increased frequency and severity compared to previous reports. We believe that the ADEs’ automatic identification rule cannot equal the drug-borne disease diagnostic criteria. The research purpose of monitoring needs and clinical issues implies that we should leave room for the system to maintain and adjust the automatic identification rules. For example, a changing range was designed for the key indicator alarm in the event configurator in this study, so the real-time intervention timing could be selected based on the event’s progress.

Determining causality between drugs and adverse events is often accompanied by more confounding factors. For severe and complex cases in particular, manual differential diagnoses remain difficult and controversial, and automatic monitoring is more difficult. Therefore, in designing the program, these cases adopted a “layered screening” strategy to allow specialized assessments. The system’s verification results showed that this design concept of reassessing the prestratified cases based on ADE confounding factors is feasible. As shown in Table 3, the cases with abnormal base values and combined use of heparin (levels 1 and 3) had much higher automatic alarm rates than did the conventional cases, which also exceeds the scope of the clinical interpretation credibility and the existing research results on the incident patterns of the events. The dominant factors leading to adverse events caused by the target drug can be ruled out, so it is difficult to assess their use for special case analyses. The positive predictive value of the alarm cases in the stratified cases with confounding factors of the original disease and tumor wards was only 13.3% (level 2) after being assessed by specialist clinical pharmacists, which was much lower than the alarm result from the conventional cases. The main reasons that the pharmacists determined the false positives include chemotherapy, radiotherapy, and molecular targeted therapy for cancer, or primary cancer invasion of the myeloid system, which is weakly associated with the monitored target drug. In these cases, differentiation is complex and may be considered unassessable because the target drug’s contribution value and the confounding factors in the ADE cannot be fully distinguished. Stratifying these complicated cases can significantly improve the detection rate and system accuracy.

Practical Value and System Scalability

The system has a unified graphic user interface (GUI) with graphic management, which can configure various hospital information systems and significantly reduce the interface development workload in the installation and the difficulty of implementation. The hospital’s original data can also be regulated and structured to achieve decoupling between the system’s core functions and each hospital’s basic data. To date, it has been applied in 10 large hospitals and shows good compatibility.

In addition, this system can connect to the hospital’s internal ADE spontaneous reporting system to generate the ICSR and finally submit the positive report to the National ADE Monitoring System in China, which is a helpful supplement to the spontaneous ADE reporting system. Moreover, the system’s verification results showed that for the manual retrospective study of linezolid-associated thrombocytopenia previously conducted by our study group²¹ with a monitoring period of 1 year, the data collection, collation, assessment, and analysis required specified personnel to perform 3 months of specialized work. After using the system in this study, the alarm time for the cases of the same monitoring period was only a few minutes, and the positive predictive value of conventional cases was better. This result indicates that this system effectively saves manpower, time, costs, and other monitoring resources, expands the monitoring scope, improves monitoring efficiency, identifies new safety signals, and achieves real-time monitoring and ADE treatment. In addition to signal detection, it is conducive to centralized research on key drug monitoring in several hospitals.

Limitations of the System

Some problems and limitations require further study. The incompatible symptoms’ description in the current patient records is the main limitation. Some patients may have systematic symptoms such as chills, fever, weakness, body aches, nausea, vomiting, headache, abdominal pain, joint pain, and itchy and flush skin. For these symptoms, the current system cannot perform the associated recognition. Therefore, in further studies, other recognition rules, such as extracting clinical narrative text, should be gradually added and integrated. Another limitation is that many important ADEs are not included, such as drug-induced pancreatitis, drug-induced coagulation disorders, and drug-induced glucose abnormalities. These will be addressed in our follow-up study.

Conclusions

To our knowledge, this study is the first multifunctional automatic monitoring system involving multiple adverse drug events in China’s medical institutions. It has been used in 10 major hospitals as an ADE automatic monitoring tool. The system developed in this study first achieved automatic screening and computer-aided ADE assessment in a large-scale general hospital in China. The system can also be integrated with the hospital’s existing spontaneous reporting system for adverse drug reactions. Hence, a practical platform linking active and passive ADE monitoring in medical institutions has been established. We have established the overall functional framework and modular design of the automatic monitoring system; we only need to continue adding the event configurator modules for different ADEs in our follow-up study. In addition, we are using text-mining techniques in electronic patient records to identify drug hypersensitivity reactions from medical use. To sum up, this system could help detect and evaluate signals, improve monitoring efficiency, save investments in monitoring resources and integrate risk characteristics for further epidemiologic studies.

Footnotes

Authors’ Note

Chao Chen and Wangping Jia contributed equally to this work.

Declaration of Conflicting Interests

No potential conflicts were declared.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was financially supported by the Innovation Fund of the General Hospital of the People's Liberation Army (10KMM41) and the key projects of the Army's logistics research programs (BWS14R039).

ORCID iD

Daihong Guo, MD

References

Bates

Cullen

Laird

. Incidence of adverse drug events and potential adverse drug events. JAMA. 1995;274:29–34.

Edwards

Aronson

. Adverse drug reactions: definitions, diagnosis, and management. Lancet. 2000;356:1255–1259.

Jolivot

Hindlet

Pichereau

. A systematic review of adult admissions to ICUs related to adverse drug events. Critical Care. 2013;18:1–17.

Hartwig

Denger

Schneider

. Severity-indexed, incident report-based medication error-reporting program. Am J Hosp Pharm. 1991;48:2611–2616.

Meyer-Massetti

Cheng

Schwappach

. Systematic review of medication safety assessment methods. Am J Health Syst Pharm. 2011;68:227–240.

Warrer

Hansen

Juhl-Jensen

. Using text-mining techniques in electronic patient records to identify ADRs from medicine use. Br J Clin Pharmacol. 2012;73:674–684.

Figueiras

Herdeiro

Polónia

. An educational intervention to improve physician reporting of adverse drug reactions: a cluster-randomized controlled trial. JAMA. 2006;296:1086–1093.

Forster

Jennings

Chow

. A systematic review to evaluate the accuracy of electronic adverse drug event detection. J Am Med Informat Assoc. 2011;19:31–38.

Kilbridge

Campbell

Cozart

. Automated surveillance for adverse drug events at a community hospital and an academic medical center. J Am Med Informat Assoc. 2006;13:372–377.

10.

Classen

Resar

Griffin

. “Global trigger tool” shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff. 2011;30:581–589.

11.

Takata

Mason

Taketomo

. Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children’s hospitals. Pediatrics. 2008;121:927–935.

12.

Burch

. Using a trigger tool to assess adverse drug events in a children’s rehabilitation hospital. J Pediatr Pharmacol Ther. 2011;16:204–209.

13.

Ferranti

Horvath

Cozart

. Reevaluating the safety profile of pediatrics: a comparison of computerized adverse drug event surveillance and voluntary reporting in the pediatric environment. Pediatrics. 2008;121:1201–1207.

14.

Matlow

Flintoft

Orrbine

. The development of the Canadian paediatric trigger tool for identifying potential adverse events. Healthc Q. 2005;8:90–93.

15.

Sharek

Horbar

Mason

. Adverse events in the neonatal intensive care unit: development, testing, and findings of an NICU-focused trigger tool to identify harm in North American NICUs. Pediatrics. 2006;118:1332–1340.

16.

Härmark

Hunsel

Grundmark

. ADR reporting by the general public: lessons learnt from the Dutch and Swedish systems. Drug Saf. 2015;38:337–347.

17.

Huang

Moon

Segal

. A comparison of active adverse event surveillance systems worldwide. Drug Saf. 2014;37:581–596.

18.

Bouvy

Bruin

MLD

Koopmanschap

. Epidemiology of adverse drug reactions in Europe: a review of recent observational studies. Drug Saf. 2015;38:437–453.

19.

Pal

Olsson

Brown

. The monitoring medicines project: a multinational pharmacovigilance and public health project. Drug Saf. 2015;38:319–328.

20.

Hou

. National ADR monitoring system in China. Drug Saf. 2016;39:1043–1051.

21.

Chen

Guo

Cao

. Risk factors for thrombocytopenia in adult Chinese patients receiving linezolid therapy. Curr Ther Res. 2012;73:195–206.