Interventions Assessment of Prescription Automatic Screening System in Chinese Hospitals: A Systematic Review

Abstract

Objectives: To systematically assess the outcomes and effectiveness of Prescription Automatic Screening System (PASS) on rational drug use in China. Methods: All studies that evaluated the effects of PASS in Chinese hospitals were electronically searched. Meta-analysis was used to assess the outcomes of intervention effects. Results: Twelve articles that appraised the interventions related to PASS were identified. All studies used before-after designs. Risk ratio was chosen as the effect size index. Heterogeneity was large (I² > 50%) in all main analyses. Effect sizes were pooled using random effects. Compared with the control group, the pooled effect sizes were 0.60 (95% CI, 0.52-0.70) for the outcome of the incident of inpatient irrational drug use (n = 5), 0.75 (95% CI, 0.59-0.96) for the outcome of the incident of outpatient patient irrational drug use (n = 2), and 0.29 (95% CI, 0.10-0.86) for the outcome of the incidence of black-light warning orders (n = 6). Conclusions: Interventions related to PASS was associated with the reduction of irrational drug use and medication errors.

Keywords

prescription automatic screening system evaluation rational use of drugs meta-analysis China

Introduction

Irrational drug use is a very serious problem in China.¹ Irrational drug use has been reported to account for 12% to 32% of drug usage in China.¹ Among the 6 million disabled persons in China, about a third of them have hearing disabilities, in which 60% to 80% were caused by poisoning through the irrational use of streptomycin, kanamycin, and gentamycin.¹ Exploring the potential of digital solutions to enhance the quality and safety of health care is of international interest, and the implementation of eHealth technologies is diffused underway.² For example, computerized physician order entry (CPOE) systems are widely referred to a variety of computer-based systems that share common features in automating the clinical ordering process to ensure standardized, legible, and complete orders.³ Moreover, CPOE in hospitals was reported able to reduce medication errors, such as adverse drug events, and improve safety during the medication process.⁴

Since 2002, a computer-based system named Prescription Automatic Screening System (PASS) has been used in many general hospitals in China. PASS is a special software system designed for prescription detection, drug information query, medication education, and medical record query, and it has serviced pharmacists and prescribers with drug alerts.⁵ These alerts often include drug interactions, repeated drug use, adverse drug reaction (ADR), in-vitro incompatibility of injections, drug-induced disorders, contraindications, and so on.⁵

PASS offers a database of drug information, including drug usage, dosage, ADR, drug allergy, incompatibility, and so on.^{6

–17} Through the analysis of the drug information and patient medical information, PASS examines the rationality of medical orders and shows the monitoring results by warning information.^{6

–17} Different levels of warning information are displayed in lights in different colors. For example, black light usually expresses absolutely forbidden order. In this way, PASS reminds medical personnel the potential problems of drug use.

Since PASS is widely used in China, systematical evaluation of intervention effects is valuable and useful for the development and testing of the system or other new rational use of drugs (RUD) software systems. A number of PASS evaluations have been conducted in China to date. Many studies focused on PASS monitoring, and a number of studies assessed the impact of PASS with respect to a variety of parameters, including irrational drug use, medication errors, costs of care, and ADR.^{6–

17} Moreover, some studies suggested that interventions related to PASS might impact drug use in certain hospitals.^{6–

17} However, these studies lack systematic evidence-basic assessment of PASS, and little is currently known about the intervention effects on RUD by means of systematic review. Therefore, the effectivity of computer-based system in contributing to the improvement of rational drug use cannot be concluded.

A systematic review on evaluating the effectiveness and consequences of the interventions of the PASS system on RUD was conducted to inform policy decisions on PASS deployments.

Methods

Search Strategy

In this study, relevant Chinese publications were searched based on the databases of Wanfang (1982 to April 2011), CNKI (1979 to April 2011), VIP (1989 to April 2011), and Chinese Selected Doctoral Dissertations and Master’s Theses Full-Text Databases (1990 to 2010). Keywords used for the search included rational drug use, PASS, drug use, and drug management combined with software, drug, medicine, appraise, monitoring, management, and strategy. Grey literature was searched after identifying key websites and search engines, such as Google and Intute. Reference lists of all papers and relevant reviews were screened for any further published or unpublished work.

Selection Criteria

All study designs, except qualitative ones, were included. The selected studies were before-after or randomized controlled study designs. Studies that examined interventions that intended to apply PASS for improving the level of RUD in Chinese hospitals were included. Studies on intervention assessment in relation to any individual patient were excluded. Cross-sectional studies were excluded because their focus was solely on evaluating the effects of PASS on intervention. Studies with repeated use of data from the same hospital were excluded. Interventions outside mainland China were not considered.

Data Collection

Standardized data abstraction forms were developed based on the Cochrane Effective Practice and Organization of Care (EPOC) Group data abstraction form¹⁸ and pilot tested by the review team, who used the protocol to guide primary and secondary outcomes.¹⁹ The following information was extracted from each study: hospital, participant, timing of the intervention, duration of the intervention, format of the intervention, effects of the intervention, nature of the intervention, number of components included in the intervention, source and authors of the intervention, reliability and validity testing of outcome measurement tools, and adherence (eg, withdrawals and dropouts).

Two independent reviewers assessed the methodological quality of all studies that were included in data abstraction. Consensus was applied to resolve discrepancies between reviewers. Authors were contacted for missing data or clarifications.

Data Analysis

Four dichotomous outcomes, including the incidence of inpatient irrational drug use, the incidence of outpatient irrational drug use, the incidence of black-light warning orders, and the correction ratio of black-light warning orders were included in the meta-analysis. Risk ratio (RR) was selected as the effect magnitude.

Meta-analysis was performed using RevMan5.0. Fixed or random effect model was used in meta-analysis. Tests for heterogeneity were performed. A P-value lower than 0.10 provided evidence of heterogeneity of intervention effects, while an I-squared value greater than 50% was considered to be substantially heterogeneous.²⁰

Results

Initial searches of electronic databases identified 20,403 records. After removing duplicates, 19,281 records were examined to determine potential relevance. By screening the titles and abstracts, 151 articles were identified to be related to PASS, and 84 full-text articles were retrieved and screened. After screening all full-text studies, 69 articles were excluded because of lack of relevant intervention or evaluation on intervention effects. Three articles were suspected to have submitted a piece of writing for multiple publications, two of which were excluded. One article without the number of medical orders was excluded. Finally, 12 articles with application and appraisal of the intervention related to PASS were included for detailed analysis, as shown in Figure 1.

Figure 1.

Flow diagram of systematic review to identify eligible studies.

All studies were focused on appraising interventions to PASS. All surveys took place in Chinese hospitals.

Description of Studies

All 12 studies used before-after designs, and no randomized controlled trial was found.^{6–

17} The definition of RUD was consistent across all articles. In all studies, RUD was defined as safe, effective, and appropriate for drug use, such as correct drugs; appropriate dose, administration route, and treatment course; no contraindication, incompatibility, and serious ADR, among others.

In all articles, lights in five different colors meant five grade levels of warning information of drug use, where black light expressed absolutely forbidden order, red light expressed serious alert, yellow light expressed general attention, green light expressed slight damage, and blue light expressed no problem.

Interventions related to PASS were implemented in general hospitals in Beijing, Shanghai, Guangzhou, Chongqing, Dalian, Fuzhou, and Chengdu in China. Most studies focused on inpatient medical orders, among which only two studies were related to outpatient prescriptions. Intervention time was from 2002 to 2009.

Four intervention methods, including installing PASS, administrative interventions, comprehensive interventions, and a comparison between technology and comprehensive interventions, were used in the studies. Administrative interventions included publishing monitoring results and linking monitoring results with performance evaluation. Technology interventions from clinical pharmacists included communicating warning information with prescribers, analyzing the reasons for medical errors, and performing education for warning information of high frequency. Comprehensive interventions included simultaneous technology and administrative interventions in addition to installing PASS. Installing PASS was the most frequently used intervention method in the studies.

Meta-Analysis

Outcome of the incidence of inpatient irrational drug use

Five studies provided data on the incidence of irrational drug use. Heterogeneity test indicated P < 0.00001 and I-squared = 88%. Random effect model was used in meta-analysis. The pooled RR was 0.60 (95% CI, 0.52-0.70), indicating that the risk of inpatient irrational drug use of the intervention group was about 60% of the control group. Compared with the control group, the interventions related to PASS reduced the risk of inpatient irrational drug use by 40%. Significant difference was found between the intervention and control groups (Z = 6.42, P < 0.00001), as shown in Figure 2.

Figure 2.

The forest plot of the incidence of inpatient irrational drug use.

Subgroup analysis was conducted according to different intervention strategies, and the subgroup of installing PASS was not heterogeneous (P = 0.81, I-squared = 0%), as shown in Table 1. Subgroup analysis indicated that a comparison between technology and comprehensive intervention were associated with larger improvements than installing PASS interventions in this outcome.

Table 1.

Subgroup analysis for the incidence of inpatient irrational drug use.

Subgroup	Studies	Participants	Heterogeneity	Effect Estimate	Z	P
Installing Prescription Automatic Screening System (PASS)	3	85,984	P = 0.81, I-squared = 0%	0.60 (0.57, 0.64)	17.33	<0.00001
Technology versus comprehensive intervention	1	46,204	Not applicable	0.41 (0.32, 0.54)	6.74	<0.00001
Special sampling method firstly extracting black warning orders	1	6676	Not applicable	0.75 (0.69, 0.81)	6.99	<0.00001

Outcome of the incidence of outpatient irrational drug use

The two studies involved in this outcome all employed the installing PASS interventions. Heterogeneity test indicated P < 0.00001 and I-squared = 96%. Random effect model was used in meta-analysis. The pooled RR was 0.75 (95% CI, 0.59-0.96), indicating that the risk of the outpatient irrational drug use of intervention group was about 75% of the control group. Compared with the control group, the interventions related to PASS reduced the risk of outpatient irrational prescriptions by 25%. Significant difference was found between the intervention and control groups (Z = 2.27, P = 0.02), as shown in Figure 3.

Figure 3.

The forest plot of the incidence of outpatient irrational drug use.

Outcome of the incidence of black-light warning orders

Six studies provided data on the incidence of black-light warning orders. Heterogeneity test indicated P < 0.00001 and I-squared = 100%. Random effect model was used in meta-analysis. The pooled RR was 0.29 (95% CI, 0.10-0.86), indicating that the risk of the black-light warning orders of intervention group was about 29% of the control group. Compared with the control group, the interventions related to PASS reduced the risk of black-light warning orders by 71%. Significant difference was found between the intervention and control groups (Z = 2.24, P = 0.03), as shown in Figure 4.

Figure 4.

The forest plot of the incidence of black-light warning orders.

Subgroup analysis was conducted according to different intervention methods. Heterogeneity was substantial or not applicable in the subgroups, as shown in Table 2. Subgroup analysis indicated that comprehensive interventions were associated with larger improvements than other three intervention methods in this outcome.

Table 2.

Subgroup analysis for the incidence of black-light warning orders.

Subgroup	Studies	Participants	Heterogeneity	Effect Estimate	Z	P
Installing Prescription Automatic Screening System (PASS)	3	391,105	P < 0.00001. I-squared = 90%	0.25 (0.06, 0.99)	1.97	0.05
Administrative intervention	1	1,000,000	Not applicable	0.93 (0.89, 0.97)	3.57	0.0004
Comprehensive intervention	1	3,262,863	Not applicable	0.14 (0.14, 0.15)	118.97	<0.00001
Technology versus comprehensive intervention	1	46,204	Not applicable	0.36 (0.23, 0.55)	4.63	<0.00001

Outcome of the correction ratio of black-light warning orders

Four studies provided data on the correction ratio of black-light warning orders. Heterogeneity test indicated P < 0.00001 and I-squared = 91%. Random effect model was used in meta-analysis. The pooled RR was 1.05 (95% CI, 0.85-1.29), indicating that the probability of the correction of black-light warning of intervention group was about 105% of the control group. Compared with the control group, the interventions related to PASS increased the probability of the correction of black-light warning by 5%. However, no significant difference was found between the intervention and control groups (Z = 0.42, P = 0.67), as shown in Figure 5. Two studies using administrative interventions and a comparison between technology and comprehensive interventions showed improvements in this outcome, while two other studies showed negative effect.

Figure 5.

The forest plot of the correction ratio of black-light warning orders.

Discussion

Principal Findings

Relevant research articles were indentified after conducting an extensive review. Most studies in the review were focused on the rates of medication errors and error correction.

Four important and representative outcomes, including the incidence of inpatient irrational drug use, the incidence of outpatient irrational drug use, the incidence of black-light warning orders, and the correction ratio of black-light warning orders, were used to measure the effects of the interventions in the meta-analysis. The first two outcomes measured the overall condition of irrational drug use, the third outcome evaluated the status of the most serious irrational drug use, and the last outcome appraised the changes in prescription behaviors. Moreover, the four outcomes were defined similarly in all studies.

There were warning lights in five colors in the studies. However, warning orders in other colors besides black were not included in the meta-analysis because they were only measured in few studies.

The data types for the outcome measurements were identified prior to meta-analysis. The four outcomes previously described were dichotomous, and RR was selected as the effect magnitude in the meta-analysis.

Interventions related to PASS was associated with better outcomes about the incidence of irrational drug use and black-light warning orders. However, the results of the meta-analysis should be interpreted with caution because of the high heterogeneity.

Subgroup analyses exploring intervention method differences failed to explain the heterogeneity. Subgroup analysis about inpatient irrational drug use indicated that the heterogeneity in installing PASS subgroup was not statistically significant. However, subgroup analysis about black-light warning orders indicated that the heterogeneity in subgroups was still statistically significant. Moreover, subgroup analysis was not performed in other two outcomes because of the inadequate number of studies. Therefore, the subgroup factor of intervention methods could not completely explain the source of heterogeneity.

The impact of the four intervention methods on the four respective outcomes was different.

Subgroup analysis indicated that comprehensive interventions and a comparison between technology and comprehensive interventions were associated with larger improvements in two outcomes, respectively. However, the subgroup analyses should be interpreted with caution because of the limitations associated with between-study (rather than within-study) comparisons.

There was insufficient evidence to draw conclusions about the impact of PASS on modifying prescriptions. The effect of the interventions was invalid on improving the correction ratio of black-light warning orders in the meta-analysis. This mechanism may be explained by the unwillingness of prescribers to cease drugs that appear to be working and importantly do no harm, and patients are unlikely to request drug stoppage.²⁰ Moreover, reminders for this kind of clinical review may be perceived as undermining professional autonomy.²⁰

Strengths and Weaknesses

To the best of our knowledge, this study is the first to systematically review and assess the impacts of PASS on medication error prevention and RUD improvement in China.

No prior review was published on the evaluation of PASS. However, some related reviews on the evaluation of similar software, such as CPOE in the United States, the European Union, Brazil, Canada, and Australia, were published.^{20–

24} A 2008 review reported that the impact of CPOE systems was positive on the category that adheres to guidelines, alerts, costs, organizational efficiency, satisfaction, and usability.²³ Assessment on the effects of information technology in health care is inherently difficult because the technology is only part of a much larger and complex social system.²³ Although the manuscript may not have included more studies compared to reviews about PASS evaluation, this study is a useful attempt for assessing information technology in health care quantitatively via meta-analysis. Furthermore, this study is a meaningful methodology exploration of medical information technology assessment.

The findings of this study must be interpreted in light of its limitations. The conclusions of this review should be interpreted with caution because of the heterogeneity. The high inconsistency among studies reflected variation not only in intervention methods but also in intervention times, research methods, sampling methods, and some other factors. Because the overwhelming majority of PASS interventions were associated with benefit, this heterogeneity could be interpreted as supporting the effectiveness of PASS across a certain range.

Before-after studies in meta-analysis were included because no randomized controlled trial that addresses the topic of the review was identified. While randomized controlled studies are considered as gold standards in study design,²⁴ they are difficult to conduct in the context of PASS implementation in a hospital, and limiting the use of intervention to specific patients is often impossible. A clinical trial, in which patients receive PASS randomly, would be costly and methodologically difficult to implement. Results of the before-after studies should be treated with caution. Before-after studies are often confounded by some confounding factors and impacted by the regression to mean, and they tend to overestimate the effects of intervention.²⁵ Therefore, the possibility of selection biases cannot be excluded, and some confounding factors may have been included in the review.

Inferences are limited by the quantity of available studies. For example, there were only two studies about outpatient, and the results of the outpatient income possibly had potential bias. Funnel plot was not used to indentify bias in the review because the number of studies of every outcome was less than 10.²⁵ Therefore, the possibility of reporting biases cannot be totally excluded.

Directions for Future Research

In light of PASS implementation, promoting the testing of PASS systems is important for producing high-quality evidence about their impacts. However, the number of PASS interventions in this study and the understanding on the impacts of PASS remain relatively limited. Furthermore, interventions to date have largely focused on warning information monitoring rather than on other outcomes. Therefore, more studies on the impact of PASS in hospitals, with respect to ADR, safety, quality, cost, ceasing therapy, and patient outcomes, are essential.

All articles in this study used before-after designs, and no randomized controlled trial was found. Therefore, more effort should not only be devoted for evaluation, but also for ensuring that the methodology adopted is multidisciplinary to untie the complex web of factors that may influence the results. Moreover, a fuller description of the fundamental for the choice of methodology employed to evaluate PASS would promote synthesis and comparison. Our understanding of the intervention effects of PASS would be much improved using more detailed reports on the systems, the particulars of environment in which they were arranged, and the manner by which they were developed.

Conclusions

Interventions related to PASS were associated with the reduction of irrational drug use and medication errors in the system review. No firm evidence that identifies the effects of PASS on modifying prescriptions was found. Interventions have largely focused on warning information monitoring rather than on other outcomes. Thus, more studies on the impact of PASS on ADR, safety, quality, cost, and patient outcome are necessary. No randomized controlled trial related to the evaluation of PASS was found, and the standard randomized controlled study was methodologically excellent for studying system or clinical performance. These findings call upon rational choice of methodological approaches for evaluating PASS.

Footnotes

Acknowledgments

We thank Difei Wang of HuaZhong University of Science and Technology for his comments about the grammar and writing revision.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded by the National Natural Science Foundation of China (Grant No. 71173082).

References

Lijuan

Peimin

Discussion on irrational drug use and counter measures in China Chin Pharm . 2005;16:1204–1206.

Black

Car

Pagliari

. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8:1–16.

Rahimi

Timpka

Vimarlund

Uppugunduri

Svensson

. Organization-wide adoption of computerized provider order entry systems: a study based on diffusion of innovations theory. BMC Med Inform Decis Mak. 2009;9:52.

Riedmann

Jung

Hackl

Stühlinger

van der Sijs

Ammenwerth

. Development of a context model to prioritize drug safety alerts in CPOE systems. BMC Med Inform Decis Mak. 2011;11:35.

Zhangwei

Jinhong

Gongcheng

. The impact of clinical drug use decision-making system on rational drug use and the economic assessment methods. Chin Pharm. 2006;17:385–387.

Chaoyan

Hai

Xiu

. Retrospective analysis of monitoring medication by using PASS. Guide Chin Med. 2009;7:34–35.

Qing

Jin

Lixian

. Comprehensive interventions to improve rational administration in medical orders for inpatients. Mil Med J S Chin. 2010;24:154–155.

Jin

Lixian

Lei

. Investigation and intervention study on monitoring medication by using PASS in our hospital. Guangdong Med J. 2006;27:1077–1078.

Xiuqin

Qiuyu

Yanchun

. Monitoring and analysis on medication orders for inpatients by using PASS. Chin J Drug Appl Monit. 2011;8:52–54.

10.

Beijiang

Fenfang

Shiping

. Using clinical rational drug use monitor systems and improving the level of rational drug use. Nurs J Chin PLA. 2005;22:72–73.

11.

Man

Daihong

Guiyang

. Evaluation of prescription automatic screening system in our hospital. Chin Pharm. 2010;19:41–42.

12.

Bin

Maobo

Xiaofang

. Monitoring and analysis of hospital medication orders using PASS. Strait Pharm J. 2010;22:200–202.

13.

Guiru

. Analysis on irrational drug use of outpatient prescription. Pharm Care Res. 2007;7:119–123.

14.

Xiuyin

Guanghui

. Development and application of prescription supervisory system for outpatient doctors work station. Chin J Pharmacovigilance. 2010;7:91–94.

15.

Yan

Juying

Peiyuan

. PASS system combining administrative intervention in promoting rational use of drugs in our hospital. Chin Pharm. 2008;19:2313–2314.

16.

Renjie

Qin

Huiqing

. Factors influencing clinical intervention of prescription automatic screening system. Pharm Care Res. 2004;4:197–200.

17.

Jianli

Xiaobo

. Intervention effect of Prescription Automatic Screening System in elderly patients. Chin J Mult Organ Dis Elderly. 2010;9:114–122.

18.

Cochrane Effective Practice and Organization of Care Group: Data Abstraction form. 2011. http://epoc.cochrane.org/epoc-resources-review-authors. Accessed September 18, 2011.

19.

Perrier

Mrklas

Lavis

Straus

. Interventions encouraging the use of systematic reviews by health policymakers and managers: a systematic review. Implement Sci. 2011;6:43.

20.

Reckmann

Westbrook

Koh

Day

. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc. 2009;16:613–623.

21.

Higgins

JPT

Green

. Cochrane handbook for systematic reviews of interventions version 5.1.0. 2011. http://www.cochrane-handbook.org/. Accessed September 18, 2011.

22.

Pearson

Moxey

Robertson

. Do computerized clinical decision support systems for prescribing change practice? A systematic review of the literature (1990-2007). BMC Health Serv Res. 2009;9:154.

23.

Chaudhry

Wang

. Systematic review: impact of health information technology on quality, efficiency, and cost of medical care. Ann Int Med. 2006;144:12–22.

24.

van der Sijs

Aarts

Vulto

Berg

. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13:138–147.

25.

Eslami

de Keizer

Abu-Hanna

. The impact of computerized physician medication order entry in hospitalized patients—a systematic review. Int J Med Inform. 2008;77:365–376.