Abstract
Multiple methods for detecting asthma encounters are used today in public surveillance, quality reporting, and clinical research. Failure to detect asthma encounters can make it difficult to measure the scope and effectiveness of hospital or community-based interventions important in comparative effectiveness research and accountable care. Given the pairing of asthma with certain respiratory conditions, the objective of this study was to develop and test an asthma detection algorithm with specificity and sensitivity using 2 criteria: (1) principal discharge diagnosis and (2) asthma diagnosis code position. A medical record review was conducted (n=191) as the gold standard for identifying asthma encounters given objective criteria. The study team observed that for certain principal respiratory diagnoses (n=110), the observed odds ratio that encounters were for asthma when asthma was coded in the second or third code position was not significantly different than when asthma was coded as the principal diagnosis, 0.36 (P=0.42) and 0.18 (P=0.14), respectively. In contrast, the observed odds ratio was significantly different when asthma was coded in the fourth or fifth positions (P<.001). This difference remained after adjusting for covariates. Including encounters with asthma in 1 of the 3 first positions increased the detection sensitivity to 0.84 [95% confidence interval (CI): 0.76–0.92] while increasing the false positive rate to 0.19 [95% CI: 0.07–0.31]. Use of the proposed algorithm significantly improved the reporting accuracy [0.83 95%CI:0.76–0.90] over use of (1) the principal diagnosis alone [0.55 95% CI:0.46–0.64] or (2) all encounters with asthma 0.66 [95% CI:0.57–0.75]. Bed days resulting from asthma encounters increased 64% over use of the principal diagnosis alone. Given these findings, an algorithm using certain respiratory principal diagnoses and asthma diagnosis code position can reliably improve asthma encounter detection for population-based health impact measurement. (Population Health Management 2014;17:239–246)
Introduction
I
Several methods are employed today in public surveillance, quality reporting, and research to identify asthma inpatient encounters using administrative data. In the United States, for example, the National Hospital Discharge Survey, published by the Centers for Disease Control and Prevention, uses the “first-listed diagnosis” to classify hospital discharges by condition for surveillance reporting. 3–4 For quality reporting, the Agency for Healthcare Research and Quality (AHRQ) Prevention Quality Indicators technical specifications outline asthma detection in young adults based on a principal asthma diagnosis (International Classification of Diseases, Ninth Revision [ICD-9]: 493.x). 5 The principal diagnosis is defined in the Uniform Hospital Discharge Data Set as “that condition established after study to be chiefly responsible for occasioning the admission of the patient to the hospital for care.” 6 In a survey of asthma research studies over the past 5 years, the authors identified at least 4 common methods for inpatient encounter detection using administrative data. These methods included use of the ICD-9 principal diagnosis alone, 7 –9 use of the ICD-9 principal or secondary diagnosis alone, 10 –12 use of the ICD-9 principal and/or secondary diagnosis along with pharmacy claims for asthma medications, 13–14 and use of ICD-9 coding with diagnostic position unspecified. 15 –17 Each of these methods introduces an uncertain level of information bias or risk into reported results.
One alternative encounter detection method that has not been examined is the potential use of the asthma diagnosis code position to detect and classify probable inpatient encounters of asthma from administrative data. Given the common pairing of asthma with certain respiratory conditions, the study team's hypothesis is that the asthma diagnosis code position, when coupled with select respiratory conditions, may offer predictive value in identifying inpatient asthma encounters. The study further suggests a novel application of receiver operating characteristic (ROC) analysis to assist users in identifying positivity criterion for asthma encounter selection given their particular information needs.
Methods
The objective of this research study was to develop and test an asthma detection algorithm using 2 primary criteria: (1) principal discharge diagnosis and (2) asthma diagnosis code position. The principal or primary discharge inpatient diagnosis (“principal diagnosis”) was defined as the ICD-9 diagnosis coded in the first diagnosis code position and chiefly responsible for the hospital admission. The asthma diagnosis code position (“code position”) was defined by ordinal rank, beginning with the principal diagnosis as the first position, and so on, up to 5 diagnosis positions. ICD-9 was used for discharge diagnosis coding purposes; the diagnosis count was restricted to the first 5 sequential diagnoses for simplicity.
To conduct the study, the study team used a population of patients from a large, urban pediatric hospital. Children's Hospitals and Clinics of Minnesota (CHC MN) acts as a safety net hospital serving the needs of urban and suburban children from multiple racial and ethnic backgrounds in the Twin Cities. Data were extracted from the electronic medical record and from an administrative data warehouse for all inpatient discharges occurring in 2011. The final sample contained 17,141 hospital discharges, including both inpatient and observation stays, with up to 5 diagnosis codes for each discharge record. Additional demographic information for each discharge also was captured.
The study team began by dividing the test population along 2 dimensions (presence of a principal respiratory diagnosis [y/n] and presence of asthma diagnosis in any code position [y/n]), classifying each population encounter into 1 of 4 classes. The study team assumed the first class of encounters (non-respiratory patients without any asthma diagnosis) would not be associated with an asthma encounter and excluded this class pending the results of testing the other classes. Given the results to follow, no further testing of this first class was conducted. The study team then made a random sample selection of patients from each of the 3 remaining classes (n=191). This sample was used to test the association between the principal diagnosis and the presence of an asthma encounter.
The study team developed objective criteria for identifying an asthma encounter through a literature review and consultation with a hospital pulmonologist at CHC MN. The team concluded that the presence of a significant number of asthma-related risk factors and asthma treatment protocols would reasonably suggest that asthma was a primary underlying factor in the admission. Criteria are noted in Table 1.
Using the criteria, the following 4 standards were developed for identifying an asthma encounter in the patient medical record. Discharges were identified as asthma encounters if (1) the physician noted explicitly in the medical record that asthma was the principal diagnosis or the principal ICD-9 coded diagnosis was asthma, and (2) the hospital stay met 3 of 4 primary criteria, as noted in Table 1. If the discharge met only 2 of the 4 primary criteria, the discharge was included as an asthma encounter if at least 3 secondary criteria also were met. If the physician did not make a principal diagnosis of asthma in the hospital record, then the discharge was identified as an asthma encounter if the discharge met all 4 primary criteria. If the physician specifically ruled out asthma as a diagnosis, the discharge was not an asthma encounter.
Data based on these criteria were then extracted from the patient medical record for the study sample of 191 discharges. These data were used to evaluate each discharge record in the sample relative to the defined classification standards described. To ensure correct application of these standards, the hospital pulmonologist reviewed a subset of these evaluations.
Based on these initial evaluations, the study team determined that certain discharge diagnoses were not significantly associated with the presence of an asthma encounter (see Results section). Therefore, the team excluded discharges with these diagnoses (n=81) from the final sample. The final sample (n=110) was then used to evaluate the impact of the code position on the presence of an asthma encounter.
After evaluating the utility of code position in identifying discharges as asthma cases, the study team used the results to develop a ROC curve by code position. A ROC curve includes a “plot of pairs of possible combinations of true-positive and false-positive ratios achievable with a test as the positivity criterion is varied.” 18 ROC analysis is commonly applied in medical decision making in testing and diagnosis to measure the risk of accurate disease detection using a particular diagnostic testing method; it also is used to compare the accuracy of predictive models. 19 In this example, use of the ROC analysis allows the comparison of the true positive (sensitivity) false positive ratios (1-specificity) and the overall accuracy of a proposed asthma encounter detection algorithm (“algorithm”) against alternate asthma encounter detection methods using (1) the primary diagnosis alone or (2) any primary or secondary diagnosis. Based on the findings, the study team finalized criteria for an algorithm using an optimal combination of the specific respiratory conditions and the asthma code positions.
To test the algorithm, the study team made a random selection of discharges (n=20) from 2009 respiratory discharges coded with a principal or secondary diagnosis of asthma. These data were used to evaluate each discharge record in the sample relative to the defined classification standards already described. The results of the medical record review were then compared with the results of the algorithm.
Logistic regression was used to test the association of code position with the dichotomous outcome asthma encounter (y/n). Additional covariate analysis was conducted to assess the impact of other factors on the presence of an asthma encounter as appropriate. Model testing included goodness of fit, area under the curve, and variance inflation factor testing. Final statistical analysis was conducted using STATA, version 13.0 (StataCorp LP, College Station, TX). Study approval was granted under CHC MN institutional review board # 1207-067 and University of Minnesota institutional review board # 1209E20042.
Results
A description of the total population of discharges from CHC MN in 2011 and the study sample are presented in Table 2. A total of 1939 encounters (11.3%) included an asthma diagnosis code in one of the first 5 code positions. Of these encounters, 680 (35.1%) had a principal diagnosis of asthma.
CHC MN, Children's Hospitals and Clinics of Minnesota; RSV, respiratory syncytial virus.
None of the asthma discharges included with a non-respiratory principal diagnosis was identified as an asthma encounter. Similarly, none of the respiratory encounters tested without an asthma diagnosis code was identified as an asthma-related hospitalization. Both classes was excluded from further testing. Of the remaining study sample (n=150), 77 encounters (51.0%) were identified as asthma encounters through medical record review. The majority of these (89.3%) were identified as asthma encounters based on (1) physician diagnosis of asthma and (2) meeting 3 of 4 primary criteria outlined in Table 1.
Further testing the study sample (n=150) by respiratory condition, asthma (493.x), pneumonia (480.x–487.x), respiratory failure (581.x), and respiratory syncytial virus (RSV)/bronchiolitis (466.x) had a meaningful association with detection of an asthma encounter as noted in Figure 1. Excluding the remaining cases that were not significantly associated with asthma, and excluding RSV discharges younger than 24 months given the specificity of the RSV diagnosis, the study team identified the final study sample of respiratory conditions significantly associated with asthma encounter detection (n=110). Excluding the additional cases not significantly associated with an asthma encounter, the asthma encounter detection sensitivity increased from 51.3% (77/150) to 66.3% (73/110).

Asthma encounter by principal respiratory diagnosis (95% CI; n=150). CI, confidence interval; RSV, respiratory syncytial virus.
Using the final study sample to test the association between the presence of an asthma encounter by asthma code position and using the principal diagnosis as the reference category, the observed odds were lower as the diagnosis code position increased as noted in Figure 2. The observed odds ratios in the second and third code positions were lower than but not significantly different from the principal diagnosis. In contrast, a significant difference was observed between the reference category and the odds that a discharge is an asthma encounter in the fourth or fifth position, as noted in Table 3. This significant difference in the fourth and fifth code position remained after adjusting for covariates including patient age, race, sex, condition count, presence of a procedure, season, and condition severity.

Observed probability of true asthma-related hospitalization by asthma diagnosis code position (95% CI; n=110). CI, confidence interval.
Reference category is asthma coded in the first ICD-9 code position.
Significant at P<.05.
CI, confidence interval; ICD-9, International Classification of Diseases, Ninth Revision.
The final summarized ROC curve analysis is included in Figure 3 along with sensitivity and specificity readings by code position in Table 4. Including encounters with asthma in 1 of the first 3 positions increased the sensitivity to 0.84 [95% confidence interval (CI): 0.76–0.92] while increasing the false positive rate to 0.19 [95% CI: 0.07–0.31].

Receiver operating characteristic curve by asthma ICD-9 diagnostic code position (1–5) for a sample of select respiratory encounters (n=110), including 95% CI for both sensitivity and false positive rate. CI, confidence interval; ICD-9, International Classification of Diseases, Ninth Revision.
CI, confidence interval; ICD-9, International Classification of Diseases, Ninth Revision.
The accuracy rate is defined as the combination of true positive (TP) and true negative (TN) counts divided by the total sample count (Total) ((TP+TN)/Total). The overall reporting accuracy increases from 55% [95% CI: 0.46–0.64] using the principal diagnosis only and peaks when using code positions 1–3 for asthma encounter detection at 0.83 [95% CI: 0.76–0.90] before falling to 0.66 [95% CI: 0.57–0.75] when including encounters coded with asthma in any one of the first 5 diagnostic positions.
Asthma-related hospital bed days using the principal diagnosis alone were 168 for this study sample, increasing by 432 when including encounters in all 5 code positions to 600. When applied using the first 3 code positions, total hospital days were 304, an increase of 136 days over use of the principal diagnosis alone.
Based on the study findings, the recommended asthma case detection algorithm using select respiratory conditions classified by asthma code position is noted as follows: • Any encounter in which asthma is the first ICD-9 diagnosis code (ICD-9 493.xx) • Any pneumonia/influenza (ICD-9 480.xx–487.xx) or respiratory failure encounters (ICD-9 518.x) in which asthma is the second or third ICD-9 diagnosis code • Any RSV/bronchiolitis encounter (ICD-9 466.xx) in which asthma is the second or third ICD-9 diagnosis code and the child is older than 24 months
Applying the algorithm to a test sample (n=20) from 2009, the study team observed an accuracy level of 0.80 with a sensitivity of 0.80 and a false-positive rate of 0.10. These observed results were within the 95% confidence range of our initial findings.
Applying the algorithm to the full 2012 discharge population increased the encounter count in the population being studied from 680 discharges identified as asthma encounters using asthma in the first coding position as the basis for classifying cases to 1056 encounters using the algorithm, a 55% increase. Bed days in the cohort population increased 64%, the increase associated with a higher mean length of stay in the incremental encounters.
Discussion
When stratified by select respiratory conditions, asthma diagnosis code position provides predictive value in identifying probable asthma encounters in pediatric populations. Several factors may explain the observed association between asthma diagnosis code position and the likelihood of an asthma encounter.
Asthma is closely associated with other respiratory conditions, including human rhinovirus, RSV, bronchiolitis, upper-respiratory tract infections, pneumonia, and respiratory failure. 20 –23 This close association as well as the common presence of transient wheezing can make differential diagnosis of asthma difficult, particularly in children. 23–24 Physicians may be hesitant to formally diagnose asthma in acute pediatric care settings without longitudinal history and follow-up. 23,25,26 The results of the present study included 1 discharge that was coded, probably in error, with a principal diagnosis of asthma, but was not identified as an asthma encounter by the algorithm. The patient did not meet the criteria for an asthma encounter given that the patient had no history of asthma and the physician did not formally diagnose asthma in the medical record.
Inpatient coding policy is generally structured to identify the most acute conditions or resource-intensive activities as the principal diagnosis. 6 The correct sequencing of hospital codes ensures that hospitals optimize reimbursement levels and cover the costs of services provided. 6 When more resource-intensive comorbidities are present, the more resource-intensive conditions will generally be coded as the primary diagnosis, with asthma coded as a later diagnosis. 27 The results of the present study confirmed this pattern. Analyzing asthma by encounter using the All Patient Refined Diagnosis-Related Groups (APR-DRG, v.24) severity rating as noted in Figure 4, the study team observed that 77% of encounters with a primary diagnosis of asthma were given a mild or moderate severity rating. In contrast, almost 80% of encounters with asthma in the fifth diagnosis position were rated major or severe. When asthma was coded in a later code position (fourth or fifth) the team also observed the presence of additional severe non-respiratory conditions associated with the hospitalization (eg, polyneuropathy, chromosomal anomalies, depression).

Asthma diagnostic code position by APR-DRG severity level (n=110). APR-DRG, All Patient Refined Diagnosis-Related Group.
The relationship between asthma code position, encounter severity, and coded condition count is interesting. Asthma as a principal diagnosis is a less severe condition than other pediatric conditions. The presence of comorbidities will generally increase the coded condition count and overall severity of the encounter. When comorbid conditions are present with asthma, they are often more severe and are coded earlier in the coding sequence than asthma. As the asthma code position moves further away from the first code position, the study team found that it signals not only that more conditions are present, but that the underlying encounter generally is more severe and that asthma is increasingly incidental to the reason for the hospital encounter.
Defining asthma encounters using the algorithm has potential decision value over the use of (1) asthma principal diagnosis or (2) asthma principal or secondary diagnosis. This finding has important implications for accountable care organizations managing pediatric patient populations. The algorithm offers a reasonable, yet simple “middle ground” for classification of asthma encounters that is more complete while limiting the impact of false positive readings in analysis. In the hospital population identified for this study, use of the suggested algorithm for encounter detection over the AHRQ definition of asthma patients, for example, increased encounter detection by 55% and hospital days by 64% over the use of the principal asthma diagnosis alone. In a related readmission study using a similar detection method, the additional encounters detected were more severe with increased length of stay. 28 Using estimated, undiscounted discharge estimates, the increased volume and complexity captured significantly increased the hospital financial burden associated with asthma encounters in the cohort from $8.6 million to $14.8 million (71%) over asthma case detection using the principal diagnosis alone.
Similarly, use of the algorithm meaningfully lowered the risk of false positive readings through asthma encounter detection using any primary or secondary diagnosis of asthma. The results revealed that using the more expansive definition, including counting all primary and secondary asthma diagnoses in the first 5 code positions, leads to significantly more false positive readings that can skew reported operating metrics such as bed days, length of stay, and cost per encounter. Use of both methods for the purposes of detecting asthma encounters should be approached cautiously and the inherent information risks should be well understood.
Some limitations exist with this study. Though guidelines exist that direct coding practices, coding may vary somewhat by individual hospital or health system. Evidence suggests that compliance with coding guidelines is improving more generally, though challenges remain. 29 –34 The study team does think that efforts to appropriately optimize reimbursements move hospitals to a set of generally common practices that encourage consistency. This consistency reinforces the potential general applicability of this algorithm in pediatric inpatient settings. Other factors not considered in this study, including potential variation in hospital and individual coder practices, may impact results. The team attempted to moderate this risk by testing the results of the algorithm in a different year. Similar results were observed when applying the algorithm to the earlier period.
Given that asthma is often classified as a more mild condition with lower reimbursement, some risk may exist that coding teams may be more inclined to code asthma in a later code position to maximize recoveries. This was not the case in the present study. Rather the study team found that asthma encounters were more likely to be coded as asthma or a closely related respiratory condition, such as respiratory failure, pneumonia, or RSV/bronchiolitis. Further study is needed to assess the use of this algorithm to detect asthma encounters in different health systems and geographies where coding practices may vary. There may be benefit as well from further research examining similar algorithms with other chronic conditions.
Although medical record review is considered a “gold standard” for identifying diagnoses, some variation has been demonstrated between the physician's diagnosis per the medical record and the underlying presence of a given condition. 35,36 In this particular study, the focus was on measuring the nature of the encounter, based on the presence of asthma risk factors and the treatment protocols administered by the medical team, an important distinction in managing patient populations under accountable care. This assumes that observed risk factors and meaningful activities related to actual treatment of asthma provide signals that a detected encounter is associated with asthma. Given the fact that the only encounters detected through this method include respiratory conditions with a diagnosis of asthma, the risk of detecting an asthma encounter that is not associated to some degree with asthma is very low. The risk this method introduces is the detection of an asthma encounter when asthma was actually incidental to the visit. The study team believes this risk is offset by the significantly improved case sensitivity essential for encounter analysis.
The US health care system is currently migrating to the International Classification of Diseases, Tenth Revision (ICD-10) in 2014, and that may impact the relevance of this particular algorithm. Although the study was conducted using ICD-9 coding, standard mapping of ICD-9 to ICD-10 coding exists that can be applied. Additionally, ICD-10 will continue to require the use of multiple diagnosis codes to capture comorbid conditions and treatments in a similar fashion to the approach taken with coding in ICD-9. Based on this, the migration to ICD-10 will require adaptation but should not meaningfully impact results.
Application of this algorithm can prove useful in both hospital and health plan settings, particularly when examining the impact of a specific asthma intervention on patient outcomes. Use of the algorithm for encounter detection may not be the best approach in all research or analytic settings. As the nature and type of information requests expand in health care, the study team believes that the need to differentiate acceptable levels of information bias and risk will increase in an effort to manage information production costs and increase speed to market. The use of ROC analysis can aid in identifying the appropriate positivity criterion, given the particular user's information requirements, and the level of acceptable information bias or risk in results using commonly understood terminology. This format may provide a useful communication tool for discussing the potential information risk associated with a given request for or presentation of health information. The study team recommends further study on methods for communicating these information risks in both internal and external reported health information to increase confidence in the reliability of information and in an effort to derive greater value from health information technology investments.
Conclusions
The results of the present study demonstrate that diagnosis code position, when coupled with certain respiratory conditions, has predictive value in detecting asthma encounters in children. Based on these findings, a fairly simple algorithm can be applied to administrative data to reliably increase the count of asthma encounters for population-based health management efforts and help us more accurately measure the burden of asthma encounters on hospital care.
Footnotes
Author Disclosure Statement
Drs. Flood and Payne, and Mr. Knighton, Mr. Harmon, Ms. Smith, and Ms. Crosby declared no conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors received no financial support for the research, authorship, and/or publication of this article.
Acknowledgments
The authors would like to acknowledge the contribution of Stephen Kuracheck, MD, for his insights into the identification of asthma encounters, including his expertise interpreting medical records. We also would like to thank Daniel E. Knighton, research assistant, for his assistance with aspects of the literature review and statistical analysis.
