PROPER: Development of an Early Pediatric Intensive Care Unit Readmission Risk Prediction Tool

Abstract

Objective:

No risk prediction model is currently available to measure patient’s probability for readmission to the pediatric intensive care unit (PICU). This retrospective case–control study was designed to assess the applicability of an adult risk prediction score (Stability and Workload Index for Transfer [SWIFT]) and to create a pediatric version (PRediction Of PICU Early Readmissions [PROPER]).

Design:

Eighty-six unplanned early (<48 hours) PICU readmissions from January 07, 2007, to June 30, 2014, were compared with 170 random controls. Patient- and disease-specific data and PICU workload factors were compared across the 2 groups. Factors statistically significant on multivariate analysis were included in the creation of the risk prediction model. The SWIFT scores were calculated for cases and controls and compared for validation.

Results:

Readmitted patients were younger, weighed less, and were more likely to be admitted from the emergency department. There were no differences in gender, race, or admission Pediatric Index of Mortality scores. A higher proportion of patients in the readmission group had a Pediatric Cerebral Performance Category in the moderate to severe disability category. Cases and controls did not differ with respect to staff workload at discharge or discharge day of the week; there was a much higher proportion of patients on supplemental oxygen in the readmission group. Only 2 of 5 categories in the SWIFT model were significantly different, and although the median SWIFT score was significantly higher in the readmissions group, the model discriminated poorly between cases and controls (area under the curve: 0.613). A 7-category PROPER score was created based on a multiple logistic regression model. Sensitivity of this model (score ≥12) for the detection of readmission was 81% with a positive predictive value of 0.50.

Conclusion:

We have created a preliminary model for predicting patients at risk of early readmissions to the PICU from the hospital floor. The SWIFT score is not applicable for predicting the risk for pediatric population.

Keywords

intensive care units pediatric patient readmission logistic models retrospective studies receiver operating characteristic (ROC) curve quality indicators health care

Introduction

In clinical practice, there is often lack of certainty on the safe timing of transferring the patient to the floor from the pediatric intensive care unit (PICU). These decisions are very often based on the workload of the staff and bed availability in the intensive care unit (ICU), which, although necessary for operational reasons, are not necessarily in the best interest of the patient. Although efforts have been made to standardize this process, most physicians rely on intuition and subjective clinical judgment to identify who can be transferred to the floor. Investigators, both in adult and in pediatric fields, have identified ICU readmissions to be associated with worse clinical outcomes, such as longer hospital stay, higher mortality, and increased health-care costs.^1,2 Previous studies have focused on 48 hours as early readmission.^2
–4 The reason for this emphasis on readmissions within 48 hours is based on the assumption that early readmissions elucidate the actions taken in the ICU and reflect upon the criteria for transferring out the patient.

From our institution, Gajic et al in 2008⁵ developed and validated a scoring tool (“the Stability and Workload Index for Transfer,” SWIFT) to assess such a risk in adult ICU patients. This score showed improved prediction of readmission over Acute Physiologic and Chronic Health Evaluation (APACHE) III scores on the day of discharge. The scoring tool has been validated for automatic calculation from the electronic medical records (EMRs) and has shown excellent correlation with manual data collection (r = .92).⁶ This score, however, performed poorly in predicting the risk of readmission in an independent, prospective implementation on 7175 ICU discharges.⁷

In line with the adult literature, patients readmitted to the PICU have been shown to have a longer median length of stay (3.1 vs 1.7 days, P < .0001) and higher mortality (4% vs 2.5%, P = .002), as compared to those who did not require readmission.¹ Although variables describing the risk factors have been previously described,⁸ the only available tool for the assessment of risk of early readmission to the PICU was developed by Linton et al³ from Melbourne, Australia. This tool was developed for the use of liaison nurses to assess the risk of readmission of patients already on the pediatric floor as opposed to at the time of PICU discharge. This limits its application as a decision support tool for PICU discharge. This study was designed to evaluate patient-, environment-, and disease-specific characteristics that may impact the risk of readmission to the PICU from hospital floor, with a specific objective to create a scoring system to predict the risk of early readmissions (PRediction of PICU Early Readmissions [PROPER]).

Patients and Methods

Following the approval by the institutional review board (IRB) of Mayo Clinic, Rochester, we conducted a retrospective chart review of EMRs from July 1, 2007, to June 30, 2014. All patients (0-17 years of age) who required unscheduled readmission to the PICU within 48 hours were included in the study. Two hundred controls were selected randomly from all patients who did not require readmission to the PICU within 48 hours. Pediatric intensive care unit at Mayo Clinic, Rochester is a 16-bed mixed medical/surgical unit (noncardiac) with an average daily census of around 10 to 12. Requirement for informed consent was waived by the IRB; however, patients were excluded from the study if they denied research authorization. All hematology–oncology patients were excluded from our study because their care is managed by a Hem-Onc service independent of the PICU providers. Patients who were discharged from the PICU to any location other than the pediatric floor were excluded. In our unit, a running database of unscheduled early readmission was maintained in our division since July 2007 as part of the quality improvement efforts. This list was validated with an automated query of the EMR to identify patients who required unscheduled readmissions within 48 hours.

For this study, we collected demographic data as well as characteristics of the patients during their stay in the PICU. Variables including age, sex, source of admission, mechanical ventilation requirement, and PICU lengths of stay were automatically extracted from the EMR. Severity of illness was calculated retrospectively as of the day of admission utilizing Pediatric Index of Mortality (PIM) software (Virtual PICU Systems LLC, Los Angeles, California). Cases of nosocomial infections (central line-associated blood stream infection [CLABSI], ventilator-associated pneumonia [VAP], catheter-associated urinary tract infection [CAUTI], pressure ulcers, Clostridium difficile colitis, and surgical wound infections) and any procedures performed during the ICU stay (gastrostomy tube placement, ventricular–peritoneal [VP] shunt placement external ventricular drain [EVD], tracheostomy, endoscopy, intracranial pressure [ICP] monitoring, hemodialysis, and plasmapheresis) were identified by conducting a free-text search on all the notes (including but not limited to admission notes, progress notes, administrative, consultations, and procedure notes). The last measured PaO₂/FiO₂ (P/F) Ratio, Glasgow Coma Scale (GCS), last blood gas PaCO₂, oxygen support at the initial discharge, and cause for PICU readmission information were extracted from the EMRs. The Pediatric Cerebral Performance Category–Pediatric Overall Performance Category (PCPC–POPC⁹) scores were calculated by 1 investigator (H.K.) based on the documented history and physical in the medical records at the time of admission. Vitals at the time of discharge were extracted from EMR electronically. These vitals were categorized as above or below the threshold based on prior published guidelines.¹⁰

Data were collected in a specific designated RedCap (Research Electronic Data Capture¹¹) instrument. These data were transferred to a JMP file and analyzed using the SAS software (Statistical Analysis System, SAS Institute for Advanced Analytics, Cary, North Carolina). Standard summary statistical analysis of categorical and binary data was utilized and presented as frequency and percentage. Data are presented as median (interquartile range [IQR]) or as the number of cases and the proportion as appropriate. Patient’s characteristics were compared between readmissions and controls with χ² and rank sum tests. The association of individual factors with PICU readmission was assessed separately in a univariate logistic regression model for each of the potential risk factors in turn. Odds ratios (ORs) greater than 1 indicate increased likelihood of readmission. From the regression analyses, factors were identified where there was good evidence (P < .05) for an association with readmission. A forward stepwise multivariate logistic regression was then used to develop a model that could be used to predict the patient at risk of readmission. The backward selection regression model confirmed the same variables. The PROPER risk score was created by assigning relative weights based on the risk factor coefficients from the regression model. Based on the parameter estimates from the logistic regression model, we assigned point values to each of the variables in the model according to the methods described by Sullivan et al.¹² To calculate the score for each patient, the points for each variable were summed. We then selected the cutoff value that would correctly identify 80% of the readmissions. The extent of model fit was based on the area under the receiver operator control curve. Variables comprising SWIFT scores were compared between the 2 groups. A mathematical score based on “SWIFT points” was calculated and compared. Its performance versus PROPER score was evaluated under the receiver operating characteristic (ROC) curve.

Results

Over the 7-year period, there were a total of 7371 patients discharged from the PICU, of which 4557 were transferred to the pediatric floor. Two hundred seventy-seven of these patients required readmission within 48 hours (readmission rate of 6.0%). Of these 277 patients, 86 patients met the inclusion and exclusion criteria and were included in the study. This cohort was compared to 170 random eligible controls from the same time period (Figure 1).

Figure 1.

Subject enrolment.

Characteristics of Readmitted Patients and Comparison With the Control Group

In our study population, patients requiring readmission were significantly younger than the control group (median [IQR] 2.7 [0.5-8.0] years vs 4.8 [1.1-12.2] years, P = .010). Thirty-three percent of the cases were <1 year of age, compared to 24% of the controls. There was a higher proportion of malnourished patients (weight < 15th percentile) in the readmitted versus the control group (OR [95% confidence interval, CI]: 2.01 [1.17-3.46]; P = .011). There were no significant differences between the 2 groups with regard to race or gender (Table 1).

Table 1.

Demographics.^a

	Not Readmitted (n = 170)	Readmitted (n = 86)	P Value
Age, years			.010
Median (Q1, Q3)	4.8 (1.1-12.2)	2.7 (0.5-8.0)
0-0.9	41 (24%)	28 (33%)
1.0-3.9	39 (23%)	24 (28%)
4.0-10.9	38 (22%)	20 (23%)
11.0 and above	52 (31%)	14 (16%)
Gender			.253
Female	80 (47%)	34 (40%)
Male	90 (53%)	52 (60%)
Body weight (percentile)			.011
0-14.9	48 (28%)	38 (44%)
15.0 and above	122 (72%)	48 (56%)
Race, Caucasian	118 (69%)	65 (76%)	.302

^aData are summarized as n (%) for categorical variables and median (Q1, Q3) for continuous variables. P values are from χ² tests, except for age that was compared using the rank sum test.

Comparing admission characteristics of the 2 groups showed that 45% of the cases were unscheduled admissions compared to 35% of the controls (P = .098). The PICU admission source was significantly different between the 2 groups. Compared to postoperative patients, patients who were admitted from the floor for index admission and patients admitted from the emergency department (ED) were more likely to be readmitted (OR [95% CI]: 3.63 [1.55-8.48] and 2.71 [1.48-4.95], respectively). Risk of mortality as assessed by the PIM2 score at admission, time from hospital admission to PICU admission, and the number of patients with trauma admitting diagnosis were not different in the 2 groups. There was a significant difference in the POPC categories between the 2 groups. Nine percent of the cases were categorized as having severe disability compared to only 2% of the controls. Patients with moderate to severe disability were at increased risk of readmission (OR [95% CI]: 2.16 [1.24-3.76]). There were no significant differences between the groups in the use of invasive or noninvasive mechanical ventilation or the use of vasopressors during the PICU stay. Surgical procedures (gastrostomy tube, VP shunt, ICP monitor, tracheostomy, endoscopy, or chest tube placement) were performed in 19% of the cases during their index PICU admission compared to 9% of the controls (OR [95% CI]: 2.36 [1.11-5.04]; P = .026; Table 2).

Table 2.

PICU Admission Characteristics.^a

	Not Readmitted (n = 170)	Readmitted (n = 86)	P Value
Time from hospital to PICU admission, hours	1.3 (0.2-9.6)	0.6 (0.2-11.6)	.656
PICU admission source			<.001
Another hospital floor/ICU	18 (11%)	5 (6%)
ED	50 (29%)	40 (47%)
Floor	14 (8%)	15 (17%)
OR	88 (52%)	26 (30%)
Unscheduled admission	59 (35%)	39 (45%)	.098
Trauma	14 (8%)	5 (6%)	.485
PIM score	0.5 (0.2-1.0)	0.5 (0.2-1.1)	.226
Invasive mechanical ventilation	46 (27%)	24 (28%)	.886
Noninvasive ventilation	18 (11%)	15 (17%)	.122
Any specified surgical procedure^b	15 (9%)	16 (19%)	.023
Use of vasopressors	21 (12%)	16 (19%)	.179
POPC			.006
Normal to mild disability	129 (76%)	51 (59%)
Moderate to severe disability	41 (24%)	35 (41%)

Abbreviations: ED, emergency department; ICP, intracranial pressure; ICU, intensive care unit; IQR, interquartile range; OR, operating room; PICU, pediatric intensive care unit; PIM, Pediatric Index of Mortality; POPC, pediatric overall performance; VP, ventricular-peritoneal.

^aData are summarized as n (%) for categorical variables and median (IQR) for continuous variables. P values are from χ² tests for categorical variables and rank sum tests for continuous variables.

^bSurgical procedures include gastrostomy tube, VP shunt, ICP monitor, and chest tube placement as well as tracheostomy and endoscopy.

Pediatric intensive care unit and floor censuses on the day of transfer from the PICU were used as surrogate measures for staff workload. Although the median PICU census for cases was slightly higher (10.0 vs 9.0), this difference was not statistically significant (P = .142). Similarly, no difference was found between weekday versus weekend discharge (P = .461) and regular hours versus call hours (5 pm to 7 am; P = .334). A higher percentage of cases were discharged in winter compared to controls (37% vs 23%, respectively; OR [95% CI]: 1.99 [1.13-3.50]; P = .017). Requirement of supplemental oxygen and GCS <15 at the time of discharge was strongly associated with readmission (OR [95% CI]: 12.32 [2.66-56.95], P = .001 and 2.16 [1.24-3.76], P = .010, respectively). Comparisons of the vitals on the day of discharge showed that the median heart rate was higher in cases than controls (P = .001); however, after characterizing the heart rate as abnormal based on age-specific parameters, the abnormal heart rate was suggestive but not statistically different between cases and controls (P = .089). Cases had significantly higher median respiratory rates than controls on the day of discharge (P < .001), and patients with abnormally high respiratory rate were at increased risk of readmission (OR [95% CI]: 3.39 [1.07-10.68], P = .038). No difference was observed in any measure of blood pressure. Oxygen saturation at the time of discharge was similar between the groups (Table 3).

Table 3.

Discharge Characteristics and Outcomes.^a

	Not Readmitted (n = 170)	Readmitted (n = 86)	P Value
PICU census	9.0 (7.0-11.0)	10.0 (8.0-13.0)	.142
Floor census	32.0 (28.0-35.0)	32.0 (28.0-36.0)	.321
Discharge time of week			.461
Week day	126 (74%)	60 (70%)
Weekend day	44 (26%)	26 (30%)
Discharge time of day			.333
Regular hours	117 (69%)	54 (63%)
Call time hours	53 (31%)	32 (37%)
Discharge season of year			.078
Spring	46 (27%)	23 (27%)
Summer	51 (30%)	20 (23%)
Fall	34 (20%)	11 (13%)
Winter	39 (23%)	32 (37%)
FiO₂ >0.21 at discharge	2 (1%)	11 (13%)	<.001
GCS <15 at discharge	41 (24%)	35 (41%)	.006
Heart rate^b	117.0 (94.0-129.0)	129.3 (109.0-142.0)	<.001
Abnormal heart rate^b	20 (12%)	17 (20%)	.085
Respiratory rate^b	23.8 (20.0-29.0)	26.5 (22.0-35.0)	<.001
Abnormal respiratory rate^b	5 (3%)	8 (9%)	.029
Systolic blood pressure^b	104.0 (97.0-115.0)	104.0 (96.0-118.5)	.626
Abnormal systolic blood pressure^b	1 (1%)	2 (2%)	.222
Diastolic blood pressure^b	58.3 (50.0-65.0)	59.8 (52.0-68.5)	.159
Mean arterial pressure^b	70.0 (62.0-77.5)	71.0 (63.0-80.0)	.368
SpO₂	97.0 (96.0-98.0)	97.0 (96.0-98.0)	.491
Complications^c	8 (5%)	4 (5%)	.984
Hospital length of stay, days	4.8 (2.9-9.0)	10.2 (5.9-19.8)	<.001
Total PICU length of stay, days	1.3 (0.9-3.5)	5.1 (2.9-10.0)	<.001
Hospital mortality	0 (0%)	2 (2%)	.112

Abbreviations: CAUTI, catheter-associated urinary tract infection; CLABSI, central line-associated blood stream infection; GCS, Glasgow coma scale; IQR, interquartile range; PICU, pediatric intensive care unit; VAP, ventilator-associated pneumonia.

^aData are summarized as n (%) for categorical variables and median (IQR) for continuous variables. P values are from χ² tests for categorical variables, except for hospital mortality that is compared using the Fisher exact test. P Values are from rank sum tests for continuous variables.

^bValues were determined by calculating the median value during the last PICU day. Abnormal values were defined using the criteria presented in Reference 10.

^cComplications include CLABSI, CAUTI, VAP, C difficile colitis, pressure ulcer, and surgical wound infection.

Clinical outcomes were significantly worse in cases compared to controls, with longer median total PICU stay (5 days [3-9] vs 1 day [1-3]; P < .01), longer hospital length of stay (9.5 days [6-18.7] vs 5 days [3-8]; P < .01), and higher mortality (2.6% vs 0%, P = .03). The proportion of patients with hospital-associated complications (including CLABSI, CAUTI, VAP, C difficile colitis, pressure ulcer, and surgical wound infection) was similar in the 2 groups (Table 3).

Tests of Validation for the SWIFT Score

The SWIFT scores were compared between our study groups. Two components of the SWIFT score were significantly different between cases and controls, including the source of ICU admission (P < .01) and GCS (P = .02), but the other 3 components, which included the last measured P/F ratio, PICU length of stay, and last measured PaCO₂, were not significantly different (P = 0.11, 0.27 and 0.60, respectively). The SWIFT scores were significantly different between the cases and controls (median [IQR]: 6.5 [1.0-12.0] vs 1.0 [0.0-9.0], respectively; P = .002). However, the SWIFT score discriminated poorly between cases and controls (AUC = 0.613; Table 4 and Figure 2).

Table 4.

SWIFT Components and Total Score.

	Not Readmitted (n = 170)	Readmitted (n = 86)	P Value^a
PICU admission source			<.001
OR^b	88 (52%)	26 (30%)
ED	50 (29%)	40 (47%)
Outside hospital	18 (11%)	5 (6%)
Ward	14 (8%)	15 (17%)
PICU length of stay, days			.27
<2	101 (59%)	42 (49%)
2-10	57 (34%)	37 (43%)
>10	12 (7%)	7 (8%)
Last PaO₂/FiO₂ ratio			.11
<100	1 (1%)	0 (0%)
100-149	7 (4%)	1 (1%)
150-400	22 (13%)	20 (23%)
>400	140 (82%)	65 (76%)
GCS at the time of discharge			.023
8-10	3 (2%)	3 (3%)
11-14	38 (22%)	32 (37%)
>14	129 (76%)	51 (59%)
Last PaCO₂			.60
<45	152 (89%)	75 (87%)
>45	18 (11%)	11 (13%)
SWIFT score			.002
Median (IQR)	1.0 (0.0-9.0)	6.5 (1.0-12.0)
0-5	94 (55%)	27 (31%)
6-10	42 (25%)	34 (40%)
11-15	14 (8%)	14 (16%)
16+	20 (12%)	11 (13%)

Abbreviations: ED, emergency department; GCS, Glasgow Coma Scale; IQR, interquartile range; PICU, pediatric intensive care unit; OR, operating room; SWIFT, Stability and Workload Index for Transfer.

^a P Values are from χ² tests for score components and rank sum test for the total score.

^bNot categorized within the SWIFT score.

Figure 2.

Receiver operating characteristic (ROC) curves.

Development of the PROPER Score

The PROPER risk predictive score elements identified by the multivariate logistic regression analysis included the PICU admission source, age at admission, oxygen requirement at discharge, discharged during the winter season, median heart rate >age-specific cutoff, moderate to severe POPC, and any specified surgical procedure (AUC = 0.794). After converting the model coefficients for each risk factor into points,¹² the mean (standard deviation) point scores for cases and controls were 17.2 (6.6) and 10.5 (5.3), respectively. Setting a scoring point cutoff at ≥12, we correctly identified (sensitivity) 70 (81%) of the 86 readmissions. The positive predictive value of this model was 70 (0.50) of 139, specificity was 101 (0.59) of 170, negative predictive value was 101 (0.86) of 117, and AUC was 0.795 (Table 5 and Figure 2).

Table 5.

Logistic Regression Results and Point Scoring Values.

Variable	Estimate	Std Error	P Value	Points
PICU admission source			<.001
Operating room^a	Ref			0
Floor	1.38	0.47		7
Emergency department	1.54	0.35		8
Age group			.047
11.0-17.9	Ref			0
4.0-10.9	1.11	0.50		6
1.0-3.9	1.36	0.51		7
0-0.9	1.16	0.49		6
FiO₂ >0.21 at discharge	3.08	0.84	<.001	15
Discharged during winter months^b	1.01	0.34	.003	5
Elevated heart rate^c	1.06	0.48	.028	5
POPC moderate to severe	0.68	0.33	.042	3
Any specified surgical procedure^d	1.00	0.46	.029	5

Abbreviations: ICP, intracranial pressure; PICU, pediatric intensive care unit; POPC, pediatric overall performance category; Ref, reference; Std, standard; VP, ventricular-peritoneal.

^aIncludes admissions from the outside hospital (n = 23).

^bMedian heart rate greater than age-specific cutoff.

^cDecember, January, or February.

^dSurgical procedures include gastrostomy tube, VP shunt, ICP monitor, and chest tube placement as well as tracheostomy and endoscopy.

Discussion

In this retrospective study, we compared patients who had early readmissions (<48 hours) after initial discharge from the PICU (cases) with patients who did not require readmission (controls). Based on this analysis, we were successful in creating an objective risk prediction score (PROPER) with a sensitivity of detecting unplanned readmissions of 81%.

Age as a Risk Factor for Readmission

In our study, patients less than 1 year of age had a higher rate of being transferred back to the ICU. Our data are similar to prior studies by Linton et al³ and Czaja et al.⁸ Younger ages are given more weightage in the PROPER scoring system based on these results.

Source of Admission and Surgical Procedure

In 2013, Czaja et al⁸ showed that unscheduled patients being admitted to the PICU initially were at an increased risk of readmission. Our data support these findings. Czaja’s observation of floor admissions to have a higher readmission risk was also replicated in our study. We found a higher risk of readmission for patients admitted from the ED (P = .001), which is in contrast to the studies by Yong et al¹³ and Gajic et al⁵ where they witnessed lesser ICU readmissions in patients admitted from the ED. As a component of the PROPER scoring system, for initial admission to the PICU, more weightage was given to admissions from the ED and floor relative to anyone whose admission source is the operating room.

Our data corroborate the findings from previous studies by Hua et al,¹⁴ Edwards et al,¹⁵ and Hain et al¹⁶ stating that nonoperated patients were more likely to require readmission. In our data, the patients who had a surgery before admission to the ICU were less likely to be readmitted to the PICU. Although our readmitted patients did have a higher rate of surgical procedures being done during the PICU stay, patients initially brought from the operating room did not have a higher readmission rate. This finding may indicate that these surgical patients only require acute care after surgery and don’t have an underlying chronic entity leading to the PICU admission. However, based on our results, any patient getting surgical procedures done in the PICU does hold weightage in the PROPER scoring system.

Physiological Parameters at the Time of PICU Transfer

Data have been published on the characteristics of patients who were readmitted at the time of admission and discharge.^3,8,17 We believed that the condition of patients at the time of discharge was more predictive of the upcoming readmission, so we concentrated on analyzing the physiological state of the patients at discharge, as well as the care they were receiving at discharge. Linton et al³ included the requirement of oxygen in their risk predictive model as a factor. Bernard and Czaja¹⁷ found that a larger percentage of patients who required readmission were on some form of respiratory support (oxygen or noninvasive ventilation). Our data correspond to this. Patients requiring more than room air were more likely to come back to the PICU (P = .001) within 48 hours, which could be an important factor when deciding PICU to floor transfer. This factor was given the maximum weightage in PROPER scoring based on the output from statistical modeling.

Vitals on the day of discharge were statistically significant on univariate analysis in our study with readmitted patients having a higher heart rate and respiratory rate at the initial discharge. Yong et al¹³ weighed higher heart rate accountable in their risk predictive model for medical ICU (MICU) readmissions. Linton utilizes unadjusted respiratory rates in their model.³ In our model, only increased heart rate (age-based cutoff) was found to be significant on multivariate logistic regression and was included in the score.

Capacity Strains and Patients’ Cognitive Development

Halpern¹⁸ mention the effect of capacity strains leading to decreased quality of care in the MICUs. We compared the PICU census for the day of discharge with floor census of the particular day to determine the capacity strains, but we were unable to find any significant correlation between census and readmission rates in our practice. After-hour discharges of the ICU have been mentioned in the literature as an important factor of readmission,^17,19 but our study did not support this finding. Unlike study findings by Bernard and Czaja,¹⁷ we saw higher readmissions in the winter months. Winter admissions to the PICU have typically different characteristics than rest of the year (higher respiratory virus infections), which lead to a different risk profile for readmission. Discharge of a patient during winter months was included in the PROPER score.

The POPC and PCPC scales are measures of a child’s cognitive and functional development.²⁰ Our study showed moderate and severe POPC rating to be a significant risk of readmission to the PICU. Moderate and severe disability in children would require dependence on nursing staff for care, which adds to the risk of readmission. The PROPER scoring takes this into account by including a higher POPC rating in the score.

Outcomes of Readmitted Patients

Our data suggest worse clinical outcomes including increased ICU length of stay, increased total hospital stay, and increased mortality in the readmission group. These findings are in concordance with prior studies both in the adult and in the pediatric literature.^18,20

Readmission Risk Prediction, Validation of SWIFT, and Development of PROPER Score

For risk prediction, adult studies by Zimmerman and Kramer²¹ utilized APACHE score as a tool. Therapeutic Intervention Scoring System and Simplified Acute Physiology Score were utilized by Campbell et al⁴ in identifying risk factors for death and readmission. These scores are not applicable to children.

The SWIFT score created by Gajic and team⁵ considered the GCS, need for oxygenation, and ICU length of stay as factors affecting the risk prediction. The SWIFT score has had contradicting results when centers tried to validate the risk predictive model.^7,22 We attempted to validate the SWIFT score for our pediatric population. Although the source of ICU admission and the GCS were statistically significant among the 5 factors and the total score was significantly different on nonparametric testing in the 2 groups, its AUC on the ROC curve was only 0.613. This may be related to the lower number of patients on its different elements. This in turn may be due to the differences in epidemiology in the PICU and MICU patients, thereby rendering the adult-specific scale invalid.

The only risk of readmission predictive score published in pediatrics is the one provided by Linton et al,³ which was created in Australia. They have 8 factors that included a subjective component of the liaison nurse assessment of the patients. Lack of liaison nurses in the United States has hindered the validation of this scoring system in our setting. Also, a specific person is needed to do this scoring, which is not favorable to come up with a risk prediction model.

Our logistic regression model provided a PROPER scoring system with factors including POPC, age, Fio ₂ at discharge, source of admission, heart rate above threshold, surgical procedure, and season of the year. Other than oxygen requirement and heart rate threshold, none of the factors are modifiable. However, the concept of this score is not to restrict transfer to the floor but to provide additional objective parameters to identify the risk. In the development of a cutoff score of 12, we have favored sensitivity (0.81) over specificity (0.59). Although this test cannot predict all readmissions, it can be utilized as a screening tool to identify patients who may have higher likelihood of readmissions. This is in contrast to the SWIFT score (cutoff 15, specificity 0.83, and sensitivity 0.54). We believe that among children, the potential higher false-positive rates with high sensitivity are justifiable due to significantly worse outcomes with readmissions and patient/family distress associated with the readmissions.

Limitations

Limitations of the study include its single-center design, retrospective nature, and no validation model. Generally, a model performs better on the development data set and will lose performance in validation. However, since the PICU at the Mayo Clinic has an organizational structure, patient load, and resources of a typical PICU in the United States, we believe the tool developed by evaluation of the characteristics of readmissions in our unit will maintain generalizability. The score also has to perform better than the “clinical judgment” of the physician, which would also require prospective, suitably designed studies. Although this tool can identify patients at risk at the time of transfer from PICU to floor, the clinical care and monitoring beyond PICU significantly impact the risk of readmission. Although the “pediatric floor”–level care is pretty uniform across medical centers in the United States, availability of intermediate care units in some hospitals can affect the validity of this model. The score does not discriminate well enough for a PICU patient with a respiratory viral illness in winter season when PICU beds are scarce. Further studies could either expand the population or categorize more specific factors.

Footnotes

Authors’ Note

This work was performed at Mayo Clinic, Rochester, Minnesota. Abstract was presented at the AAP National Conference 2015 and published in the December edition of journal “Pediatrics” as conference abstracts.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by grant support from the Critical Care Research Committee (Mayo Clinic).

References

Odetola

Clark

Dechert

Shanley

. Going back for more: an evaluation of clinical outcomes and characteristics of readmissions to pediatric intensive care unit. Pediatr Crit Care Med. 2007;8(4):343–347.

Brown

Ratcliffe

Kahn

Halpern

. The epidemiology of intensive care unit readmissions in the United States. Am J Respir Crit Care Med. 2012;185(9):p 955–964.

Linton

Grant

Pellegrini

Davidson

. The development of a clinical markers score to predict readmission to paediatric intensive care. Intensive Crit Care Nurs. 2009;25(6):283–293.

Campbell

Cook

Adey

Cuthbertson

. Predicting death and readmission after intensive care discharge. Br J Anaesth. 2008;100(5):656–662.

Gajic

Malinchoc

Comfere

. The Stability and Workload Index for Transfer score predicts unplanned intensive care unit patient readmissions: initial development and validation. Crit Care Med. 2008;36(3):676–682.

Chandra

Agarwal

Hanson

. The use of an electronic medical record based automatic calculation tool to quantify risk of unplanned readmission to the intensive care unit: a validation study. J Crit Care. 2011;26(6):634.e9–634.e15.

Kastrup

Powallik

Balzer

. Predictive ability of the Stability and Workload Index for Transfer score to predict unplanned readmissions after ICU discharge. Crit Care Med. 2013;41(7):1608–1615.

Czaja

Hosokawa

Henderson

. Unscheduled readmissions to the PICU: epidemiology, risk factors, and variation among centers. Pediatr Crit Care Med. 2013;14(6):571–579.

Fiser

. Assessing the outcome of intensive care. J Pediatr. 1992;121(1):68–74.

10.

Pediatric Trauma: Advanced Trauma Life support, Student Course Manual. 9th ed. Chicago, IL: American College of Surgeons; 2012:P246–P284.

11.

Harris

Taylor

Thielke

Payne

Gonzalez

Conde

. Research electronic data capture (REDcap—a metadata driven methodology and workflow process for providing translational research information support. J Biomed Inform. 2009;42(2):377–381.

12.

Sullivan

Massaro

D’Agostino

Sr . Presentation of multivariate data for clinical use: the Framingham study risk score function. Stat Med. 2004;23:1631–1660.

13.

Yong

Yeon

Jong

. Readmission to medical intensive care units: risk factors and prediction. Yonsei Med J. 2015;56(2):543–549

14.

Hua

Gong

Brady

Wunsch

. Early and late unplanned re hospitalizations for survivors of critical illness. Crit Care Med. 2015;43(2):430–438.

15.

Edwards

Lucas

Stone

Boscardin

Dudley

. Frequency, risk factors and outcome of early unplanned readmissions to the PICU. Crit Care Med. 2013;41(12):2773–2783.

16.

Hain

Gay

Berutti

Whitney

Wang

Saville

. Preventability of early readmissions at a children’s hospital. Pediatr. 2013;131(1):e171–e181.

17.

Bernard

Czaja

. Unplanned pediatric intensive care unit readmissions: a single center experience. J Crit Care. 2013;28(5):625–633.

18.

Halpern

. ICU capacity strain and the quality and allocation of critical care. Curr Opin Crit Care. 2011;17(6):648–657.

19.

Pilcher

Duke

George

Bailey

Hart

. After-hours discharge from intensive care increases the risk of readmission and death. Anaesth Intensive Care. 2007;35(4):477–485.

20.

Pollack

Holubkov

Funai

. The relationship between the functional status scale and the pediatric overall and cerebral performance categories. JAMA Pediatr. 2014;168(7):671–676.

21.

Zimmerman

Kramer

. A history of outcome prediction in the ICU. Curr Opin Crit Care. 2014;20(5):550–556.

22.

Oakes

Borges

Forgiarini Junior

Rieder Mde

. Assessment of ICU readmission risk with the Stability and Workload Index for Transfer score. J Bras Pneumol. 2014;40(1):73–76.