Abstract
Background
Prolonged mechanical ventilation (PMV) is associated with poor long-term outcomes. The ProVent14 score was developed to estimate one-year mortality among patients receiving mechanical ventilation for at least 14 days. However, contemporary validation of the model and its performance earlier during mechanical ventilation remain uncertain. We aimed to evaluate the validity of the ProVent14 score in a modern ICU cohort and assess its predictive performance when applied earlier during mechanical ventilation.
Methods
We conducted a retrospective cohort study using the Medical Information Mart for Intensive Care (MIMIC-IV) database (2008-2022). Adult ICU admissions receiving at least seven days of mechanical ventilation were included. Patients with chronic neuromuscular disease, extensive burns, or chronic mechanical ventilation were excluded. Three non-mutually exclusive cohorts were constructed based on duration of ventilation (≥7,≥10, and ≥14 days). Predictors from the original ProVent14 model (age, platelet count, vasopressor use, hemodialysis requirement, and trauma admission status) were assessed at each time point. The primary outcome was one-year mortality from the onset of prolonged ventilation. Model discrimination was assessed using the area under the receiver operating characteristic curve (AUC), and calibration was evaluated using the Hosmer–Lemeshow test.
Keywords
Introduction
Prolonged mechanical ventilation (PMV) has become increasingly prevalent due to advancements in critical care and an aging population.1,2 While multiple definitions of PMV have been reported (eg, invasive mechanical ventilation for at least 14 days, 21 days, admission to a ventilator-weaning unit, mechanical ventilation for 96 h or more with a tracheostomy),1,3 regardless of definition, PMV outcomes are generally poor. 4 Prior studies have shown that approximately 1 in 10 patients with PMV achieve functional independence at home, 3 of 4 do not survive to one year,3,5,6 and nearly half do not survive beyond 90 days after tracheostomy placement. 7
Although outcomes of PMV are poor, estimating the prognosis of individual patients facing PMV to assist in tracheostomy and care goals discussions has been difficult due to heterogeneity of population, comorbid conditions, and a dynamic clinical course. The Prolonged Ventilation (ProVent) 14 score was developed as an objective, bedside-assessable tool to estimate one year mortality risk,8,9 incorporating age, platelet count, vasopressor requirement, need for hemodialysis, and trauma admission status on PMV day 14 to calculate mortality risk at 1 year 10 to potentially inform clinical decision-making, with moderate to good discrimination. 8
However, contemporary implementation of the ProVent14 is limited by several factors including lack of validation of ProVent14 in contemporary cohorts and unclear accuracy to inform decisions at earlier times during mechanical ventilation (eg, days 7 or 10) when decisions might be made. We sought to address these knowledge gaps by evaluating both contemporary validity of ProVent14 and its performance when applied earlier during mechanical ventilation.
Methods
We used data from the Medical Information Mart for Intensive Care (MIMIC-IV) database, a publicly available database sourced from the electronic health record of the Beth Israel Deaconess Medical Center from years 2008-2022.
We identified all hospital admissions with at least continuous 7 days of mechanical ventilation based on ventilator mode and oxygen delivery data, excluding admissions for patients aged 17 or younger and those with International Classification of Diseases, ninth (ICD-9) and tenth (ICD-10) revision codes for chronic neuromuscular disease, extensive burns, and chronic mechanical ventilation. 11 We constructed non-mutually exclusive cohorts of patients with PMV for 7, 10, and 14 consecutive days to evaluate PROVENT performance at different time points. We randomly selected one admission in patients with multiple hospital stays.
Predictors from the original Provent14 score were assessed: age at admission, platelet count, receipt of vasopressors, receipt of hemodialysis within ±48 h, and admission with trauma on the seventh, 10th, and 14th consecutive day of mechanical ventilation for the respective cohorts. Trauma was not a native MIMIC data field; instead, admissions with trauma were identified using ICD-9 and ICD-10 diagnosis codes recorded in the EHR, based on published injury code groupings, excluding ICD-9 codes ‘995.50'-'995.59’ for child maltreatment syndrome. 12 The outcome was 1-year mortality after the onset of PMV.
We calculated the Provent14 score for each mechanical ventilation cohort by summing the point values according to the presence of predictors from the original ProVent14 model. 10 We re-fitted multivariate logistic regression models with data for each of the study cohorts to predict 1-year mortality using the original ProVent14 predictors. We assessed the models’ discrimination using the Area Under the Receiver Operating Characteristic (AUC) curve and calibration using the Hosmer–Lemeshow goodness-of-fit test. We also tested the regression models using 10-fold cross-validation to assess model performance, specifically discrimination and calibration and to confirm stability of predictors across different data subsets, which allowed us to use the whole study samples for model building.13–14 All analyses were performed with SAS version 9.4 (SAS Institute Inc) and Stata version 18 (StataCorp). This study was conducted as a complete case analysis, as shown in Table 2.
Results
We created three cohorts of patients who remained on mechanical ventilation until day 7 (N = 1344), 10 (N = 634) or 14 (N = 249). Baseline characteristics are shown in Table 1, with the cohort predominantly White and male. Vasopressor use declined from 43.7% at day 7 to 35.3% at day 14, while hemodialysis requirements increased from 35% to 41%. One-year mortality was similar across all three cohorts (64-67%).
Characteristics of Patients Receiving Prolonged Mechanical Ventilation and Mortality Outcomes by Duration.
ProVent14 score validation at days 7, 10, and 14 during mechanical ventilation showed consistent performance. Calibration was similar across all three points (H-L P-values .5, .18, .21) and discrimination was also consistent (AUC 0.74, 0.72, 0.71) as shown in Table 2. Individual model covariates for the risk category model are also shown in Table 2. When calculated separately for 2008-2013 and 2017-2022 cohorts demonstrated comparable calibration (0.99 vs 0.96) and same discrimination of 0.73.
ProVent Score Validation at day 7, 10 and 14 of Mechanical Ventilation.
Platelet count was the only variable with missing data, (96 in day-7, 58 in day-10, 20 in day-14 groups).
Discussion
ProVent14 scores maintain stable discrimination and calibration within contemporary data. In addition, the ProVent scores calculated using day 7 and 10 values had similar predictive validity as the original day 14 data, extending opportunities to use ProVent14 for prognostication earlier during clinical course and among a much larger group of eligible patients. Similarly, accuracy of ProVent scores did not change over years of the study, further highlighting stability of prognostic accuracy.
Subsequent validation studies of the ProVent framework have shown generally moderate to good predictive performance across a range of cohorts, although calibration has varied. In a retrospective single-center study from Austria, including 780 critically ill patients who required at least 21 days of mechanical ventilation between 2015 and 2019, the ProVent probability model demonstrated adequate discrimination (AUC 0.76, 95% CI 0.72-0.79) but was miscalibrated. 15 In Korea, the ProVent14 score was validated in 366 patients who received at least 14 days of mechanical ventilation between 2011 and 2015, with an AUC of 0.74 (95% CI 0.69-0.78) and acceptable calibration. 16 Likewise, in a retrospective cohort analysis from Lowa including patients who received 14 or more days of mechanical ventilation between 2006 and 2008, the ProVent model showed AUCs of 0.69 and 0.75 in the ≥21-day and 14–20-day cohorts, respectively, with acceptable calibration in both groups (Hosmer-Lemeshow P = .24 and .22, respectively). 17 Compared with these prior studies, our analysis evaluated a contemporary ICU cohort at earlier time points, specifically days 7, 10, and 14 of mechanical ventilation, and demonstrated stable discrimination with acceptable calibration across all three time points.
This study has some limitations. The validation was within single-center data with narrow patient characteristics, which limits how broadly the findings can be applied to other hospitals or health systems. We did not test patient or clinician acceptability for using prognostic risk scores with Provent's level of accuracy.
Conclusion
The ProVent score continues to perform reliably in a modern ICU population. In addition to its original validation at day 14, our findings show consistent performance at days 7 and 10 of mechanical ventilation. This supports further studies evaluating use of Provent across the course of critical illness to facilitate decision making.
Footnotes
Contribution of Authors
Substantial contribution to the
-Conception and design of the work, analysis, interpretation of data.
-Drafting the work critically for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
-Conception and design of the work, acquision, analysis, interpretation of data for the work
- Drafting the work critically for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Design of the work, analysis, interpretation of data.
-Drafting the work critically for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
-Conception and design of the work, acquision, analysis, interpretation of data for the work
- Drafting the work critically for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Design of the work, analysis, interpretation of data.
-Drafting the work critically for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
-Conception and design of the work, acquision, analysis, interpretation of data for the work
-Reviewing it for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
-Conception and design of the work, acquision, analysis, interpretation of data for the work
-Reviewing it for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
-Conception and design of the work, acquision, analysis, interpretation of data for the work
-Reviewing it for important intellectual content
-Final approval of the version to be published.
-Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.