Abstract
Objective. To identify clinical and psychosocial factors associated with patient experience with care. Methods. We analyzed patient experience surveys, corresponding clinical and psychosocial the data of 1567 encounters using survey-weighted multivariate logistic regression analysis with willingness to recommend the facility as outcome variable. Results. Parents are less likely to recommend the facility if there were custody issues with the child during their stay, if the child has history of chronic medical condition, and if the child is male with odds ratio and corresponding 95% confidence interval of 0.38 [0.21, 0.69], 0.43 [0.24, 0.80], and 0.67 [0.45, 0.99] respectively. Parents of older patients (1-year difference) and parents of low socioeconomic status are more likely to recommend the facility (1.05 [1.01, 1.09] and 2.74 [1.72, 4.37] respectively). Conclusions. Clinical and psychosocial factors significantly affect patient experience scores together with parent perception of provider-family communication and relationship, and hospital environment conducive for children.
Keywords
Introduction
The focus on patient experience with care has increased with the inclusion of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores in the Value-Based Purchasing Program (VBP) of the Centers for Medicare and Medicaid Services (CMS), and with the public reporting of patient experience scores of participating institution.1-4 The HCAHPS is a standardized tool for measuring patients’ perception of care at a hospital or health care facility and its use is mandatory for hospitals that receive any payments from Medicare. CMS, using results from the HCAHPS survey as part of the VBP, penalizes healthcare institutions through certain reductions in total Medicare payments for institutions that fail to meet defined performance standard or achievement thresholds.1,2,5 In this study, the Child HCAHPS survey is used instead of the adult-oriented HCAHPS survey 6 since the population in question is pediatric.
Several studies on patient experience have been carried out in adult population with focus on a wide range of subpopulations such as inpatient, primary care, and specialist encounters.7-11 While there have been studies on patient experience with treatment choice in pediatric specialist and emergency clinics,12-14 on prescription of antibiotics, 15 on immunization, 16 and on quality of parent-provider communication, 17 there is a paucity of research into patient experience with general inpatient pediatric encounters using validated survey tools such as the Child HCAHPS survey while considering additional clinical and psychosocial variables. The purpose of this study is to provide an analysis of factors associated with patient experience in inpatient pediatric populations by considering clinical and psychosocial variables, and responses to the Child HCAHPS survey. The underlying hypothesis is that novel clinical and psychosocial factors associated with patient experience will be identified. Furthermore, we will identify the most important set of variables from the Child HCAHPS survey (after accounting for any statistically significant clinical and psychosocial factors). The clinical variables include demographics/socioeconomic variables, length of stay, pain intensity scores, number of chronic conditions, number of hospitalizations and emergency room visits within the past year, inpatient surgeries, and maximum Pediatric Early Warning System (PEWS) score during the encounter. On the other hand, the psychosocial variables considered includes presence of legal custody issues, domestic (patient/parent) concerns, parent practices that might impact care, use of alternate medical treatment, and request for spiritual care. The PEWS score is a patient deterioration score built on four nursing assessment domains corresponding to respiratory, cardiovascular, neurobehavioral, and extra points (for nebulizer use, tracheostomy/high-flow nasal cannula, and hypertension).18,19 Each domain has a maximum of 3 points resulting in a maximum total score of 12. Consequently, the higher the PEWS score, the higher the acuity of the patient.
Methods
We obtained institutional review board approval to undertake this study and retrieved the Child HCAHPS survey data of patients admitted at the pediatric hospital between July 2014 and March 2016. The inpatient facility is a state-of-the-art 279-licensed-bed hospital that is part of a comprehensive health system serving the Southern California region. It is sectioned into several nursing units, including medical, surgical, and intensive care units (pediatric, neonatal, and cardiovascular). The number of direct care hospital/inpatient nurses is estimated to be 665 as at December 2016. The number of attending physicians and residents were estimated at more than 600 and 500 respectively at the time of writing. This estimate for physicians cut across all service areas including ambulatory, primary, and specialty care clinics. The inpatient facility shares the same building as a pediatric emergency room center. Other ambulatory (primary care, specialty care, outpatient surgical) facilities share the same building as the inpatient facility or are located at adjoining or nearby buildings.
During the study period, 11 455 surveys were sent to patients discharged from 9 nursing units (corresponding to the medical/surgical and intensive care units). Of these surveys, 225 were undeliverable and 2038 were returned with a response rate of 18.1%. We however focused our analysis on all inpatient visits excluding neonatal visits and outpatient overnight observation encounters. Consequently, our sample size is 1567 after exclusion of all encounters corresponding to neonates, outpatient overnight observations, and surveys for which the parent/guardian was not with the child during the visit.
The Child HCAHPS survey questionnaire consists of 11 main sections (when child was admitted, child’s experience with nurses, and doctors; parent’s experience with nurses, and doctors; experience with providers, child’s care, hospital environment, when child left the hospital, teens, and overall rating of the hospital) and 3 additional sections on the ICU, about child, and about parent. We focused only on survey questions that target the general pediatric population and not a subset such as questions depending on the child being verbal or being a teenager. From the “overall rating of the hospital” section, we chose the question on recommending the hospital (“Would you recommend this hospital to your family and friends?”) as the response/outcome variable for the study. All survey variables were coded using the HCAHPS “top-box” and “positive” scores standard for actionable and overall rating questions where only the top-rated responses are considered positive. 20 For questions using the 4-point scale, values corresponding to “Yes, definitely,” “Definitely yes,” or “Always” were the top responses and were coded positive, otherwise negative. On the question on parent perception of child’s health, we coded as “Good” if parent selected “Excellent,” “Very good,” or “Good” and as “Poor” if patient selected “Fair” or “Poor”. Refer to Appendix A for a sample copy of the survey questionnaire.
In addition to the survey data, we retrieved clinical data corresponding to information on patient demographics and proxy for socioeconomic status, and variables that may be proxy for how sick the child was (including maximum PEWS score, number of inpatient procedures, number of previous hospitalizations and emergency department visits within the past year, number of chronic conditions, and length of stay). We also retrieved psychosocial variables including identifiers for families who had custody issues, presence of domestic (patient/parent) concerns, use of alternative medical treatment, observation of practices that might affect care and requested for spiritual counsel/prayer. We did not include information about suspected abuse/neglect due to insufficient sample size. Furthermore, we retrieved the maximum pain intensity scores of the child during the encounter. In total, we considered 33 general survey questions, and 16 additional clinical and psychosocial factors retrieved from our Cerner Millennium database.
We analyzed missing data using the classical nearest neighbor hot deck imputation. 21 The classical nearest neighbor hot deck imputation is a method for handling survey nonresponse and missing data by replacing with values from complete records most “similar” to the record with missing data22-24 and is generally regarded as better practice than removing all records with missing data.24-26 Since most analysis of complex survey samples are design based, 27 we proceeded with the analysis of the data by building corresponding design-based models. The surveys were sent to parents stratified by the nursing units from which the child was discharged. Consequently, we treated the data as a stratified sample for analysis using survey-weighted logistic regression. 27
We analyzed the data in 2 stages. First, we provided summary statistics/bivariate analysis with the estimated population proportions of patients with positive and negative recommendation across the levels of each categorical variable in the data, and corresponding means and standard errors (SEs) for the continuous variables. In addition, we provided odds ratio, corresponding 95% confidence intervals (CIs) of the odds ratio, and chi-square P values obtained between the outcome variable and all other variables. In Table 1, we provide results of this stage of the analysis for a subset of variables corresponding to the clinical and psychosocial variables, and variables from each section of the survey with the smallest unadjusted P value. We, however, provided supplementary material on the summary statistics/bivariate analysis of all variables in Appendix B.
Psychosocial Factors.
In the second stage of the analysis, we conducted model selection by stepwise reduction of Akaike information criterion (AIC) controlling for patient’s age, gender, ethnicity, socioeconomic status, and the maximum PEWS score of the child. The AIC measures the degree of deviation of a model from its theoretical probability distribution and can therefore be used for comparing models derived from the same data set.28,29 Finally, we checked the multivariate model for possible problems with multicollinearity by estimating the generalized variance inflation factors for all variables in the model. The variance inflation factor is an estimate of how much the estimated variance of a predictor variable is increased above what it would be if it is orthogonal to other independent variables in the analysis. If the variance inflation factor is not unusually higher than 1 (there are rule of thumb recommendations of 4 or 10), then multicollinearity is not suspect.30-32 All data preprocessing and model development were performed using the R Statistical Programming Language, and the survey and hot deck imputation packages.21,27,33,34
Results
The mean age of patients in the study was 7.2 years (SD 5.6 years); 44.7% were female, 48.0% were of Hispanic, Latino, or Spanish Origin, and 46.5% were on a low-income insurance plan.
The sample size of this study was 1567 with a positive recommendation rate of 86.2%. Of the 16 clinical and psychosocial variables, only 6 (corresponding to age, ethnicity, socioeconomic status, chronic conditions, the maximum PEWS score, and number of emergency department visits within the past year) had unadjusted significance. In contrast, 24 of the 33 survey variables had unadjusted significance. Results of bivariate analysis for selected variables is presented in Table 2.
Bivariate Analysis for Selected Variables (see Appendix for All Variables).
Abbreviations: SE, standard error; OR, odds ratio; CI, confidence interval; PEWS, Pediatric Early Warning System.
The corresponding multivariate model obtained revealed that socioeconomic status, custody issues, chronic conditions, and patient age and gender were significant predictors associated with patient experience from all 16 clinical and psychosocial variables we considered (Table 3). The survey variables that were identified as significant predictors of patient experience include variables on environment conducive for a child to learn, play, and sleep; and 2 communication themes (parent-provider communication about child care, and parent perception of child-provider communication). All variables in the model achieved a generalized variance inflation factor of less than 2, well below the rule of thumb of 4 or 10 indicating that multicollinearity is not a problem.
Results of Multivariate Analysis.
On Clinical Variables
Parents of low socioeconomic status are more likely to recommend the hospital. If the family is dealing with a custody issue during the hospitalization, they are less likely to recommend the hospital. Parents of older patients are more likely to recommend the hospital. Parents of children with history of a chronic medical condition are less likely to recommend the hospital than parents whose children have no such history. If the child is male, parents are less likely to recommend.
On Conducive Environment for Child to Learn, Play, and Sleep
If parents perceive that the hospital does not have appropriate toys, books, mobiles, and games during the child’s stay they are less likely to recommend. If they perceive that the area around their child’s room was not kept quiet at night, they are also less likely to recommend the facility.
On Parent Perception of Child-Provider Communication and Relationship
Parents who perceive that providers did not talk with and act toward their child in ways appropriate for the child’s age are less likely to recommend the hospital.
On Parent-Provider Communication About Child Care
Parents are less likely to recommend the hospital if they perceive that providers at the hospital did not always explain symptoms/health problems to look for after discharge. They are less likely to recommend the hospital if they perceive that doctors did not listen to them always; if providers did not keep them informed about their child’s treatment; if they perceive that providers did not ask questions that a family might know best about the child; they perceive that the nurses did not listen to them always and were not courteous and respectful.
Discussion
Our result indicates that parents of low socioeconomic status are more likely to recommend the hospital (than parents of high socioeconomic status). In other words, they are more likely to have positive perception of care, which may be driven by differences in expectations of care. 9 The distress associated with custody issues on both parent and child may further increase stress and anxiety35,36 associated with the hospitalization of the child. Consequently, the association between parent/families undergoing custody issues and a tendency to not recommend may be capturing the overall interplay of a host of factors associated with custody issues and potential intervention by social services during hospitalization.
It is unclear why an association between the age of the child and parent recommendation patterns exist (in such a way that parents of older patients are more likely to offer positive recommendation). However, this result is consistent with studies on patient experience in adult population where older patients tend to be more satisfied with care than younger patients. 9 But it is important to note that besides the obvious difference between pediatrics and adult medicine, it is the parent of the patients that assess the quality of care in pediatrics and not the patients themselves as in adult populations. Additional studies may be required to understand this association while taking into consideration that there may be underlying difference in care by age of patient, and in the anxiety levels of parents by the age of their sick child.
Results on history of chronic conditions indicate that the parent of a child with history of a chronic condition is less likely to recommend the hospital compared with parents whose children have no history of such conditions. Both parent and child may suffer additional stress in management of chronic disease, 37 and in some cases the disease may be activity-limiting. 38 These children may require more complex care, have need for utilization of more hospital resources, and have a complex parent-child relationship. 37 Our result indicates that additional focus on the families of children with chronic diseases may be required to improve their experience with care and the perception of care by their parents. It is not exactly clear why an association between patient gender and parental recommendation scores may exist—further studies may be required to understand it.
While previous studies have focused on the relationship between physical hospital environment and satisfaction of nurses,39-41 our result suggests that focus should also be placed on parent perception of a hospital environment that is conducive and child-friendly. Areas around patient rooms should be kept as quiet as possible at night to ensure patients can rest unperturbed and resources to promote playful and educating activities should be provided through continued consultation with parents of patients across the entire pediatric age group.
On parent-provider communication/relationship, our result is in line with findings from previous studies. Parents are concerned about doctors and nurses listening to them, nurses being respectful and courteous, and providers consulting them on issues that they may know best about the child. Furthermore, parents are paying attention to how providers act around their children and are concerned with the care of their children after discharge from the hospital.
Our study was focused on the inpatient facility of a single tertiary pediatric institution. This is a limitation because, although we expect that studies from other inpatient institutions would establish the same theme of results as ours, we cannot guarantee that these results would extrapolate to all other pediatric inpatient facilities. Furthermore, we considered only survey questions that address patients of all ages. This implies that we did not consider questions that depended on the patient being verbal or being able to directly communicate with providers. Such a study would have to focus specifically on verbal patients and may illustrate more specific points on provider-child communication that drive patient experience with care. We note, however, that our study did capture a more general variable (parent perception of whether providers acted in ways appropriate for a child’s age or not) indicating the importance of good provider-child communication and relationship. Our selection of psychosocial variables was based simply on variables captured within our nursing admission history reports. We note that there may be other psychosocial factors associated with care (including social service intervention due to suspected child abuse or neglect, which had insufficient sample size to be included in our study). A much larger study may reveal additional psychosocial factors associated with patient experience with care.
The need to focus on patient experience with patient care has increased with the inclusion of HCAHPS scores into the VBP of the CMS.1-3 Consequently, there is increased need for understanding factors associated with patient experience in pediatrics. This study is an attempt to fill a void in research into drivers of patient experience with pediatric care. Our results indicate that in addition to focusing on improving patient care (and parental perception of care as captured within the Child HCAHPS survey), we should also consider clinical and psychosocial factors about the child and family to obtain a clearer picture of drivers of desirable patient experience with care. Furthermore, the significant clinical and psychosocial variables may serve as a starting point for building of real-time predictive models for identifying families who may not be having a good experience or perception of care during the hospitalization of a child. Since the values of these variables are known during hospitalization of the child, they may be helpful in aiding proactive measures to mitigate undesirable patient experience with care.
Author Contributions
LE designed the study, ran statistical analysis, drafted the initial manuscript, revised and approved the final manuscript. SS provided access to the survey data, helped with understanding the survey data, assisted in the interpretation of results, reviewed, revised, and approved the final manuscript. WF conceptualized the study, provided access to clinical and psychosocial data, contributed to interpretation of results, reviewed, revised, and approved the final manuscript.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.