Abstract
Introduction
The American Society of Health-System Pharmacists (ASHP) requires all residency programs to include a research project to receive accreditation. 1 However, there are a variety of factors limiting the success of residency research projects. Residents’ understanding of research methodology and ability to perform statistical analyses are major hindrances. Many pharmacy residents embark upon residency research with limited prior experience in selecting and performing statistical analyses. 2 Multiple surveys of pharmacy residents have demonstrated pharmacy residents’ knowledge of biostatistics and research study design are self-identified deficits.2-6 In a 2004 national survey of pharmacy residents, Ellis et al found overall knowledge of biostatistics was the lowest rated skill in execution of a research project. Moreover, the survey demonstrated that residency experience correlated with an improvement in self-assessed understanding of statistical tests; however, these results did not correlate with an objective improvement in research knowledge test scores. Residents’ expectations that the residency would increase research knowledge, skills, and desire for research involvement were also unmet. 5 In a 2013 survey of 98 pharmacy residents who attended the Western States Conference, 39.8% rated analyzing research data as a barrier to completing and publishing research. 6 Studies have shown that the publication rates for residency research projects are only 4.3% to 16%.7-9
A strong foundation of statistical knowledge is important for pharmacists to be successful in sharing new innovations and evaluating published literature. This is evidenced by the Board of Pharmacy Specialties certification examinations including statistics as a core concept of the exam. 10 For residents to have future success and confidence in performing research, they must gain experience with statistical manipulations and software. While there is significant literature detailing residents’ lack of preparedness in biostatistics, no information is currently available about what specific statistical guidance is frequently provided by residency programs.
This study addressed 2 primary goals: (1) to describe the breadth of statistical training across postgraduate year 1 (PGY1) pharmacy residency programs and (2) to examine the relationship between the rigor of statistical training and self-reported residency publication rates as well as residency program directors’ (RPDs) confidence in residents’ abilities to become independent investigators after their residency training.
Methods
A comprehensive list of email addresses for PGY1 RPDs was obtained from the ASHP online residency directory in December 2015. The list included all ASHP residencies that were accredited or in candidate status for accreditation. An email invitation was sent on April 11, 2016, to each RPD to participate in the study survey. Two reminder emails were sent out, and the survey was closed on May 13, 2016.
The 13-item survey, administered through Opinio (Oslo, Norway), included questions regarding the residency program location, size, institution type, statistical training methods provided to and desired for residents, publication rate of residency research projects, RPD’s confidence in residents’ statistical manipulation abilities, and unique aspects of the program.
The email announcement provided contact information for the primary investigator and stated that, by completing the survey, participants were granting consent to be included in the survey. All information was collected and stored anonymously in accordance with institutional procedure. All respondents who answered the survey were included in this study for analysis. The University of New Mexico Health Science Center Institutional Review Board (IRB) approved this study.
Statistical Analysis
Type of statistical training residents received was placed into 4 groups based on the highest complexity of statistical training provided to residents. Programs that provided training in basic multivariate statistics (eg, multivariable linear regression, multiple logistic regression, analysis of covariance [ANCOVA]) or any more advanced statistical techniques (eg, nonparametric tests, polychotomous regression, principal components analysis, repeated-measures analyses, mixed modeling, Poisson regression) were classified as providing complex statistics. Programs were classified as providing introductory statistics if the highest level of training was descriptive statistics (eg, means, frequencies to characterize study population) or basic univariate statistics to compare means or proportions (eg, chi-square, t test, analysis of variance [ANOVA], correlation). Programs that only provided study design and database building training were grouped together, and the final group was programs that did not provide any statistical training.
RPD confidence in residents’ statistical abilities was assessed on a 5-point Likert-type scale with “5” indicating complete confidence and “1” indicating no confidence. On the premise that after completion of a PGY1 residency, RPDs should have at least average or greater than average confidence in the resident’s statistical ability; the responses were dichotomized as low confidence (category 1 or 2) and high confidence (categories 3-5). Other variables were also dichotomized for analysis such as age of the program (>5 vs ≤5 years), hospital beds (<250 vs ≥250), the presence of a postgraduate year 2 (PGY2) residency program, use of statistician (<50% vs ≥50%), and resident publication rate (<50% vs ≥50%).
Descriptive statistics were used in this study with median for nonnormally distributed continuous data. Categorical data were compared using chi-square. Predictors of RPD confidence and self-reported publication rate were analyzed via multivariate logistic regression with only variables with P < .05 in the univariate model included in the multivariate analysis. SPSS (IBM Corporation, Armonk, New York) was used for statistical analysis.
Results
Of the 1054 residency programs included in the survey, 202 (19.2%) responded. Nine programs (0.9%) reported having no PGY1 or PGY2 candidates in the 2014-2015 cycle: However, all 9 respondents completed the survey in its entirety with additional comments that were reflective of resident training. The decision was made to include these respondents in the analysis. Overall, the included programs were geographically diverse across the United States as represented in Figure 1.
Geographical distribution of survey respondents.
Programs Characteristics
The program data of survey respondents are shown in detail in Table 1. The respondents had a diverse mix of how long the program has been established as well as the size of the institution. Nearly half the residency programs were through privately owned institutions. Of the 202 respondents, 77 (38.1%) had at least 1 PGY2 program associated with the PGY1.
Residency Program Characteristics (N = 202).
Note. PGY1 = postgraduate year 1; PGY2 = postgraduate year 2.
Categories are not mutually exclusive.
Community hospital, managed care, health insurer.
Statistical Training
Nearly 25% of the PGY1 programs surveyed offered no statistical training to their residents (Table 2). The most common types of training reported for PGY1 pharmacy residents were study design considerations/selecting statistical tests (64.9%), descriptive statistics (59.9%), and database development/data manipulation (46.5%). Over 60% of the respondents reported low confidence in their residents’ statistical capabilities (123 vs 79, P < .05). The univariate analysis of the categories found that presence of a PGY2 residency program, academic medical center setting, higher level statistical training, greater than 5 hours of statistical training, and a publication rate ≥50% were significantly associated with RPDs having higher confidence in residents’ statistical abilities. The multivariate analysis found that only the presence of complex statistical training and publication rates remained statistically significant (Table 3).
Description of Statistical Training, Publication Rates, and Confidence in Residents’ Abilities (N = 202).
Note. ANOVA = analysis of variance; RPD = residency program director.
Categories are not mutually exclusive.
R, Sigma programs.
Predictors of Residency Directors’ Confidence.
Note. OR = odds ratio; CI = confidence interval.
Basic Statistics = Descriptive statistics (e.g., means, frequencies) and basic univariate statistics (e.g., chi-square, t-test, ANOVA, correlation)
Advanced Statistics = Basic multivariate statistics (e.g., multiple logistic/linear regression), any advanced statistical techniques (e.g., nonparametric tests, polychotomous regression, principal component analysis, repeated measure analyses, mixed modelling, Poisson regression)
Many of the RPDs identified statistical training topics they would like to incorporate into their residency programs. The most common area was problem-based learning, 124 of 202 (61.4%), whereas least common was advanced statistical techniques (eg, nonparametric tests, polychotomous regression, principal components analysis, repeated-measures analyses, mixed modeling, Poisson regression), 78 of 202 (36.6%). For database development, study design, descriptive statistics, univariate analysis, and multivariate analysis, about half of all respondents, 93 of 202 (46%), 97 of 202 (48%), 92 of 202 (45.6%), 111 of 202 (55%), 107 of 202 (53%), respectively, wanted to incorporate these practices into their resident training. Forty-three of the respondents listed unique aspects of their resident research training program. Common themes included designated research/statistical mentor who may or may not be directly associated with the residency (pharmacist, physician, university employee), presence of statistical/research training program, focus on non-IRB/quality improvement projects, and identification of the need for more statistical training.
Discussion
To our knowledge, this study is the first description of statistical training practices in US pharmacy residencies. The responses from programs in this survey indicate that there is little statistical training and there may be a low level of confidence in many residents’ capabilities to perform statistical analysis. There are several possible explanations for limited statistical training, low confidence of residents’ statistical abilities, and low project publication rates. First, the residency research project represents the introduction to clinical research for many residents. A minority of pharmacy schools have a requirement for a research project built into their schools’ curriculum. 11 This means that the majority of residents entering their PGY1 residency understand research and statistical theory, but have not had the practical application that can help build proficiency. Because of this deficit in application, it may be impractical to expect complete confidence in all residents’ statistical capabilities after completion of their research project. An additional explanation for low project publication rates may be that IRB review is required for projects to be published. IRB approval can be a lengthy process at many institutions and may not be possible in the time constraints of a residency. This may relegate residents to more simple projects that are IRB exempt and do not necessitate advanced statistical training to complete the project and are less likely to be published.
Previously published effective attempts at improving resident research experiences have included developing a team of pharmacy preceptors with expertise in research to assist residents throughout their research process, including some training in statistics. 12 The University of Minnesota Pharmacy Residency Program described a mini-fellowship to help residents gain skill in statistical analyses. This program of weekly meetings throughout residency produced several manuscripts for publication. 13 Residency programs in New Mexico and Colorado have implemented research and statistical training programs with 7 sessions spread over the course of the residency year.14,15 An analysis of the Colorado program found that, compared with residents who were not involved in the program, there was an increase in residents’ confidence, but not knowledge of research methodology and biostatistics. 15 Other disciplines have also expanded training to improve statistical training and dissemination of research findings. A survey of internal medicine physician residents found implementation of a program including accessible biostatistics support increased publication and national presentations of resident physician research. 16
While having residency preceptors provide extensive statistical training to residents can be desirable, many preceptors are either unfamiliar with research requirements and statistical manipulations or do not have the necessary time to build a statistical training course for residents. In this case, another avenue to improve knowledge is to pursue a third-party statistical training program. The ASHP Foundation and the American College of Clinical Pharmacy (ACCP) both conduct research training programs; however, these are limited in number, highly competitive, and intended for those in academia, not residents.17,18 Both Stanford University School of Medicine and University of Cape Town offer an online statistical training course for medical students and residents.19,20 They are low to no cost ($0-$29) and cover the basics of biostatistics. The total time commitment for the learning sessions is 18 to 24 hours which is not insignificant with the already busy PGY1 schedule. However, select lectures could be audited to supplement residents’ education.
Statistical training should not be an afterthought in the residency process. In our sample, there was a significant relationship between publication rates and RPDs’ confidence. One explanation is that programs that dedicate more time and resources to research, and possibly statistics, are likely to have both higher confidences in its residents and higher publication rates. Causality cannot be determined from this survey, but this correlation may warrant further investigating. Stranges and Vouri assessed the predictive value of publication of residency research projects. They looked at 152 resident research projects, of which half were published and half were unpublished. The residents whose research projects were published were twice as likely to have publication success over the 5 years following residency than those residents whose projects were not published. This demonstrates the importance of providing quality research education during residency and the lasting effects this instruction may have. 21
This study was limited by a low response rate of 19.7%, but this response rate is not uncommon among online surveys sent to pharmacists.22-24 Potentially residency programs that did not complete the survey have a higher degree of statistical training associated with the residency program. The study population is not generalizable to all pharmacy residencies as only ASHP-accredited or candidate status residencies were included in this study. Precandidate or nonaccredited residencies may have different training practices for residents. However, this study did include community-based PGY1 residencies, which increased the heterogeneity of respondents. In addition, this study evaluates the subjective responses of residency directors and therefore limited by a response bias. There is currently no objective validated model for testing resident’s statistical knowledge. Future research is needed to identify the most effective methods of teaching statistical knowledge to residents.
Conclusion
A sampling of ASHP PGY1 RPDs found that statistical training for many residents is limited and many RPDs are not confident in their residents’ abilities to perform statistical analysis of research projects. Statistical training and opportunities to enhance research skills may be an area for future growth in pharmacy residency training programs.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.