Abstract
This study sought to identify factors that increase or decrease patient time with a physician, determine which combinations of factors are associated with the shortest and longest visits to physicians, quantify how much physicians contribute to variation in the time they spend with patients, and assess how well patient time with a physician can be predicted. Data were acquired from a modified replication of the 1997–1998 National Ambulatory Medical Care Survey, administered by the Kentucky Ambulatory Network to 56 primary care clinicians at 24 practice sites in 2001 and 2002. A regression tree and a linear mixed model (LMM) were used to discover multivariate associations between patient time with a physician and 22 potentially predictive factors. Patient time with a physician was related to the number of diagnoses, whether non-illness care was received, and whether the patient had been seen before by the physician or someone at the practice. Approximately 38% of the variation in patient time with a physician was accounted for by predictive factors in the tree; roughly 33% was explained by predictive factors in the LMM, with another 12% linked to physicians. Knowledge of patient characteristics and needs could be used to schedule office visits, potentially improving patient flow through a clinic and reducing waiting times.
Keywords
Recently there has been considerable interest in the time that primary care physicians spend with patients.1–7 Much of this interest stems from the perception that physicians have been increasingly pressured to see more patients and hence spend less time with some patients than would be ideal.8,9 Activities that would benefit some patients may not occur during office visits. Thorndike and others 10 observed that physicians counseled smoking patients to quit less often in 1995 than in 1993. Ruffin and others 11 documented disappointingly low rates of delivery for cancer screening tests. Tesar noted that the short times allocated for primary care office visits left as many as 50% of depressed patients undiagnosed. 12
Results from the National Ambulatory Medical Care Survey (NAMCS) indicate that the average patient time with a physician (based on ambulatory visits in general, not just primary care) plateaued after a sharp increase during the early 1990s. The average patient time with a physician in 2005 was 19.7 min, compared with 19.3 in 1995 and 16.7 in 1990.13–15 Despite the plateau, the fraction of patients spending more than 15 min with a physician increased from 33.8% in 1995 to 37.8% in 2000 and 42.9% in 2005.14–16 The average patient time with a family physician or general practitioner increased from 17.0 min in 2000 to 19.9 in 2005.15,16 Over the same period, primary care physicians were also facing increased pressures to see more patients. 17 Increased pressures to see more patients, and to do more for each patient seen, are inspiring new paradigms for health care delivery, such as the patient-centered medical home. 18
The preceding developments motivate research into factors influencing visit length. (For brevity, we often write “visit length” instead of “time with a physician”; we intend only that portion of the visit involving a physician.) Previous studies have examined the roles of specific factors, such as whether the visit was prepaid, 2 whether the patient had an acute illness or a chronic illness, 3 the patient’s gender, 4 and the patient’s age. 7 Blumenthal and others 1 considered several factors simultaneously and found that advanced age, psychological problems, and large numbers of diagnostic or screening tests were associated with longer visits. However, their data were from 1991 and 1992.
Our study, based on more recent data, uses 2 complementary multivariate statistical approaches to address 4 research questions about primary care practice: 1) What factors can increase or decrease patient time with a physician? 2) What combinations of factors are associated with the shortest and longest visits? 3) To what extent do physicians contribute to variation in visit length? 4) How accurately can visit length be predicted?
Besides addressing the 4 research questions, a second purpose of this study is to suggest that clinics can develop their own in-house statistical models for predicting visit length. These models can then be applied in deciding how much time to schedule for patients. Such models do not appear to be in widespread use for scheduling patients, despite previous literature calling for improvements in scheduling based on computer simulations. Hashimoto and Bell 19 considered scheduling and staffing in tandem, reporting that average patient time in a clinic was reduced from 75.4 to 57.1 min with operational changes informed by their results. Clague and others 20 identified a desirable ratio of new to follow-up patients and suggested spreading out new patient appointments. Elkhuizen and others 21 considered how to reduce the time between a patient requesting an appointment and the patient being seen.
This study makes a novel contribution for several reasons: multiple factors potentially related to visit length are assessed simultaneously rather than individually; the data are relatively recent; dual analytic strategies are used to answer research questions that cannot be addressed with a single statistical approach; strengths and weaknesses of the analytic strategies are explicitly compared; 1 of the analytic strategies (regression tree) is so underused in the health care literature that a demonstration of its capabilities is illuminating; and these analytic strategies can be applied by clinics in deciding how to schedule patients.
Methods
Data Acquisition
Our data were obtained by the Kentucky Ambulatory Network (KAN), which performed a modified replication of the 1997−1998 NAMCS involving 56 community-based primary care clinicians at 24 practice sites between May 2001 and June 2002. (There were 82 clinicians eligible to participate, of which 26 declined.) Participating clinicians met the following criteria as of December 1, 2001: 1) being in the KAN database of members or referred to us by a member; and 2) being a community-based clinician practicing primary care during the time of data collection. Each practice site submitted information describing a sample of office visits over a 1- to 2-week period, yielding a total of 2228 records. 22
Table 1 lists 22 potential predictors of visit length on which the practice sites submitted information. There were 1522 visits in which physicians were seen. The present analyses are based on the 1484 records with nonmissing and nonzero values for time with a physician, corresponding to 44 clinicians at 23 practice sites. The minimum, median, and maximum numbers of such records by clinician were 1, 39.5, and 73 respectively; the corresponding numbers by site were 23, 51, and 161. The vast majority of the clinicians were themselves physicians reporting how much time they spent with patients; a few were nonphysicians documenting how much time others spent with patients.
Potential Predictors of Time Spent With a Physician
Note: LPN = licensed practical nurse; NP = nurse practitioner; PA = physician assistant; RN= registered nurse.
All variables except Z16 through Z20 are coded as 1 = yes, 0 = no.
This pertains to all variables except Z16 through Z20.
This pertains to Z16 through Z20.
Clinicians in this study more often practiced in rural areas than Kentucky family physicians generally or US family physicians. However, previous analyses suggest that the KAN data are comparable to data from national primary care practice. 22
All data collected represented clinician perceptions about the patient visits, as in the NAMCS. The item for “time spent with physician” was the same as in the NAMCS form, for which the respondent entered the number of minutes: “Time spent with physician. If not seen by physician, enter zero.” Some NAMCS items were modified, and other items were added, because KAN clinicians believed the revised instrument would better capture the process of primary care practice. 22 Added items were height and weight to calculate body mass index and whether, in the clinician’s opinion, chronic pain was addressed and depression or anxiety contributed to the visit. Item modifications included allowing clinicians to report up to 6 (v. 3) patient complaints or reasons for visits, up to 8 (v. 3) diagnoses per visit, and up to 13 (v. 6) medications per visit. Because such factors may relate to visit length, these modifications make the KAN data particularly appropriate for studying visit length. The University of Kentucky Institutional Review Board (IRB) reviewed and approved the study. For some participating practices, their own IRB also approved the study.
Statistical Analyses
Two complementary analyses were performed since neither analysis alone would answer all 4 research questions. A regression tree and a linear mixed model were constructed to relate visit length to the potential predictors in Table 1.
A regression tree partitions observations into relatively homogeneous subgroups, and then the expected value of the response variable is estimated within each subgroup. 23 Here the subgroups are patient visits with similar values on some of the predictors.
We divided the 1484 records into training (742 records), validation (371), and test (371) data sets 24 with the RANSPLIT macro for PC-SAS. 25 The training data set is used to create a hierarchy of trees. The first tree assigns all observations to a single subgroup, the second allocates each observation to 1 of 2 subgroups, the third places each observation into 1 of 3 subgroups, and so forth. Each tree is constructed so the average squared error for predicting the response variable within the training data set is minimized, subject to the constraint on the number of subgroups and the requirement that the next tree in the hierarchy be created by splitting 1 of the existing subgroups from the previous tree. The validation data set chooses the tree with the best potential for predicting the response variable in a new data set; the tree is chosen whose average squared error for predicting the response variable within the validation data set is minimal. The test data set evaluates the chosen tree; the average squared error for predicting the response variable within the test data set is computed for whichever tree was selected by the validation data set. Enterprise Miner for Version 9.1 of PC-SAS was used for this entire process.
A linear mixed model (LMM) is a generalization of a linear regression model. 26 Like a linear regression model, an LMM has fixed effects. These are coefficients corresponding to variables for which the values observed in the present study are essentially all possible values, such as “yes” or “no” for whether depression/anxiety contributed to the visit. Unlike a linear regression model, an LMM also has random effects distinct from the error terms. These are coefficients corresponding to variables for which the values observed in the present study are like a random sample of all possible values, such as the identity of a physician or practice site. Thus, the LMM includes not only predictors describing visit characteristics but also dummy variables for each physician and practice site. The coefficients for such dummy variables are not customarily reported, since they are not meaningful outside the pool of physicians and practice sites present in the data set used to fit the LMM. Instead, the coefficients for such dummy variables are summarized by their variances. This allows us to estimate how much physicians and practice sites contribute to variation in visit length, even though specific characteristics of physicians, such as gender or years of experience, are not among the predictors in the LMM.
Random effects also enable the LMM to capture dependencies among multiple values of the response variable associated with a particular physician or practice site. 27 For example, suppose that 2 patients are seen for 10 and 15 min respectively by a physician whose random effect is 0. If these same patients had instead been seen by a physician with random effect 5, their projected visit lengths would have been 15 and 20 min, respectively. The common addition of 5 min in both cases indicates that visit lengths associated with the latter physician are dependent and, more specifically, tend to be of greater duration than visit lengths to the former physician.
Since random effects are assumed normally distributed with mean zero, their presence in the LMM does not preclude predictions for patient visits to physicians and practice sites absent from the data set used to fit the LMM. Such physicians and practice sites are treated as though they had zero random effects. However, a nonzero random effect for a physician or practice site in the data set can be used in making predictions for patient visits to that physician or practice site.
Predictors were selected and interactions specified for the LMM in a 2-phase procedure. First, the regression tree and an exploratory linear regression model constructed with Enterprise Miner were used to develop a preliminary LMM. Second, the preliminary LMM was refined through deletion of predictors not achieving statistical significance and stepwise addition of new predictors attaining statistical significance. Nonsignificant main effects were retained for predictors involved in significant interactions. Following standard conventions for an LMM, we used all 1287 records with nonmissing values on the selected predictors to estimate parameters and carry out hypothesis tests; we did not use training, validation, and test data sets. Statistically significant P values were those less than 0.05. The LMM was fit with PROC MIXED in version 9.1 of PC-SAS.
Sensitivity Analyses
We fit a second regression tree and LMM after excluding the 51 visits in which a physician assistant (PA), nurse practitioner (NP), or nurse midwife was seen along with the physician (or in which this information was missing). These “sensitivity analyses” reveal how much such visits influenced the original regression tree and LMM.
Results
Visit Characteristics
Figure 1 displays the times spent with a physician in the 1484 office visits. The average time was 14.5 min (s 8.0, minimum 3, maximum 65). Summary statistics for potential predictors appear in Table 1. The average number of diagnoses was 2.3 (s 1.3, minimum 0, maximum 8). For the vast majority of visits (88.9%), patients had been seen previously by the physician or someone at the practice. For a slim minority of visits (12.7%), patients received non-illness care, of which 1 example would be pregnancy supervision.
Histogram of times spent with a physician in 1484 office visits. The average patient time with a physician was 14.5 min, but some visits were shorter and others were considerably longer. The regression tree and LMM were constructed to help explain the heterogeneity depicted in Figure 1.
Regression Tree Analysis
The regression tree is depicted in Figure 2; variable symbols are defined in Table 1. To make a prediction, one asks whether Z18 is less than 2.5 (whether there were fewer than 3 physician diagnoses). If no, one proceeds along the right branch at Split 1 and asks whether Z15 is less than 0.5 (whether non-illness care was not received); if yes, one proceeds leftward. One continues asking questions until arriving at a terminal node. The number inside the terminal node is the prediction. Missing values on the predictors are accommodated by “surrogate rules” (not portrayed in Figure 2). For instance, if the value of Z18 is missing, then a branch is selected at Split 1 based on whether Z20 is less than 2.5 (whether there were fewer than 3 medications). The variables involved in surrogate rules may or may not appear elsewhere in the tree. The implication of surrogate rules is 2-fold: none of the 1484 records was discarded in the process of constructing and evaluating the tree, and future visit lengths can be predicted even if some visit characteristics are unknown.
Regression tree for predicting time spent with a physician. The top node of the regression tree branches into 2 nodes at Split 1, these 2 nodes branch into 4 nodes at Split 2, and so forth. One follows the left branch if the stated logical condition is true; otherwise one follows the right branch. Nodes from which no branches emanate are called terminal nodes. The 23 terminal nodes define homogeneous subgroups of patient visits. For example, taking the right branch at Split 1 (Z18 < 2.5 is not true), the right branch at Split 2 (Z15 < 0.5 is not true), and the left branch at Split 3 (Z6 < 0.5 is true) reveals a homogeneous subgroup of patient visits in which there were 3 or more physician diagnoses, non-illness care was provided, and the patient did not see his or her own primary care physician. The average visit length in this subgroup was 41.9 min. The symbols Z1 through Z22 are as defined in Table 1. The coloring scheme of the nodes is that darker colors correspond to longer predicted visits, so one can see at a glance which combinations of factors are predicted to yield extremely long, moderately long, moderately short, and very short visits.
The terminal nodes represent 23 relatively homogeneous subgroups of patient visits within the training data set. The largest subgroup encompassed 296 visits with patients who were seen previously by the physician or someone at the practice, had 2 or fewer diagnoses, had 3 or fewer medications, and were not receiving non-illness care. Their average visit length was 11.1 min (s 5.1). The subgroup with the longest average visit (41.9 min, s 10.3) included 8 visits with patients who were not seen by their primary care physicians, had between 3 and 8 diagnoses, and were receiving non-illness care. The subgroup with the shortest average visit (8.2 min, s 4.3) comprised 11 visits with patients who were seen previously by the physician or someone at the practice, had between 3 and 8 diagnoses, saw a PA/NP/nurse midwife along with the physician, and were not receiving non-illness care.
Evaluation of the tree entailed making a prediction for each visit in the test data set and comparing the prediction to the actual visit length. The typical prediction error (the square root of the average squared error) was 6.3 min. Approximately 38% of the variation in visit length was explained by predictors in the tree.
Linear Mixed Model Analysis
Table 2 presents estimated coefficients of the LMM. The number of diagnoses not only influences visit length but also interacts with several other predictors. Expected visit length is changed by an estimated
minutes with each additional diagnosis. To illustrate Equation 1, each additional diagnosis would increase expected visit length by 4.6 = 1.596 + 1.976 + 0.327 × 3 min, if the patients had not been seen previously by the physician or someone at the practice (Z7 = 0), were not receiving routine care for a chronic problem (Z12 = 0), were receiving non-illness care (Z15 = 1), and had 3 complaints/reasons for visits (Z19 = 3).
Linear Mixed Model (LMM) Results Relating Predictors to Expected Visit Length
Note: NA = not applicable; NP = nurse practitioner; PA = physician assistant.
The P value pertains to a test of the null hypothesis that the present coefficient is zero.
The P value pertains to a test of the null hypothesis that all coefficients associated with the present predictor, including those for interaction terms, are zero. Thus, for example, even though the −1.044 coefficient estimate for Z7 (previously seen by physician or someone in practice/department) has a P value of 0.322, the −1.044 and the −0.927 coefficient estimate for interaction of Z7 with Z18 (no. of physician diagnoses) collectively have a P value of <0.001. The latter P value is more appropriate for evaluating the overall relevance of Z7 to the LMM.
Table 2 can be used to derive an expression like Equation 1 for any other predictor in the LMM. For instance, each additional complaint/symptom changes expected visit length by an estimated −0.458 + 0.327 Z18 min.
Table 3 summarizes the variation in visit length. Approximately 33% of the variation was explained by predictors in the LMM. About 13% of the variation not explained by predictors (~9% of the total) was ascribed to practice sites, roughly 19% (~12%) was linked to physicians, and approximately 68% (~46%) could not be attributed. The intraclass correlation coefficient (correlation between the lengths of 2 randomly selected visits to the same physician at the same practice site) was an estimated 32% = (100% − 68%). If we made predictions with the LMM relying solely on the predictors of visit length and disregarded the random effects, as would be necessary if the physicians and practice sites were not in the original data set used to fit the LMM, then the typical prediction error would be 6.5 min. This could be reduced to 5.4 min if predictions were tailored to individual physicians and practice sites using the random effects, which would be possible if the physicians and practice sites were in the original data set.
Linear Mixed Model (LMM) Results Describing Variation in Visit Length
Total variation in visit length (63.9) partitions into the sum of variation explained by predictors (21.3) and variation not explained by predictors (42.6).
Variation not explained by predictors (42.6) partitions into the sum of variation linked to practice sites (5.5), variation linked to physicians (7.9), and variation linked neither to practice sites nor to physicians (29.2).
This represents a typical prediction error if we predict that each patient will have a visit length of 14.5 min.
This represents a typical prediction error if we use the results from Table 2 to predict visit length for each patient.
This represents a typical prediction error if we use the results from Table 2 as well as random effects for practice sites and physicians (not shown) to predict visit length for each patient.
Sensitivity Analyses
The second LMM, constructed from visits in which a PA/NP/nurse midwife was not seen (thus omitting 51 out of 1484 records), was similar to the original. Whether a PA/NP/nurse midwife was seen could no longer be used as a predictor, but all 7 other predictors and all 4 interactions from the original LMM retained statistical significance. Also, 32% of the variation in visit length was explained by predictors, 12% by physicians, and 10% by practice sites. Typical prediction errors were 6.6 min if random effects were not used and 5.4 min if they were.
The second regression tree largely resembled the original. Except for Z16 (age) and Z21 (saw PA, NP, or nurse midwife along with physician), the second tree included the same predictors. However, the second tree based Split 1 on Z15 (non-illness care) and Split 2 on Z18 (number of physician diagnoses). Still, the second tree led to similar conclusions about what combinations of factors led to the longest visits: patients receiving non-illness care, not seeing their primary care physicians, having 3 or more medications, and having 4 or fewer diagnoses had an average visit length of 41.0 min, whereas those receiving non-illness care and having 5 or more diagnoses had a corresponding figure of 40.2 min. A typical prediction error was 6.6 min.
Discussion
The novel combination of regression tree and LMM approaches addresses all 4 research questions. As summarized in Table 4 and described below, the tree yielded responses to questions 1, 2, and 4 and the LMM furnished answers to questions 1, 3, and 4. What follows is based on the original analyses; as noted above, the sensitivity analyses were mostly concordant.
Summary of Answers to Research Questions
Note: LMM = linear mixed model; LPN = licensed practical nurse; NP = nurse practitioner; PA = physician assistant; RN= registered nurse.
Factors Increasing or Decreasing Patient Time With a Physician
The most important predictor of visit length was the number of diagnoses. This factor determined Split 1 in the tree; if we stopped after Split 1, then patients with 3 or more diagnoses would be predicted to spend 4.7 min more with physicians than patients with 2 or fewer diagnoses. The LMM indicated that each additional diagnosis could increase predicted visit length by up to 5.5 min, depending on other visit characteristics. That the number of diagnoses influences visit length is intuitively reasonable, but our analyses have gone beyond intuition to quantify the relationship between the number of diagnoses and visit length. Quantifying this relationship is valuable for practical purposes such as scheduling patient visits. Indeed, clinics could replicate our efforts with their own data and develop in-house statistical models for predicting visit length. When a patient calls to request a visit, the receptionist could collect information from the patient and, for a returning patient, use the patient’s medical record to supply inputs to the statistical models, which would yield a suggested amount of time to schedule for the visit.
A second factor prominent in both the regression tree and LMM analyses was whether a patient received non-illness care. To understand why non-illness care may increase visit length, we reviewed the ICD-9 codes for the primary diagnoses of patients receiving non-illness care. Many of these patients had codes of V20.2, V22.1, and V70.0. Thus, infant/child health checks, supervision of pregnancy, and routine general medical examinations entail longer visits.
A third factor common to both analyses was whether a patient had been seen previously by the physician or someone at the practice. Hence, continuity of care is not only a contributor to patient satisfaction and better health outcomes but also a key to efficient time management in a practice.
However, there were differences between the regression tree and LMM analyses. The tree suggested that for some patients, visit length would decrease if they saw their own primary care physicians or would increase if they had more medications; these factors did not appear in the LMM. The LMM indicated that visit length would increase when depression/anxiety contributed to the visit and, for some patients, would decrease with routine care for a chronic problem; these factors were absent from the tree.
The differences between the 2 analyses may be understood via the hierarchical nature of the regression tree: we ask a sequence of questions, each pertaining to 1 of the predictors, but the order of the questions matters because the next question asked and hence the next predictor consulted depend on the answer to the current question. Thus, the path taken through the tree and the specific predictors consulted vary from patient to patient. This hierarchical structure helps the tree to identify interactions, in which a factor greatly affects visit length for some but not necessarily a large number of patients, without requiring that the investigator anticipate or explicitly test for them. To illustrate, continuity of care at the physician level affects visits involving non-illness care and 3 or more diagnoses, whereas continuity of care at the practice level is important when the primary reason for the visit is other than non-illness care. The LMM does not have a hierarchical structure and consults the same set of predictors for every patient. An investigator may place interactions in the LMM, but considering all possibilities (especially if one does not limit oneself to 2-way interactions) is impracticable. The LMM does well at identifying factors that affect visit length for most or all patients, such as depression/anxiety contributing to the visit, even if that impact is modest.
One may wonder whether the regression tree and the LMM should be forced to use the same predictors. We believe not, for 2 reasons. First, predictions from the LMM would not necessarily be optimized with exactly the same set of predictors that works best for the tree, or vice versa. Requiring one method to use the optimal predictors for the other method would place the former method at a disadvantage, biasing the comparison in favor of the latter method. Second, forcing the LMM to use the same predictors as the tree could impede the discovery of factors modestly affecting visit length for many patients, whereas forcing the tree to use the same predictors as the LMM could prevent the identification of factors greatly affecting visit length for a few patients. The 2 analyses together paint a more complete picture about the factors increasing or decreasing visit length than would be possible with either analysis alone.
Shortest and Longest Visits
The regression tree revealed that a short time spent with the physician was likely when the patient, besides having been seen previously by the physician or someone at the practice and not receiving non-illness care, saw a PA/NP/nurse midwife along with a physician. The PA/NP/nurse midwife might deliver many of the services required by such a patient, so the patient would divide time between the PA/NP/nurse midwife and the physician. A long physician visit was likely when the patient, along with having a large number of diagnoses and receiving non-illness care, did not see his or her own primary care physician. Such a patient could require more time because an extended dialogue might be necessary for the physician to assess the patient’s medical history and determine appropriate treatment.
Variation Associated With Physicians
Almost a fifth of the variation in visit length not explained by predictors in the LMM was linked to physicians. About half of physicians have random effects greater in magnitude than 1.9 min, meaning that these physicians increase or decrease visit length by at least 1.9 min; about a fifth of physicians add or subtract at least 3.6 min. Whether a given physician inclines toward shorter or longer visits is potentially useful information for scheduling patients with that physician.
Accuracy of Predicted Times
A typical prediction error is 6.5 min with the LMM, 6.3 min with the tree, and 8.0 min (the standard deviation of visit length in the full data set) if neither is used. The 6.5 min with the LMM decreases to 5.4 min if the physician and practice site are among those in our data set. Given that a clinic can create its own in-house LMM with random effects for its physicians, a typical prediction error of 5.4 min may be attainable rather generally. Furthermore, although the difference between 8.0 min and 5.4 min may seem modest, there is a substantial cumulative effect over the course of a day in which a physician sees, say, 25 patients; we will say more about this below when we speak of a computer simulation that we conducted.
Rules of Thumb
To assist in the interpretation of results, we specify straightforward rules of thumb that approximate the predictions of the regression tree and the LMM. By confining attention to Splits 1, 2, and 3 in the regression tree and rounding, we approximate its predictions as follows: For individuals not receiving non-illness care, begin with an assignment of 12 min and add 4 min if they have not been seen at the practice before, 3 min if they are anticipated to have many diagnoses, and 4 min more if both of the above apply. For individuals receiving non-illness care, assign 42 min if they are anticipated to have many diagnoses and are not seeing their own physicians, 23 min if they are anticipated to have many diagnoses but are seeing their own physicians, 26 min if they are anticipated to have few diagnoses but require many medications, and 15 min if they are anticipated to have few diagnoses and require few medications.
By rounding we approximate the predictions of the LMM as follows: Begin with an assignment of 12 min and add 3 min for every 2 anticipated diagnoses, 2 min if depression/anxiety contributed to the visit, and 1 minute for an adult patient or 2 min for an elderly patient. Subtract 3 min from physician time if a PA/NP/nurse midwife is also to be seen. Let D denote the anticipated number of diagnoses. Subtract (1 + D) min if the patient has been seen at the practice before, add (1 + 2D) min if the patient is receiving non-illness care, add (D − 1) min for every 3 complaints/symptoms, and subtract (D − 1) min if the patient is receiving routine care for a chronic problem.
Merits of the Two Statistical Approaches
Strengths of the regression tree are that 1) interactions between predictors can be captured without being anticipated in advance; 2) records with missing values on predictors can still be used to fit the tree; 3) predictions of future visit lengths can be made even if some visit characteristics are unknown; and 4) combinations of predictor values yielding long or short visits are readily apparent. Weaknesses are that 1) a different initial division of records into training, validation, and test data sets could produce a somewhat different tree; 2) the identities of physicians and practice sites cannot be included as predictors since this would render the tree incapable of making predictions for visits to physicians and practice sites not in the training data set; and, consequently, 3) the tree cannot assess how physicians and practice sites contribute to variation in visit length.
Strengths of the LMM are that 1) P values are readily available and can be used for model building in lieu of training, validation, and test data sets; 2) the identities of physicians and practice sites can be taken into account via random effects; and, therefore, 3) the LMM decomposes variation in visit length into parts unexplained and explained by visit characteristics, physicians, and practice sites. Weaknesses are that 1) unanticipated interactions are easily missed because testing for all possible interactions is impracticable, so that fitting a tree first may be necessary to identify some of the important interactions for inclusion in the LMM; 2) observations with missing values on 1 or more of the predictors in the LMM are discarded when fitting the LMM; 3) prediction of future visit lengths may be difficult if some visit characteristics are unknown; and 4) combinations of predictor values yielding long or short visits are not apparent.
Limitations
The data do not indicate whether the patients were fluent in English (or other languages spoken by the physicians), even though language issues may underlie variation in visit length.28,29 Furthermore, the data are from Kentucky, so predictions from the tree and LMM may not apply to family medicine practice in all parts of the United States. Another limitation is that not all factors relevant to visit length are known in advance. For instance, the number of diagnoses emerges during the visit. Even so, a reasonable guess for the number of diagnoses can be made based on a patient’s record or information acquired when an appointment is requested. Also, the number of diagnoses may not yield as good a prediction as the specific nature of the diagnoses. However, the specific nature of the diagnoses is more difficult to guess than the number of diagnoses when an appointment is requested.
Implications for Primary Health Care
Clinics can develop their own in-house statistical models for predicting visit length. Depending on what information is maintained in patient records at a particular clinic, the in-house statistical models may consult some variables that were not available to us in the present study (including some attributes of the physician, like the physician’s age or years of experience) and thereby attain even lower prediction errors than documented herein. The random effects of the in-house LMM can be used to tailor its predictions to individual physicians at the clinic since the physicians with whom visits would be scheduled would be the same physicians in the data set used to construct the LMM. As we have seen in the present study, the prediction errors from the LMM are reduced when random effects are considered. Once the in-house statistical models have been developed, they can be incorporated into a user-friendly spreadsheet program that the receptionist can operate whenever a patient requests an appointment.
The benefits of using the tree or the LMM to schedule office visits emerge as we consider scheduling, say, 25 consecutive visits to the same physician. We conducted a computer simulation to estimate the probability that at least 1 patient out of 25 would wait more than 30 min beyond his or her appointed time. (We used version 2.9.0 of the R statistical software package; code is available from the corresponding author.) When all 25 patients were allocated 14.5 min, corresponding to the average visit length in our data set, there was an estimated 39% probability of at least 1 patient waiting more than 30 min. However, the 39% declined to 22% when patients were allocated the times suggested by the LMM. Although the simulation was simplistic (neglecting, for instance, the possibilities that a patient would show up late or not be ready if the physician were running ahead of schedule), there is nonetheless an indication that knowledge of patient characteristics and needs could improve patient flow through a clinic and reduce waiting times.
There remain some obvious practical challenges to the development and use of in-house statistical models for predicting visit length. Besides the limitations noted earlier, some clinics might not have anyone on staff with the training to develop the statistical models. Although simply adopting the statistical models exactly as they were presented in this article would be one option, another possibility would be to retain the services of a statistical consultant. If budget considerations then limited a clinic to 1 statistical model, we suggest the LMM over the regression tree. The LMM’s predictions could be tailored to individual physicians (via random effects) and thus might be superior to the tree’s predictions. In contrast, the LMM’s predictive capabilities might be somewhat reduced if the LMM were constructed without the insights about interactions that might be acquired from fitting the tree first. Therefore, if a clinic could fit both statistical models, we recommend doing so. A clinic should then use for scheduling purposes whichever model had lower prediction error, except that there might be some circumstances under which a clinic could reasonably alternate between the statistical models. For instance, the LMM might emerge as the default choice, but the tree could be called upon to help schedule visits when patients had missing values on predictors appearing in the LMM. In any event, once the statistical models were developed, a careful evaluation would be required to assess whether patient waiting times were actually reduced and whether the receptionist retained efficiency in scheduling appointments while collecting information for use in the statistical models. Finally, we hope that clinicians, statisticians, and other decision makers will continue the discussion we have begun here, especially regarding how to address the practical challenges identified above as well as any other implementation issues that may be revealed as this line of research progresses.
Footnotes
Acknowledgements
We thank Dr. Robert Hamm and 4 anonymous reviewers for feedback leading to substantial improvement of this manuscript.
There is no external funding source to acknowledge.