A Practice Innovations Decision Model

Introduction

Health-care managers make decisions under conditions of uncertainty and in the absence of consensus about the relative merits of various outputs. To the extent possible, they base these decisions on research evidence or actual experience with the practice. They must also incorporate subjective judgments when hard evidence is lacking. This is called fuzzy decision analysis. A variety of methods have been developed for multiple criteria decision-making under conditions of uncertainty. ^1,2 This study uses one of the simpler approaches.

The purpose of this commentary is to propose a generic practice innovations decision model (PIDM) that relies on a combination of subjective and evidence-based probabilities. The usefulness of the model is illustrated by applying it to a particular case: the decision to open a retail clinic. The sensitivity of the result to key assumptions can be tested using sensitivity analysis and simulation methods.

The Model

A generic decision model that can be used for a variety of practice innovations would be useful to administrators. This could jump-start the analysis of options. It can also simplify the planning decision. In addition, a fuzzy model, which does not pretend that its elements can be measured with precision in advance of a decision, is realistic. The elements of the proposed model are as follows:

Acceptability to primary care providers is ok?

The decision about whether to open a clinic in a retail location offers a useful application of the model that serves to demonstrate its simplicity and flexibility. Retail clinics have opened in many locations across the United States and Canada. ³ Despite widespread skepticism from the clinical community, the research evidence has overwhelmingly supported the quality of services. ^{4 -6} The financial benefits have been substantial, as indicated by the rapid spread. These clinics generate orders for laboratory and ancillary services, just as do traditional clinics, but no reports indicate that the costs of such services differ from those incurred in traditional clinics. ^7,8 Patient volume clearly is adequate, as is patient satisfaction. ⁹

Other issues include space, public perceptions, and staffing. In some medical centers, the cost per square foot of space on campus is higher than the cost of renting retail space. Community scoring of health-care providers is publicly reported by some agencies. Whether retail clinics will achieve comparable scores to traditional clinics is unknown. However, since research evidence and clinical experience is supportive, there is no reason for pessimism. Perhaps the most serious potential problem is the availability of primary care providers to staff these clinics, including advanced practice nurses. As demand increases, salaries also can be expected to climb.

The previous paragraph provides a narrative summary of the elements of the PIDM as applied to the retail clinic problem. Probabilities can be assigned to each of the criteria based on the likelihood that they will be met. The probabilities can be derived from 3 sources: research evidence, practice experience, and subjective judgment.

These scores on the criteria are summed to form an index to guide decision-making after rounding. It is important to note that some fuzzy decision models use scoring methods that offer greater definition than 0 versus 1. In this model, 0 and 1 are used because no consensus is likely to be reached on more precise scores. However, future research could test the usefulness of different scoring methods.

As previously stated, the published research evidence supports the quality of clinical services, as measured by return visits, visits to EDs, and hospital admissions (criterion 3). Therefore, a probability of 0.9 can be assigned to this criterion.

The author was employed in a large medical care organization when retail clinics were opened. Based on this experience, I can say that criteria 4, 5, and 6 were met. The national trends indicate sufficient consumer demand to yield an adequate volume of visits (criterion 10).

Support from the clinical staff (criteria 1 and 2) is less certain, so lower probabilities should be assigned (eg., P = .5). This is a subjective assessment. It is based on the author’s experience. Some physicians were not enthusiastic about opening the retail clinic and some seized on a few examples of ineffectiveness to reject the entire concept. Resistance to change might be expected to differ in other medical care organizations, so the probability could be adjusted upward or downward.

Criterion 8 (availability of space) is always met prior to opening a clinic. However, rental rates are subject to renegotiation (P = .9). An unexpected increase in the rental rate seems unlikely, but it is not impossible.

Public websites report scores on clinical organizations. The accuracy of these scores might be debatable. Also, they may not be based on current information. Based on these considerations, assigning a probability is difficult. A subjective assessment has resulted in a .8 in this analysis for criterion 9, but other investigators may want to revise it.

The availability and affordability of clinical staff (criterion 7) is vital to the success of the retail clinic. As previously stated, demand has been increasing for primary care providers and advanced practice nurses. For the purposes of this example, a probability of .8 was used. However, the importance of the criterion combined with the uncertainty justifies a sensitivity analysis in which P is lowered, perhaps to .05.

Each criterion is scored as 1 (met) versus 0 (not met) by rounding. Possible scores range from 0 to 10. If .5 is rounded up, the simple deterministic solution is a sum of 10. If the probabilities are summed in a simple linear model, the final score equals the sum of the probabilities. The a priori criterion, set arbitrarily, was that managers should be confident that 8 or more criteria would be met if the clinic was opened. In other words, success was defined as meeting 8 or more criteria. Meeting less than 8 would be failure. This is comparable to getting a B or an A in a university class (80% or higher). Once again, users of the model could modify this decision rule depending on their aversion to risk.

The example leads to a score of 10 and to the conclusion that the retail clinic should be opened. However, lowering the confidence that staffing will be available makes this conclusion less certain. Monte Carlo simulation would show that while on average the decision will lead to the desired outcome, sometimes it may not. Failure might occur as often as 30% of the time. I leave the readers to develop their own estimates of failure rates.

Conclusion

The PIDM is a simple general model that may accommodate both subjective and evidence-based weights and could be applied to a variety of decisions. However, applied in a linear fashion, the model leads to conclusions that do not fully consider uncertainty. The contribution of this commentary is 2-fold: to propose the generic model and to recommend that it be applied with sensitivity analysis and Monte Carlo simulation. Neither the fuzzy decision models nor the use of sensitivity analysis with simulation is innovative. However, the combination is unique when applied to health services. Applied to the decision about opening a retail medicine clinic, the analysis suggests that if managers are comfortable with meeting 80% of criteria, with about a 30% chance of failure, they should open the clinic. However, this conclusion is predicated on the probabilities incorporated into the model. These should be reconsidered in each local application of it.

Most decision analyses of practice innovations are financial in character. The expected net present value of the investment is based on discounted costs and revenues. This approach assumes future events can be known with some degree of accuracy. It also assumes that only financial considerations are important. In PIDM, the 10 performance criteria are treated as being equal. Adequate visit volume and acceptable costs are assumed to generate acceptable financial results. Any medical care organization applying the model could weigh the criteria to emphasize financial concerns.

Future research projects could address the following questions: (1) Can the model be applied to other practice innovations, such as embedding specialists into primary care or opening a telemedicine service? (2) Is the model improved by changing the scaling of each criterion so that each is measured by more categories than simply met vs not met? (3) Would systematic reviews or meta-analyses increase the accuracy of some of the evidence-based probabilities? (4) Can variations of the model be developed? Are longer versions more helpful? (5) Should formal methods be used to develop the subjective probabilities, such as focus groups or surveys?

Despite its simplicity, the model illustrates a useful approach to analysis of practice innovation decisions. Further testing is needed and the model should evolve with use. However, combining subjective probabilities with evidence-based probabilities and incorporating uncertainty via simulation methods is an innovative strategy that could become important in health-care management.