From Data to Improved Decisions: Operations Research in Healthcare Delivery

Deterministic Optimization Models

Optimization is a methodological approach often considered to be a core part of what defines the field of OR. Optimization models comprise 3 major components: objective function(s), decision variables, and a set of constraints. An objective function provides a mathematical representation of key performance measures for different choices of decision variables, the value of which are limited by the constraints on a set of feasible choices. The main deliverable of an optimization model is the values of the decision variables that optimize the key performance measure.

Deterministic optimization models are suitable for problems where decision alternatives must be selected to optimize system-wide objectives, which are often subject to constraints (e.g., scarce resources). In these models, the inputs and mechanisms connecting inputs and decision alternatives to consequences are known with certainty. For example, if a specific treatment changes the health condition of the patient predictably, a deterministic model will assume that the treatment impact on the health condition is known with certainty before the treatment initiation decision. However, such relationships are sometimes ambiguous in healthcare. In these circumstances, if a deterministic optimization model provides a good approximation, the resulting model will be easier to implement and interpret. Linear, integer and non-linear programming are the most commonly used deterministic optimization methods.

Linear programming is a mathematical optimization method where the objective function and constraints are linear functions and decision variables can take any numerical value (e.g., dose of a treatment).^15,16 Integer programming is a special case of linear programming where decision variables are restricted to integers (e.g., number of medications as part of a treatment plan). ¹⁷ The discrete nature of these problems often requires an entirely different approach to an algorithmic solution, and many integer programs are computationally challenging, or even intractable. Non-linear programming relaxes the assumption that the relationships between the objective function, decision variables, and constraints must be linear. For example, if a decision problem aims to find the optimal quantity of medications as part of a treatment plan to maximize health benefits, the health benefits may increase with the number of medications in a nonlinear fashion. ⁵ In healthcare, deterministic optimization models are frequently applied to treatment decision making, such as radiation therapy, ⁵ and scheduling, such as surgery scheduling ¹⁸ and staff scheduling. ¹⁹

Stochastic Models

Healthcare decisions inherently involve uncertainty, which can arise from several sources including underlying physiological processes that are not fully observable, error in diagnostics tests, or variations in outcomes from medical interventions. ²⁰ Furthermore, the diversity of patients contributes to variability in individual responses to treatments, where a treatment may work perfectly for some patients but fall short for others. Stochastic models may offer an appropriate solution if the consequences of available decisions are not known with complete certainty at the time of decision making.

Decision trees, Markov models, and simulation are among the widely used stochastic models. A decision tree, originating from the field of decision analysis, is a tree structure in which branches at decision nodes represent decision alternatives, and branches at chance nodes represent competing chance events. ²¹ Costs, life years, or other performance measures can accrue as successive branches are traversed. It provides a visual display of a decision problem that allows the modeling of risk.

Markov models are mathematical models that represent a system that evolves over time by probabilistically transitioning from one state to another.^22,23 Here, the term state refers to any description of the system at a given time point. For example, a state may be a combination of the patient’s age and the presence of specific clinical symptoms. In a Markov model, the future behavior of the system depends on the past only through the current state of the process. Markov Decision Process (MDP) models are an extension of Markov models in which each state has associated decision alternatives that can influence the subsequent behavior of the system.^22,23 For example, an MDP can represent a patient’s disease progression, where initiating certain treatments for low-acuity conditions may impact how the disease progresses by decreasing the probability of progression to a higher acuity state. The goal of an MDP model is to select optimal decisions in each state while considering immediate and subsequent costs or rewards.

If the model is simple enough, mathematical optimization models can provide an exact analytical solution. Simulation models are suitable for complex decision problems when an analytical solution requires extensive computing time and resources. Colloquially, “to simulate” means to approximate reality. In the context of OR, simulation refers to the use of computer algorithms to emulate the operations of a system that is modeled numerically with assumptions formulated as mathematical or logical relationships. ²⁴ In OR, when used without a qualifier, simulation usually refers to discrete-event simulation. Other kinds of simulation in OR are referred to with a qualifier, e.g., Monte Carlo simulation, agent-based simulation. ²⁵ One sometimes sees authors state that a stochastic or Markov model simulates a specific system. Here the meaning is generic, referring to the fact that any model simulates reality, and does not refer to discrete-event, Monte Carlo, or agent-based simulation.

Discrete-event simulation refers to the modeling of a system over time by capturing how the state changes at discrete time points. In this context, a system is a well-defined collection of entities that are characterized by attributes, the state of which may change through instantaneous events. For example, in an outpatient clinic, the providers and patients can be the entities with the attributes “service status” (busy or idle), “time of arrival,” and “health state,” respectively. Examples of system characteristics that can be estimated include, but are not limited to, expected utilization of providers defined as the expected proportion of time that a provider is busy delivering care to a patient, or maximum waiting time of patients with a certain health state until they receive care. ²⁴

Monte Carlo simulation is a methodology that is broadly applicable to many problems. It uses random numbers to generate a sequence of independent and identically distributed observations of a quantity to approximate the mean of that quantity.^26,27 Monte Carlo simulation models are commonly classified into first-order (i.e., microsimulation) and second-order models (i.e., stochastic sensitivity analysis). ²⁸ Microsimulation and stochastic sensitivity analysis are commonly used in healthcare applications. Microsimulation consists of simulating sample paths of the healthcare trajectories of individual patients over time by utilizing random numbers and computing performance measures, such as cost or gain for the sampled trajectories, where the average of the performance measure serves as an estimate of the mean performance measure for the system.^26,27 Stochastic sensitivity analysis addresses parameter uncertainty by treating model parameters as random variables using prior distributions associated with the parameters. The resulting distribution of the performance measure over many samples is used to characterize model sensitivity.

Agent-based simulation is used to model systems with numerous independent agents interacting with each other and the environment. The behaviors and interactions of the agents may be described by mathematical equations or decision rules. Agent-based simulation is particularly suitable in healthcare decision problems where numerous agents (e.g., patients, providers, policy makers) interact and collaborate to achieve certain healthcare goals.

Another major category of stochastic models includes queueing models. Queueing models are suitable for problems in which customers arrive at a system to receive a service, wait in queue to be served, and generally leave the system after the service is completed. ²⁶ The goal of such models is generally to analyze queue lengths and waiting times given uncertainty in arrival and service times of customers. An example of a queueing model in medical decision making is the organ transplant waiting list, where patients (i.e., customers) wait for an organ to become available for transplantation (i.e., service). ²⁹ An analysis of this queueing model can provide insight into the performance of the transplant waiting list and evaluate the organ allocation system. ²⁹

Practical Approaches to Optimization

Deterministic and stochastic modeling include a wide range of tools that can address complex problems. In practice, the type of modeling is often chosen based on the problem structure, availability of data, and computational burden, among other aspects. A common practical approach is to combine both deterministic optimization and simulation to address the problem of interest. For instance, first, a deterministic model of the system is formulated and optimized using mathematical programming methods, followed by a simulation of the system to evaluate the optimal solution from the deterministic model. Often this solution is compared with results of a simulation for common practice or the status quo to establish whether the solution obtained from the deterministic model leads to improvement. This approach is often used for practical purposes, as optimizing the simulation may be too computationally burdensome or intractable.

Diagnostic and Treatment Decisions

OR provides rigorous approaches for describing problems at the individual level by capturing patients’ preferences, utilities, attitude to risk, among other factors, to inform medical decisions in settings where clinical evidence is insufficient.^32-34 For example, Lobo and others ³⁵ used decision analysis to examine prostate cancer treatment decisions in the post-prostatectomy setting. They compared usual care with genomic classifier estimates of cancer progression to guide the use of therapies with radiation and hormones. A Markov model of cancer progression was evaluated by simulating individuals from 2 prostate cancer cohorts, where each simulated patient had individual estimates of cancer progression informed by a genomic classifier. The goal of the study was to find the care strategy that maximizes quality adjusted life years (QALYs) and the 5- and 10-year cancer progression outcomes. The multidisciplinary nature of the approach was highlighted using statistical methods to conclude that treatment decisions based on the genomic classifier risk of cancer progression was associated with statistically significant improvements in outcomes.

OR has also been used for determining the optimal time of a diagnostic intervention (e.g., screening test). For instance, Ayvaci and others ³⁶ studied breast cancer diagnostic decisions to choose the optimal intervention—routine follow-up mammography in a year, short-term follow-up mammography in 6 months, and biopsy (i.e., pathological examination of the breast tissue obtained by needle biopsy or surgical excision)—based on a patient’s risk assessment following mammography screening. The study focused on the tradeoff between choosing biopsy when the risk level is low (which may result in a false-positive test result) and delaying biopsy (which may impact the timely detection of cancer). The decision problem was formulated as an MDP with the goal of maximizing the patients’ QALYs while considering the impact of budget constraints on the optimal decision. Ayvaci and others found that the risk threshold for the optimal decision changed as a function of patients’ age and budget constraints. Their findings highlighted the importance of individualized screening policies.

As for optimizing the time of a treatment intervention, Denton and coworkers ³⁷ focused on determining the optimal time for patients with type 2 diabetes to initiate statin therapy to lower cholesterol and reduce the probability of heart attack or stroke, considering the uncertainty in how patients’ cholesterol levels change over time. The authors developed an MDP in which a Markov model defined transitions among states categorized by cholesterol levels and the occurrence of heart attack, stroke, or any other cause of mortality. The decision for each year was whether to initiate drug treatment or to delay treatment for an additional year. Their study provided the optimal decision in each possible health state to maximize QALYs.

Patient Preferences

Patient-reported features, such as patient preferences and satisfaction, are recognized as key factors contributing to successful care delivery. OR models can take patient preferences into consideration as part of an optimization framework. For instance, Yan and others ³⁸ considered integrating pregnant women’s preferences associated with Down syndrome (DS) screening focusing on 2 adverse outcomes: birth of a baby with undetected DS and a fetal loss due to screening/diagnostic procedures. DS screening outcomes were evaluated against set risk-cutoff values. The study focused on the tradeoff that a low risk-cutoff value may result in a high DS detection rate but may also lead to more unnecessary diagnostic procedures. The authors developed a Monte Carlo simulation model to determine the optimal risk-cutoff value for DS screening by considering patient preferences represented by the relative weight assigned to the outcomes. Their results showed that the optimal cutoff values could be significantly different than the one-size-fits-all threshold, depending on patient preferences, consent rate to procedures given positive results of screening tests, and procedure-related fetal loss rate.

Organ Transplantation

Discrepancies in organ allocation based on geographic location of the recipients are a major challenge for deceased-donor organ access and donor-recipient matching within the United Network of Organ Sharing (UNOS). Considering the deceased-donor organ shortage and increased number of candidates for transplantation, geographic discrepancies result in inequity of waiting times and pretransplant mortality. ³⁹ Gentry and others ⁴⁰ studied the impact of geographical redesign, which involved a regrouping of donation service areas (DSAs) into regions to enhance the equity of liver allocation as measured by equity metrics including variance of the Model for End-stage Liver Disease (MELD) score at the time of transplantation among DSAs, and summative metrics such as waitlist deaths per year. They used discrete-event simulation to obtain outcome metrics associated with organ allocation prioritization. The outcomes of the geographical redesign under different local and regional prioritization scenarios were measured using integer programming. Their results showed that, without optimization, there is a tradeoff between reducing disparity and reducing waitlist deaths, and that the tradeoff is avoided when optimization models are used. The optimal maps resulted in a reduced number of regions that are more equitable compared with the current policies. This study is an example of a commonly used OR approach to formulate a deterministic model of the system, to optimize by using mathematical programming, and to evaluate the optimal solution using simulation, e.g., comparing with the status quo.

Another challenging aspect of organ transplantation is the matching of donors to recipients. Segev and others ⁴¹ studied decisions about how to match donors and recipients for kidney transplantation. The optimization aspect of the study focused on the large number of potentially feasible matches of donors and recipients depending on blood type and crossmatch compatibility. They formulated this optimization problem as a matching problem. Each possible match among an exponentially large number of possible matches of donors to recipients was evaluated based on a score that considered various characteristics of a match, such as the quality of the match, regional proximity of donor–recipient pairs, and other factors. The match with the highest total score was selected as the optimal choice.

Patient Flow

OR methods have also been used to improve patient flow, which encompasses the movement of patients between services or facilities as part of their care pathway. For example, improving the surgical suite patient flow is an area of interest in health systems that provide surgical services. ¹⁸ After surgery, patients recover in the post-anesthesia care unit (PACU) before being transferred to another inpatient unit. Price and coworkers ¹⁸ used integer programming and discrete-event simulation to develop surgery schedules that optimize PACU length of stay by minimizing the number of expected new surgical cases exceeding the expected number of discharged patients. The optimization aspect targeted the length of stay in PACU, as it is associated with hospital length of stay, cost of care, and PACU capacity to accommodate other patients scheduled for recovery. They used discrete-event simulation to compare the schedules provided by integer programming with alternative schedules. Their results provided more flexibility in setting the schedule blocks for each service group, reduced the variability of daily occupancy of inpatient beds, and improved PACU utilization. This study is another example of formulating a deterministic model of a system and optimizing the use of mathematical programming followed by simulation to evaluate the optimal solution. Furthermore, OR methods have been used to investigate optimized patient flow through a health system by studying disease progression, ⁴² patients who are at high risk of developing complications, ⁴³ or unplanned re-admissions. ⁴⁴

Drug Allocation

As another example, California’s Medicine for People in Need program collaborated with OR experts to determine an optimal solution for allocating $150 million in drugs as part of a litigation settlement. ⁴⁵ The allocation task required the distribution of 2.6 million 30-day prescriptions equitably among several clinics while considering constraints, such as limits on the total dollar value of contributed drugs over a specific time, minimum order sizes, and requirements to show that the drugs ordered by the clinics were most critical to the needs of the clinics’ patient base. The optimization aspect of the problem was two-fold: 1) to maximize the dollar value of the drug allocation, and 2) to minimize the difference between the ratio of the allocations and the weighted orders from clinics. The OR application supported the optimal and equitable distribution of pharmaceuticals and enabled many uninsured patients, who had little or no access to medication, to receive prescribed drugs.

Medical Device-related Decisions

Biomedical engineering is the application of critical thinking and problem solving skills from engineering to complex medical and biological problems to improve diagnosis, monitoring, and therapy. ⁴⁶ Biomedical engineering has given rise to important diagnostic or therapeutic devices such X-ray machines, magnetic resonance imaging, and implanted cardiac devices. ⁴⁷ Although these devices have been widely used in practice, there are still major challenges surrounding these devices. For instance, many treatment strategies involving these devices are one-size-fits-all and rely on practitioners’ and manufacturers’ intuition and experience. ⁴⁸ Considering the complexity of medical device-related decisions, OR methods can support obtaining the optimal treatment strategies in a timely manner.

For instance, Khojandi and others ⁴⁹ considered the problem of abandon/extract decisions for failed cardiac leads. To deliver the appropriate therapy, cardiac devices require a predetermined number of functioning cardiac leads, which are thin wires that connect the device to the heart. These leads fail stochastically, requiring surgery to add a new lead, during which any subset of the fail lead(s) may be extracted. The optimal decision to abandon/extract each failed lead requires balancing the short- and long-term consequences of these actions while accounting for the increasing risk of complication following lead extraction in lead age, space limitations in the vein, possibility of infection, and competing risk of death. Khojandi and others ⁵⁰ formulated semi-MDP (SMDP) models for the most common types of cardiac devices to obtain the lifetime-maximizing, patient-specific lead extraction strategies. Instead of using SMDPs, simulation could be used to obtain the optimal solution. However, Khojandi and others estimated that using simulation to obtain the optimal treatment strategy for each patient would take years to complete, as opposed to only several minutes to solve the SMDP model. Hence, mathematical optimization models may be better suited for addressing complex problems with numerous candidate strategies, such as medical device-related decision problems. As medical devices increasingly concentrate on advancing personalized healthcare delivery, using OR methods in this area can provide further research opportunities.

Complex Patients with Comorbidities

Patients increasingly present with complex health conditions, often with comorbidities and competing risks of complications. ⁹ Comorbidities commonly refer to chronic or long-term conditions that are present in the same person at the same time. ⁵¹ Studies show that approximately 75 million people in the United States have 2 or more conditions that require ongoing medical attention or limit the activities of daily living, or both.^52–53 Multiple chronic conditions are associated with high level of morbidity, and poor physical functioning, quality of life, and social well-being. ⁵⁴ For healthcare decisions, representing comorbidities can add complexity, where an optimal treatment for one condition may not be optimal for the other condition. Modeling comorbidities as competing risks can require advanced models to obtain optimal or near-optimal decisions at the individual and systems level.

For instance, liver cirrhosis can evolve to hepatocellular carcinoma (HCC) and ultimately death, and the presence of comorbidities can influence the terminal events. ⁵⁵ Bartolomeo and others ⁵⁵ used a Markov model to capture the disease pathway of individuals affected by liver cirrhosis leading to HCC and/or death. Their model explored whether chronic viral hepatitis C infection, Charlson Index (which is based on diagnosis codes and contains 17 categories of comorbidities), and alcohol abuse impacted the development of HCC and/or death in individuals with liver cirrhosis. Their model showed that state transition probabilities of a cirrhotic subject to HCC were not impacted by hepatitis C virus infection or alcohol abuse. The state transition probabilities from liver cirrhosis to HCC and to death were found to be higher in individuals who have a more severe case mix (defined in the study as age >65 years and Charlson Index >3) as compared with individuals with a less severe case mix. As the patients’ complexity increases, applying stochastic multi-state models, like Markov models, to develop treatment strategies that consider competing conditions is an emerging future research field.

Guideline Evaluation

Many clinical guidelines report comparative effectiveness analysis of different interventions and recommend intervention strategies. When the number of potential interventions or the number of possible ways of implementing an intervention strategy become too extensive, OR methods can be used to evaluate the clinical guidelines compared with alternative intervention strategies. One such example occurs in the active surveillance for low-risk prostate cancer. Published strategies suggest various frequencies at which to conduct follow-up biopsies following the diagnosis of a low risk prostate cancer. ⁵⁶ The follow-up strategy must tradeoff multiple criteria including the harm from frequent biopsies (e.g., pain, anxiety, cost, infection) as well as the benefits of frequent biopsies, such as early detection and subsequent treatment of disease progression. If biopsies can be conducted at any 6-month interval, then it would imply that over a 10-year period alone, there are 2×10 ²⁰ possible surveillance strategies and the number of possibilities increases exponentially as the length of the time period considered increases. Optimization models offer a means to solve problems like this using algorithms based on integer programming methods that can efficiently search a large set of potential strategies to decide whether to perform biopsy at each 6-month period and to find the strategy that provides the best tradeoff between benefits and harms. Solving the resulting optimization model and comparing the results to existing guidelines would allow a more informed evaluation of published clinical guidelines and, possibly, the generation of new and improved guidelines.