High-Risk Case Identification for Use in Comprehensive Complex Care Management

Abstract

The current study aimed to develop a patient selection process for muiltimorbid care management that balances the needs to accurately identify patients who are at risk for future high costs and assures that those selected can clinically benefit from proactive care management. Six physicians were surveyed on characteristics of their current (2012) patients to elicit clinical considerations for high-risk patient identification. Data from 2010–2011 were extracted from Clalit Health Services' (Israel's largest managed care organization) comprehensive database to derive the Adjusted Clinical Groups (ACG) predictive model risk scores for risk of future high costs. Model discriminatory power was assessed using the c-statistic and positive predictive value (PPV), before and after application of the clinical exclusion criteria. Inclusion criteria were refined based on physician input from a survey on 375 patients. Recommended reasons for exclusion: active cancer, schizophrenia, dialysis, residence in nursing homes or long-term care facilities, and age 95 years or older. The high-risk group included 5341 patients (mean 50 patients per physician). The c-statistic of the ACG model before and after exclusions applied was 0.80 and 0.75, respectively. After exclusion, the PPV for the 6% highest risk patients was 40%. High-risk patients' age, number of chronic conditions, and utilization were substantially higher than those of all other patients. This study shows that a validated predictive modeling tool provides acceptable discriminatory power for selecting multimorbid patients for participation in proactive care management, even after some of the highest risk patients are excluded because of priori clinical considerations. (Population Health Management 2015;18:15–22)

Introduction

Patients with multiple chronic conditions require proactive coordinated care management to effectively manage numerous health conditions comprehensively and not each as a discrete problem.^1,2 A shift away from condition-specific disease management programs toward targeting patients with complex medical needs³ entails greater challenges in providing comprehensive care and further complicates the selection process for identifying patients to participate in such programs. With a broad target population and wide array of preventive, resource-intensive interventions, the participant selection process for care management should identify those patients who are likely to deteriorate, while also ensuring that those selected have “actionable” health needs for preventive care.⁴ For example, a known driver of risk for future high utilization is current cost,⁵ yet those already in the top percentile of current resource use might not be amenable or might not represent a high opportunity for preventive intervention.⁶ Critical to proper implementation of such an intervention program is not only identifying those at highest risk, but also a better understanding of who might benefit from intervention.

There are various screening techniques employed to select patients for care management programs, ranging from clinically-based physician referrals and questionnaires, to empirically-based high-risk case identification models, criteria eligibility models, and gaps in care analyses.^6,7 Each approach has its advantages and inherent biases.⁷ Empirically-based screening techniques, such as criteria-based screening or predictive high-risk case identification models, offer objective and standardized approaches but commonly encounter challenges in data inconsistencies or deficiencies in providing clinically meaningful outputs that indicate whether patients will be amenable to care management programs. In contrast, clinically-based approaches to patient selection, such as clinician referrals, can better detect the nuances of patients' unmet health needs to identify patients suitable for care support (termed “impactibility” by Lewis⁸ and “care sensitivity” by Freund et al⁹) but are not systematic and are considered subjective with demonstrated selection biases.¹⁰

Although high-risk identification models have emerged as efficient and systematic approaches to patient selection for care management,^4,5,11
–13 they often are used as they were developed—“off the shelf.” Applying generic risk algorithms to specific patient selection processes requires 2 complementary tailoring processes. First, there is a need to determine exclusion criteria to ensure that those selected by a computerized predictive model have actionable health needs for such programs.¹⁴ Although a combined predictive algorithm-physician selection approach to the patient selection has been suggested previously, in which physicians make decisions about individual patients after receiving a list of potential participants for care management, the authors indicate that further research is warranted to identify a selection approach for practice-based care management.¹⁵ Secondly, there is a need to decide on the cut points (ie, what risk thresholds will be used to determine eligibility).⁷ Decisions of risk score thresholds that govern inclusion in a care management program often are made based on available resource capacity; however, considerations of the model's predictive accuracy also must be accounted for to ensure that patients at risk for deterioration are being targeted. Therefore, the current study aims to develop a patient selection process for multimorbid care management that balances the needs to systematically identify patients who are at risk for future high costs and to distinguish who can benefit from proactive care management.

Methods

Study Design

The research team conducted a physician survey to determine a set of clinically-defined patient exclusion considerations according to current (2012) patient characteristics. A separate retrospective cohort analysis of data from 2010 was performed in parallel: first, to evaluate exclusion criteria, and subsequently to assess the discriminatory power of a predictive algorithm to identify patients at risk of incurring future high cost in the following year (2011). Clinically-defined exclusion criteria obtained from the physician survey were used to inform the final list of exclusions that were applied to derive the final study population for the cohort analysis.

Setting

The study was conducted in Clalit Health Services (Clalit), the largest of 4 health funds in Israel.¹⁶ Clalit is an integrated care system, covering more than 4 million members, that has a comprehensive data warehouse encompassing electronic clinical and administrative records. Clalit's multimorbidity care management program incorporates a nurse-primary care physician (PCP) team with a focus on preventive interventions to enhance coordination of care and continuity in health services for approximately 50 patients with complex health needs per physician. The program encompasses care plan creation, self-management education, continuous monitoring, and coordination of care providers, especially during care transitions.

Study Population

The cohort study population, drawn from 2010–2011 data, comprised adult patients of PCPs from 2 of 8 districts that Clalit serves. The smaller sample of patients for the survey also comprised adult patients of PCPs from the same districts, with data sampled from 2012. A preliminary set of exclusion criteria were decided by a clinician panel of PCPs and nurses and was applied to patients in both the larger cohort and the survey sample. These exclusions were: patients currently receiving oncology care, patients who were part of another disease or specialty care management program, or patients who were partaking in a research study. Lack of consensus on several diagnoses, dialysis use, highest 1% costs, and age criteria merited further investigation to determine additional exclusion criteria.

Clinically-Defined Exclusions

A panel of 6 Clalit PCPs was surveyed to obtain clinical insight on relevant exclusion criteria. Clinical characterizations of patient medical complexity encompassing medical, social, and behavioral factors from previous research were used to guide the development of the physician survey in this study.³ Participating PCPs were informed about the care management program's structure, goals, and focus prior to the survey. The physicians were provided randomized lists of 55–70 of their patients (about 6% of the average physician panel size), and were asked to indicate whether they thought a patient was or was not a good fit (ie, at risk of deterioration and having clinical characteristics amenable to preventive care) for care management, including reasons for that decision. Physicians had the option to check as many reasons as they deemed applicable, and also were given the opportunity to provide their own reasons. Reasons for inclusion in or exclusion from care management provided in the survey were categorized into 3 broad classifications: (1) Medical Complexity, (2) Patient Characteristics, and (3) Social Reasons. Some of the reasons specified within each of these 3 categories were the same for inclusion in and exclusion from care management (eg, poor medication adherence, multiple missed scheduled physician visits). These reasons are the same because they can potentially be perceived by different physicians as either an indicator that a patient needs further guidance in care or as an indication that a patient is not amenable to care management given low engagement in care activities. Further details of the physician survey template can be found in Figure 1.

FIG 1.

Physician survey template.

Analysis

Exclusion criteria assessment

A preliminary analysis was conducted to determine an initial risk score threshold, delineating “high” and “low” risk patients that would correspond to the program's capacity of ∼50 patients per physician (results not shown). This initial risk score cut point was applied to the 2012 study sample. Both high- and low-risk patients were included in the survey sample. Results of the survey were evaluated for the most common reasons for inclusion and exclusion. Assessment of the highest cost patients (those with highest 1% costs in 2010) using patient-level utilization and cost data from 2010–2011 was conducted to determine if current year highest 1% costs should be an exclusion criterion. The evaluation was based on whether or not the health services consumed were representative of conditions that were potentially amenable to preventive intervention. Additionally, mortality rates and chronic condition counts by age group were assessed to determine exclusion thresholds. Exclusion criteria were refined based on survey results, findings from analysis of health services use and patient characteristic data, and additional consultation with the clinical advisory panel about diagnoses-related criteria.

Predictive accuracy and threshold assessment

Data from electronic health records (EHR) in 2010–2011 were extracted to derive the Adjusted Clinical Groups (ACG) predictive model risk scores for risk of future high costs. The Johns Hopkins' ACG case-mix algorithm stratifies patients according to their diagnoses, age, and sex. Each diagnosis taken from any medical visit (outpatient or hospital care) during a defined period (usually a year) is coded into one of 32 morbidity types. This classification is based on the duration, severity, diagnostic certainty, etiology, and need for specialty care that is associated with the diagnosis. According to a combination of the 32 morbidity types, as well as age and sex, the system then designates individuals to one of about 90 morbidity categories representing the spectrum from low to high levels of medical need.¹⁷ The ACG System also can be used to predict risk, assigning a risk score to each individual based on his or her morbidity categories and a defined list of covariates. The research team used the combined diagnostic/medication predictive model, which includes age, sex, diagnoses, medications, and prior costs as input to predict the risk for incurring future high costs.⁵ The ACG Case-Mix System, Version 9.1 software (Johns Hopkins Bloomberg School of Public Health, Baltimore, MD) was used in this study.

The c-statistic was calculated to evaluate the overall predictive accuracy of the model in the cohort population. Cross tabulations were performed for cut points of 2010 risk probability levels and the highest 10% of total 2011 costs, based on the original adult population cost distribution before any exclusions were applied. Predictive accuracy, measured as the positive predictive value (PPV), and the size of the patient pool at each risk threshold, were evaluated to determine the appropriate risk probability threshold. Cut points yielding the highest PPV while still within the considerations of program capacity were selected. Statistics on age, sex, total costs, total pharmacy costs, chronic condition counts, number of hospitalizations, number of generic medications, number of PCP visits, and number of specialist visits were examined for the top risk probability groups from the original and final study populations. Chronic condition counts were derived from Clalit's registry, which includes 180 chronic conditions. Statistically significant differences between the original and final samples were tested using t tests and chi-square analysis. This study received approval from Clalit Health Services' and the University of Haifa's institutional review boards.

Results

Of the 375 patients included in the survey, physicians indicated that 53.1% (180) were a good fit for care management, and 46.9% (159) were not a good fit for care management. Medical complexity was the dominant category indicated by physicians as to why patients were or were not a good fit for care management (Table 1). Reasons most cited by physicians for inclusion in care management were: patients had 1 or more uncontrolled medical condition, needed close monitoring to balance health conditions, needed support or caregiver support to meet treatment guidelines, and were poorly adherent to medications. Medical diagnosis was the most cited reason for exclusion from care management, as well as other medical complexity reasons for 36 patients. Of the reasons that were options for both inclusion in and exclusion from care management (adherence-related reasons), most were indicated by physicians as reasons for inclusion in care management (ie, needing further guidance) rather than as reasons for exclusion. Almost all other reasons were that patients were healthy, had controlled medical conditions, or were living in a nursing home.

Table 1.

Reasons Provided by Surveyed Physicians in the Survey for Inclusion In or Exclusion From a Care Management Program

Group	Reason	Total N Reasons	Percent for Reasons (out of group total)	Total N of Reasons for Groups	Group Percent of Total
PATIENT IS A GOOD FIT FOR CARE MANAGEMENT
				MC	60.7%
Medical Complexity (MC)	One or more MCs not controlled	136	30.9%	440
	Needs close monitoring to balance health conditions	130	29.5%
	Multiple caregivers (doctors, specialists)	66	15.0%
	Taking multiple medications	102	23.2%
	Other	6	1.4%
				PC	28.5%
Patient Characteristics (PC)	Need support to meet treatment guidelines	95	41.9%	227
	Multiple missed doctor visits	22	9.7%
	Poor medication adherence	45	19.8%
	Need primary caregiver support	61	26.9%
	Other	4	1.8%
				Soc	10.8%
Social Reasons (Soc)	Economic reasons	13	15.9%	82
	Lives alone	34	41.5%
	Cultural factors	20	24.4%
	Language barrier	3	3.7%
	Other	12	14.6%
Total reasons for why patient fits in care management		749		749	100.0%
PATIENT IS NOT A GOOD FIT FOR CARE MANAGEMENT
				MC	89%
Medical Complexity (MC)	Medical diagnosis	33	37.1%	89
	Recently underwent surgical procedure	1	1.1%
	Mental condition	8	9.0%
	Very low cognitive ability	8	9.0%
	Poor functional status	3	3.4%
	Other	36	40.4%
				PC	8%
Patient Characteristics (PC)	Patient too old	3	37.5%	8
	Patient behavioral factors	0	0.0%
	Multiple missed scheduled physician appointments	1	12.5%
	Poor medication adherence	2	25.0%
	Other	2	25.0%
				Soc	3%
Social Reasons (Soc)	Economic reasons	0	0.0%	3
	Lives alone	0	0.0%
	Cultural factors	2	66.7%
	Language barrier	1	33.3%
	Other	0	0.0%
Total reasons for why patient does not fit in care management		100		100	100%

After the physician survey was conducted, additional clinical criteria were evaluated as exclusions. The original study population comprised 133,514 adult patients (aged 18 years and older). Review of patients with the highest 1% costs (in 2010) indicated that some of the services and procedures used by these patients were not actionable (such as dialysis, transplant services, long-term care, and rehabilitation); however, other health care services and resources accessed by some (such as internal medicine hospitalizations, other hospitalizations, and ambulatory services) were potentially clinically actionable for intervention. An age cutoff was determined at 95, given that 2-year survival was less than 50% for those members with high-risk scores who were age 95 or older (results not shown). The final study population consisted of 82,247 patients after all exclusion criteria were applied. Figure 2 shows the study population exclusions diagram.

FIG 2.

Study cohort population exclusion flow chart.

Program capacity and predictive accuracy were considered in the analysis assessing cut point thresholds for the top 10% costs in both study populations (Table 2). In the original population, the 4% risk probability cut point was selected (corresponding to an ACG risk score of ≥0.25), given an N of 64 patients per physician and a PPV of 45.2%. In the final population, after all exclusion criteria were applied, the highest 6% risk probability threshold (corresponding to risk scores of ≥0.19) was selected, given an N of 49 patients per physician and a PPV of 39.9%. The c-statistic for the model in this population was 0.75 (95% confidence interval 0.74–0.76). The final high-risk group of patients identified as a potential target intervention group included 5341 patients. High-risk patients had an average of 5.78 chronic conditions, compared to 2.02 chronic conditions for all other patients, and their age and utilization were substantially higher than that of all other patients, with all differences statistically significant (P<0.001). Additional descriptive statistics for the high-risk sample can be found in the Supplementary Table. (Supplementary material is available in the online article at www.liebertpub.com/pop).

Table 2.

Cross Tabulations of the Top 10% 2011 Total Costs by High-Risk Probability Group: Original vs. Final Study Populations

Original Total Costs 2011 - Top 10%		Original Adult Study Population n=133,514	Final Study Population (additional patient characteristic criteria) n=82,247
High-Risk Probability 4%	N (patients per physician)	64	32
	PPV	45.2%	45.3%
High-Risk Probability 6%	N (patients per physician)	80	49
	PPV	42.8%	39.9%

The 4% risk probability cut point corresponded to an Adjusted Clinical Groups (ACG) risk score of 0.25 in the original study population, and the 6% risk probability cut point corresponded to an ACG score of 0.19.

PPV, positive predictive value.

Comparison Across Study Populations

Descriptive statistics comparing the highest 6% risk patients in the original and the final study populations show that these groups are similar in utilization parameters except for total costs and pharmacy costs (Table 3). The average costs are lower in the final study population, given that certain very high cost patients (eg, transplant patients, those on dialysis, those in long-term nursing or rehabilitation centers) were excluded according to highest 1% costs analysis. On average, patients in both populations have many chronic conditions (on average 9–10 conditions), are taking a multitude of medications (>17), and are visiting their PCP about once every 2 months (mean 7.74 PCP visits in the final population compared to 6.92 in the original population).

Table 3.

Comparison of Demographics and Care Utilization Parameters between The 6% Highest Risk in the Original and Final Study Populations

	Original Study Population (6% highest risk)			Final Study Population (6% highest risk)
	N	2010 Mean or Percent	Std Dev (2010)	N	2010 Mean or Percent	Std Dev (2010)
Age	8716	73	13	5341	74	12
Sex
F	4596	52.7%	-	2755	51.6%	-
M	4120	47.3%	-	2586	48.4%	-
Total cost 2010 (mean, NIS)	8716	11.7^a	64,419	5341	7.3^a	42,057
Pharmacy cost 2010 (mean, NIS)	8716	10.4^a	28,270	5341	7.8^a	24,090
Number of chronic conditions	8716	9.24	4.06	5341	9.37	3.85
Number of hospitalizations	8716	1.87	2.52	5341	1.43	2.07
Generic drug count	8716	17.15	7.58	5341	17.45	7.27
Number of PCP visits	8716	6.92	6.70	5341	7.74	6.82
Number of specialist visits	8716	3.94	6.21	5341	3.16	4.51
Emergency visits count	8716	0.52	1.11	5341	0.48	1.12

Total average costs and total pharmacy costs are relative to the overall Clalit average for total costs and total pharmacy costs (fixed values).

NIS, New Israeli Shekel; PCP, primary care physician.

Discussion

This study builds on model cut point optimizing techniques and high-risk patient identification to derive an accurate and systematic patient selection process, aimed at the greatest impactibility, for care management. This study demonstrates that a more targeted population “amenable” to care management can be derived through the systematic integration of clinical decision making as a priori inclusion and exclusion criteria, while maintaining predictive accuracy. Freund et al⁹ originally suggested that a combination of predictive modeling and patient identification by physicians, wherein clinical discretion is applied after risk model identification would help advance selection methods for care management. Although this approach ensures clinical “impactibility,” it still contains biases of physician subjectivity¹⁰ that limit its ability to be applied systematically for entering patients into large-scale programs and/or care management evaluation trials.

As an alternative approach, the method in the present study incorporates clinician-determined indicators for exclusion (eg, patients on dialysis or with active cancer) into the systematic risk identification model. These exclusions to narrow the target population circumvent the subjective bias that would be introduced if physicians referred individual patients for care management participation. It can be argued that quantitatively derived risk scores simplify a patient's complex condition to a single score, thereby neglecting to account for many factors relevant to care management (eg, social factors, ability to participate). In an attempt to address these concerns and identify patients amenable to care management, the ACG predictive risk model was utilized with a refined selection of clinically defined morbidity inputs, as well as tested for multiple clinical considerations for establishing exclusion criteria. Employing a high-risk case identification tool for care management patient selection offers high predictive value and systematic advantage^5,6 over other objective and subjective screening methods.

Although narrowing the patient pool with exclusion criteria to target impactibility is important, an organization also must be judicious not to introduce bias when excluding patients a priori, and eliminate patients who may potentially benefit from care management. Lewis⁸ cautions that without the appropriate incentives and oversight, organizations focused primarily on cost-effectiveness in care management would be inclined to use risk modeling for impactibility to “dump” or exclude difficult patients from participation in the program, as they might negatively impact the program's results. In a closed payer and provider system like Clalit, in which the organization cannot prevent enrollment and individuals rarely switch health funds throughout their lifetime, it is in the organization's interest to provide preventive care—particularly to clinically complex patients—to avert future deterioration and expensive health service utilization. Furthermore, using cost thresholds may offer easily accessible exclusion parameters, such as excluding patients with the highest 1% current costs; however, the findings of this study suggest that using specific types of services as exclusion criteria are preferable to target “actionability” and appropriateness for care management.

As the patient population was refined by excluding those who were less amenable to preventive care interventions, the risk score threshold was adjusted to accommodate changes in the risk makeup and overall size of the study population. Adjusting for risk score cut points and evaluating resultant PPVs accounts for program capacity considerations while also ensuring that using an “off-the-shelf” risk model maintains statistical validity within the specific population.⁷ When comparing the PPVs of the highest 6% risk groups across the study populations, it was observed that there is little change in the PPV from the original adult population to the final study population, indicating that the model did not lose its predictive value despite a substantial drop (a 38% decrease) in the size of the population. The highest 6% risk probability threshold yielded a reasonable patient sample size for program resources and a comparable PPV to those found by other similar risk identification studies, indicating that 40% of those predicted to be high risk would incur future high costs.^7,18,19 This demonstrates relatively strong predictive power, as past research has shown that screening tools yielding even 20% PPV are cost-effective.²⁰ Where to draw the line for thresholds has greater implications for targeting appropriate patients than just program capacity limitations, and this study demonstrates the importance of emphasizing predictive accuracy as part of the decision-making process.

Limitations

There are several limitations to consider in this study, including the measures of predictive performance used. Although these are standard measures, Hu and Root²¹ discuss the shortcomings of predictive performance measures such as PPV in the care management context. While these measures have their weaknesses, PPV is still helpful in determining whether or not a risk algorithm offers sufficient value when integrated into the patient identification process. This is further supported by the moderately high overall accuracy demonstrated by the algorithm's c-statistic. Lastly, as with all research reliant on large-scale databases, the risk score valuation and exclusion criteria are limited to the parameters and data available in the system. Despite these limitations, the results of this study demonstrate that a combined predictive risk model-clinical input approach to patient selection for care management can be developed for use in practice, accounting for impactibility, predictive accuracy, and resource capacity.

Implementation in Practice

This study reports on the process developed for care management patient selection incorporated into a pilot program at Clalit, and also generalizes this process for broader application in other organizations and care management contexts (Figure 3). Depending on the structure of the organization and the health system in which it operates, the target patient pool (inclusion criteria) can be determined through various frameworks. Choosing a risk identification algorithm to be used in the process is contingent on available data for model input, and entails testing its predictive power within the organization's population and within the target patient pool.¹⁸ With access to patients' medical record data, utilization data, and cost data, diagnosis-based, health service-based, and patient characteristic-based criteria can be established, as well as the assessment of age thresholds, chronic condition counts and high costs. The concept of accountable care organizations and other integrated models are gaining traction as viable models to improve care delivery and efficiency, with payer-provider organization partnerships being developed.^22,23 These partnerships, coupled with the advent of EHRs, can lead to wider availability of linked and comprehensive data sources (eg, linked administrative records to outpatient and inpatient medical records) for more accurate risk prediction. Ultimately, linked EHRs and utilization data sources can enable evidence-based decision making for care management program patient selection. Based on the current research study, the research team, therefore, suggests a 4-step process: establish eligibility criteria, determine program capacity, apply risk model, and assess predictive accuracy.

FIG 3.

Generalized systematic patient selection methodology for care management program. ACG, Adjusted Clinical Group; PPV, positive predictive value.

Conclusion

Determining which patients will be most appropriate to participate in a care intervention program is increasingly a priority of health program managers and intervention coordinators. The value of employing the predictive power of quantitative models in this selection process is fundamental because of the preventive and diverse nature of the care delivered through a care management program. This study shows that applying criteria, intended to increase “impactibility,” to a “generic” high-risk patient identification model is a feasible and accurate selection approach that can be used continuously. If selection was dependent on subjective practitioner assessment alone, program managers would not be able to confirm that the patients selected are in fact those at high risk for deterioration of health status; likewise, relying solely on a generic quantitative algorithm cannot ensure that the element of impactibility is addressed in patient selection. A method that integrates these concepts into a standardized approach for patient selection gives intervention coordinators and program managers a powerful tool to implement in practice.

Footnotes

Acknowledgments

The authors would like to thank the 6 primary care physicians from the Tel Aviv and Central districts of Clalit Health Services who participated in the survey and volunteered their time to offer valuable insights for this research. The authors also thank the physicians and nurses in the Community Division of Clalit Health Services who served on the clinician advisory panel providing additional clinical guidance for this project.

Author Disclosure Statement

Ms. Cohen, Ms. Flaks-Manov, Mr. Low, Dr. Balicer, and Dr. Shadmi declared no conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors received the following financial support for the research, authorship, and/or publication of this article: This study is part of a larger care management study supported by the Clalit Research Institute.

References

Salisbury

, Johnson

, Purdy

, et al. Epidemiology and impact of multimorbidity in primary care: a retrospective cohort study. Br J Gen Pract., 2011; 61(582):e12–e21.

Boyd

, Boult

, Shadmi

, et al. Guided care for multimorbid older adults. Gerontologist., 2007; 47:697–704.

Grant

, Ashburner

, Hong

, et al. Defining patient complexity from the primary care physician's perspective: a cohort study. Ann Intern Med., 2011; 155:797–804.

Bodenheimer

, Berry-Millett

. Follow the money–controlling expenditures by improving care for patients needing costly services. N Eng J Med., 2009; 361(16):1521–1523.

Forrest

, Lemke

, Bodycombe

, et al. Medication, diagnostic, and cost information as predictors of high-risk patients in need of care management. Am J Manag Care., 2009; 15:41–48.

Bernstein

. New arrows in the quiver for targeting care management: high-risk versus high-opportunity case identification. J Ambul Care Manage., 2007; 30(1):39–51.

Murphy

, Castro

, Sylvia

. Predictive modeling in practice: improving the participant identification process for care management programs using condition-specific cut points. Popul Health Manag., 2011; 14:205–210.

Lewis

. “Impactibility models”: identifying the subgroup of high-risk patients most amenable to hospital-avoidance programs. Milbank Q., 2010; 88:240–255.

Freund

, Mahler

, Erler

, et al. Identification of patients likely to benefit from care management programs. Am J Manag Care., 2011; 17:345–352.

10.

Shadmi

, Freund

. Targeting patients for multimorbid care management interventions: the case for equity in high-risk patient identification. Int J Equity Health., 2013; 12(1):70.

11.

Wharam

, Weiner

. The promise and peril of healthcare forecasting. Am J Manag Care., 2012; 18:e82–e85.

12.

Cousins

, Liu

. Cost savings for a preferred provider organization population with multi-condition disease management: evaluating program impact using predictive modeling with a control group. Dis Manag., 2003; 6:207–217.

13.

Charlson

, Charlson

, Briggs

, et al. Can disease management target patients most likely to generate high costs? The impact of comorbidity. J Gen Intern Med., 2007; 22:464–469.

14.

Sylvia

, Shadmi

, Hsiao

, et al. Clinical features of high-risk older persons identified by predictive modeling. Dis Manag., 2006; 9:56–62.

15.

Freund

, Wensing

, Geissler

, et al. Primary care physicians' experiences with case finding for practice-based care management. Am J Manag Care., 2012; 18:e155–e161.

16.

Gross

, Rosen

, Shirom

. Reforming the Israeli health system: findings of a 3-year evaluation. Health Policy., 2001; 56(1):1–20.

17.

Weiner

, Starfield

, Steinwachs

, et al. Development and application of a population-oriented measure of ambulatory care case-mix. Med Care., 1991; 29:452–472.

18.

Weir

, Aweh

, Clark

. Case selection for a Medicaid chronic care management program. Health Care Financ Rev., 2008; 30(1):61–74.

19.

Meenan

, Goodman

, Fishman

, et al. Using risk-adjustment models to identify high-cost risks. Med Care., 2003; 41:1301–1312.

20.

Mukamel

, Chou

, Zimmer

, et al. The effect of accurate patient screening on the cost-effectiveness of case management programs. Gerontologist., 1997; 37:777–784.

21.

, Root

. Accuracy of prediction models in the context of disease management. Dis Manag., 2005; 8:42–47.

22.

Epstein

, Jha

, Orav

, et al. Analysis of early accountable care organizations defines patient, structural, cost, and quality-of-care characteristics. Health Aff (Millwood)., 2014; 33:95–102.

23.

Song

, Lee

. The era of delivery system reform begins. JAMA., 2013; 309:35–36.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB