Predicting 3-Year Survival in Patients Receiving Maintenance Dialysis: An External Validation of iChoose Kidney in Ontario,Canada

Abstract

French

Background:

Many patients with end-stage kidney disease (ESKD) do not appreciate how their survival may differ if treated with a kidney transplant compared with dialysis. A risk calculator (iChoose Kidney) developed and validated in the United States provides individualized mortality estimates for different treatment options (dialysis vs living or deceased donor kidney transplantation). The calculator can be used with patients and families to help patients make more educated treatment decisions.

Objective:

To validate the iChoose Kidney risk calculator in Ontario, Canada.

Design:

External validation study.

Setting:

We used several linked administrative health care databases from Ontario, Canada.

Patients:

We included 22 520 maintenance dialysis patients and 4505 kidney transplant recipients. Patients entered the cohort between 2004 and 2014.

Measurements:

Three-year all-cause mortality.

Methods:

We assessed model discrimination using the C-statistic. We assessed model calibration by comparing the observed versus predicted mortality risk and by using smoothed calibration plots. We used multivariable logistic regression modeling to recalibrate model intercepts using a correction factor, when appropriate.

Results:

In our final version of the iChoose Kidney model, we included the following variables: age (18-80 years), sex (male, female), race (white, black, other), time on dialysis (<6 months, 6-12 months, >12 months), and patient comorbidities (hypertension, diabetes, and/or cardiovascular disease). Over the 3-year follow-up period, 33.3% of dialysis patients and 6.2% of kidney transplant recipients died. The discriminatory ability was moderate (C-statistic for dialysis: 0.70, 95% confidence interval [CI]: 0.69-0.70, and C-statistic for transplant: 0.72, 95% CI: 0.69-0.75). The 3-year observed and predicted mortality estimates were comparable and even more so after we recalibrated the intercepts in 2 of our models (dialysis and deceased donor kidney transplantation). As done in the United States, we developed a Canadian Web site and an iOS application called Dialysis vs. Kidney Transplant- Estimated Survival in Ontario.

Limitations:

Missing data in our databases precluded the inclusion of all variables that were in the original iChoose Kidney (ie, patient ethnicity and low albumin). We were unable to perform all preplanned analyses due to the limited sample size.

Conclusions:

The original iChoose Kidney risk calculator was able to adequately predict mortality in this Canadian (Ontario) cohort of ESKD patients. After minor modifications, the predictive accuracy improved. The Dialysis vs. Kidney Transplant- Estimated Survival in Ontario risk calculator may be a valuable resource to help ESKD patients make an informed decision on pursuing kidney transplantation.

Keywords

risk calculator maintenance dialysis kidney transplantation mortality survival external validation iChoose Kidney

What was known before

Many patients with end-stage kidney disease (ESKD) do not appreciate how their survival may differ if treated with a kidney transplant compared with dialysis. iChoose Kidney is a risk calculator developed in the United States that provides individualized mortality estimates for different treatment options (dialysis vs living or deceased donor kidney transplantation) based on ESKD patients with similar characteristics. Whether this tool can provide accurate individualized mortality estimates in Canadian ESKD patients is unknown.

What this adds

These results suggest the original iChoose Kidney risk calculator is able to estimate the likelihood of mortality with different ESKD treatment options with a reasonable degree of accuracy in Ontario patients receiving maintenance dialysis. After minor modifications, the predictive accuracy improved. The Canadian version of the iChoose Kidney risk calculator (Dialysis vs. Kidney Transplant- Estimated Survival in Ontario) may be a valuable resource to help ESKD patients make an informed decision on pursuing kidney transplantation.

Introduction

Kidney transplantation is the preferred treatment option for most individuals with end-stage kidney disease (ESKD), offering improved survival and a better quality of life compared with dialysis.^1-3 With more than 50% of patients dying within 5 years of initiating dialysis, discussions between care providers and patients about prognosis and treatment are essential.^4,5 However, most ESKD patients never discuss prognosis with their physicians⁶ and many patients report that a main barrier to kidney transplantation is limited education.⁷ Offering personalized and evidence-based prognostic information has the potential to help patients understand their treatment options and make more informed decisions about their health.^8,9

Patzer et al developed the iChoose Kidney risk calculator to help ESKD patients make more educated treatment decisions (http://ichoosekidney.emory.edu/).¹⁰ iChoose Kidney provides individualized 1- and 3-year mortality estimates for different options to treat ESKD (dialysis vs living or deceased donor kidney transplantation) using a prediction model that includes age, sex, race, ethnicity, time on dialysis, and comorbidities (ie, diabetes, hypertension, cardiovascular disease, and low albumin).¹⁰ This tool is currently only validated in the United States.¹⁰ Therefore, we conducted this study to externally validate the iChoose Kidney risk calculator in ESKD patients from Ontario, Canada.

Methods

Design and Setting

We conducted an external validation study using linked administrative health care databases held at the Institute for Clinical Evaluative Sciences (ICES) in Ontario, Canada. Ontario residents have universal access to physician and hospital services. These data sets were linked using unique encoded identifiers and analyzed at ICES. The use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a Research Ethics Board. We followed the reporting guidelines for studies using routinely collected observational health data (see Supplementary Table 1) and prediction model studies (see Supplementary Table 2).^11,12

Data Sources

We used 7 linked databases. We obtained patient information on maintenance dialysis and kidney transplantation from the Canadian Organ Replacement Register (CORR). CORR accurately captures maintenance dialysis patients and kidney transplant recipients, with a completeness for dialysis records of 99% and a sensitivity >95% for kidney transplant recipients when compared with transplant center data.^13,14 We supplemented missing race information in CORR with data from the Ontario Renal Reporting System database. We used the Ontario Registered Persons Database for demographic and vital status information, while the Ontario Health Insurance Plan database provided information on physician health service claims. We retrieved diagnostic and procedural information for hospitalizations and day surgeries from the Canadian Institute for Health Information (CIHI) Discharge Abstract Database and Same Day Surgery Database, respectively. Information on emergency room visits was provided by the CIHI National Ambulatory Care Reporting System. Follow-up in our databases is complete with emigration from the province being the only reason for lost to follow-up (<0.5% annually).¹⁵

Study Populations

Maintenance dialysis

We included all incident maintenance dialysis patients in Ontario from January 1, 2004, to December 31, 2014. We excluded the following individuals: non-Ontario residents, patients aged <18 years or >80 years, and a history of chronic dialysis or solid organ transplant (including kidney) at cohort entry. We defined the index date (cohort entry date) as the maintenance dialysis initiation date.

Kidney transplant recipients

We included all incident kidney transplant recipients in Ontario from January 1, 2004, to December 31, 2014. We excluded the following individuals: non-Ontario residents, patients aged <18 years or >80 years, simultaneous multiorgan transplant recipients (eg, kidney-pancreas), and a history of a solid organ transplant (including kidney) at cohort entry. We defined the index date as the date of kidney transplantation.

Study Variables

Our outcome was all-cause mortality (see Supplementary Table 3) and is accurately identified in our databases.^16,17 We followed individuals for 3 years from the date of cohort entry or until the end of the study period (December 31, 2016). Due to the low number of events, we did not examine 1-year mortality as was done in the original validation of iChoose Kidney. Using logistic regression, the variables used to calculate mortality risk in the United States version of the iChoose Kidney risk calculator included sex, age (continuous), race (white, black or African American, other), ethnicity (Hispanic, non-Hispanic), time on dialysis (0-<6 months, 6-12 months, >12 months), cardiovascular disease (ie, congestive heart failure, coronary artery disease, stroke/transient ischemic attack, peripheral vascular disease, myocardial infarction, chronic obstructive pulmonary disease, and other cardiac disease), hypertension, diabetes, and low albumin (<35 g/L). In our databases, data were missing for race (dialysis 1.8%; 6.3% transplant) and donor type (0.9%). For race, we imputed white race, and for donor type, we imputed receipt of a kidney from a deceased donor. Serum albumin values were also missing in 10.5% of maintenance dialysis patients and 32.9% of kidney transplant recipients (with >95% missing for recipients of a preemptive transplant). As a result of the high missingness, we excluded serum albumin from our model. For ethnicity, our databases had no Hispanic variable. However, <0.5% of Canada’s population identifies as Hispanic.¹⁸ Our prediction model included the following variables: age (18-80 years), sex (male, female), race (white, black, other), time on dialysis (<6 months, 6-12 months, >12 months), and patient comorbidities (hypertension, diabetes, and/or cardiovascular disease). By removing albumin and ethnicity from our model, we classified all individuals as having normal albumin and as being of non-Hispanic ethnicity. We defined cardiovascular disease as a composite of congestive heart failure, coronary artery disease (including angina), stroke/transient ischemic attack, peripheral vascular disease, myocardial infarction, chronic obstructive pulmonary disease, and other cardiac disease. Comorbidities were ascertained in the 5 years prior to cohort entry. Formulas used to derive mortality risk can be found in the Supplementary Materials.

Statistical Analysis

Continuous variables were described as medians (25th, 75th percentile) and categorical variables as counts (percentages). We used the area under the receiver operating curve (C-statistic) to determine how well the iChoose Kidney risk calculator could discriminate between individuals who did and did not die within 3 years (a value of 0.5 indicates that the ability of the risk calculator to discriminate mortality is no better than chance alone). We assessed calibration (ie, agreement between the observed and predicted probabilities) by visually comparing observed and predicted mortality risk by deciles of predicted mortality probability. We further assessed calibration using loess-smoothed calibration plots, which graphically compare the observed versus predicted mortality risk.^19,20 Poor calibration is indicated by deviation of the smoothed calibration curve from the ideal line (ie, a 45° diagonal line indicates perfect calibration). To correct calibration-in-the-large, intercepts were recalibrated using a correction factor, when appropriate.²¹ In a sensitivity analysis, we restricted our accrual period from January 1, 2004, to December 31, 2013, to allow all individuals to be followed for a maximum of 3 years. We then examined the discrimination and calibration. All analyses were conducted with Statistical Analysis System software (SAS Institute, Cary, North Carolina), version 9.4.

Results

Baseline Characteristics

We included 22 520 maintenance dialysis patients (see Supplementary Figure 1) and 4505 kidney transplant recipients (see Supplementary Figure 2). Of kidney transplant recipients, 39.6% (n = 1786) received kidneys from living donors and 60.4% (n = 2719) from deceased donors. Compared with kidney transplant recipients, maintenance dialysis patients were older (median: 64 vs 53 years), more likely to be white race (72.4% vs 69.2%), and have diabetes (58.4% vs 32.8%) (Table 1). Compared with the original US population, Canadian ESKD patients had significantly more comorbidities. For example, in our cohort, approximately 62% of dialysis patients and 60% of transplant recipients had cardiovascular disease compared to 58% and 22% in the original cohort, respectively.¹⁰ Demographic characteristics were similar between the 2 populations except for race, where the US population had a higher proportion of black patients.¹⁰ In our study, compared with living donor kidney transplant recipients, recipients of a deceased donor kidney were older (57 vs 48 years) and less likely to be white race (62.8% vs 78.8%) (Table 1).

Table 1.

Baseline Characteristics for Maintenance Dialysis Patients and Kidney Transplant Recipients.

Characteristic	Maintenance dialysis(n = 22 520)	Kidney transplant recipients(n = 4505)	Living donor kidney transplant recipients(n = 1786)	Deceased donor kidney transplant recipients(n = 2719)
Demographics
Age, years	64 (54, 73)	53 (42, 62)	48 (37, 57)	57 (47, 64)
Age category, years
18-39	1716 (7.6)	882 (19.6)	547 (30.6)	335 (12.3)
40-49	2438 (10.8)	935 (20.8)	429 (24.0)	506 (18.6)
50-59	4175 (18.5)	1213 (26.9)	457 (25.6)	756 (27.8)
60-69	6249 (27.7)	1136 (25.2)	309 (17.3)	827 (30.4)
70-80	7942 (35.3)	339 (7.5)	44 (2.5)	295 (10.8)
Female sex	8808 (39.1)	1651 (36.6)	682 (38.2)	969 (35.6)
Race
White	16 305 (72.4)	3116 (69.2)	1407 (78.8)	1709 (62.8)
Black	1395 (6.2)	365 (8.1)	91 (5.1)	274 (10.1)
Other	4820 (21.4)	1024 (22.7)	288 (16.1)	736 (27.1)
Rural residence^a	3016 (13.4)	498 (11.1)	226 (12.7)	272 (10.0)
Income quintile^b
1 (low)	5823 (25.9)	992 (22.0)	287 (16.1)	705 (25.9)
2	5108 (22.7)	981 (21.8)	352 (19.7)	629 (23.1)
3 (middle)	4375 (19.4)	911 (20.2)	361 (20.2)	550 (20.2)
4	3959 (17.6)	849 (18.9)	395 (22.1)	454 (16.7)
5 (high)	3255 (14.4)	772 (17.1)	391 (21.9)	381 (14.0)
Patient clinical characteristics
Cause of end-stage kidney disease
Glomerulonephritis/autoimmune	2776 (12.3)	1310 (29.1)	529 (29.6)	781 (28.7)
Cystic kidney disease	933 (4.1)	592 (13.1)	255 (14.3)	337 (12.4)
Diabetes	8958 (39.8)	966 (21.4)	316 (17.7)	650 (23.9)
Renal vascular disease	3326 (14.8)	512 (11.4)	140 (7.8)	372 (13.7)
Other	3595 (16.0)	722 (16.0)	296 (16.6)	426 (15.7)
Missing/unknown	2932 (13.0)	403 (8.9)	250 (14.0)	153 (5.6)
Dialysis modality
Hemodialysis	18 009 (80.0)	2792 (62.0)	932 (52.2)	1860 (68.4)
Peritoneal dialysis	4511 (20.0)	1259 (27.9)	410 (23.0)	849 (31.2)
Preemptive kidney transplant^c		454 (10.1)	444 (24.9)	10 (0.4)
Dialysis vintage
Preemptive kidney transplant^c		454 (10.1)	444 (24.9)	10 (0.4)
<6 months		199 (4.4)	172 (9.6)	27 (1.0)
6-12 months		315 (7.0)	253 (14.2)	62 (2.3)
>12 months		3537 (78.5)	917 (51.3)	2620 (96.3)
Body mass index (kg/m²)	27 (23, 32)	26 (23, 30)	26 (23, 30)	26 (23, 30)
Cardiovascular disease^d	14 001 (62.2)	2683 (59.6)	915 (51.2)	1768 (65.0)
Diabetes^e	13 141 (58.4)	1478 (32.8)	503 (28.2)	975 (35.9)
Hypertension^e	17 670 (78.5)	3411 (75.7)	1293 (72.4)	2118 (77.9)
Charlson comorbidity index^f	4 (2, 5)	2 (2, 4)	2 (2, 4)	2 (2, 4)

Note. Comorbidities were ascertained in the 5 years prior to cohort entry. All other patient data, except time on dialysis, were captured at dialysis initiation or date of preemptive kidney transplantation. Data are presented as n (%) or median (25th, 75th percentile).

Refers to location of residence with a population size <10 000 person.

Fifths of average neighborhood income were used to categorize income.

Preemptive kidney transplant defined as no dialysis prior to the kidney transplant date.

Cardiovascular disease was defined as a composite of congestive heart failure, coronary artery disease (including angina), stroke/transient ischemic attack, peripheral vascular disease, myocardial infarction, chronic obstructive pulmonary disease, and other cardiac disease.

Diabetes and hypertension defined as the presence of either 2 Ontario Health Insurance Plan codes or 1 hospitalization with a diagnosis of diabetes or hypertension in the 5 years prior to cohort entry date.

Presence of kidney disease is a variable in the Charlson comorbidity index, which automatically results in all individuals receiving a minimum score of 2. Individuals with a Charlson comorbidity index of 0 were given a score of 2 and individuals with a score of 1 were given a score of 3.

Over 3 years, maintenance dialysis patients were followed for 54 398 person-years and kidney transplant recipients for 13 052 person-years (5271 person-years in living donors and 7781 person-years in deceased donors). During this time, 7506 (33.3%) dialysis patients died and 278 (6.2%) kidney transplant recipients died (61 [3.4%] living donors and 217 [8.0%] deceased donors). In comparison, in the US cohort, 40.0% of dialysis patients and 4.5% of kidney transplant recipients died.¹⁰

Predictive Model Discrimination

The discriminatory abilities of the models were moderate. For example, the C-statistic for 3-year mortality was 0.70 (95% confidence interval [CI]: 0.69-0.70) for maintenance dialysis patients and 0.72 (95% CI: 0.69-0.75) for kidney transplant recipients. Similar C-statistics were found for both recipients of living and deceased donor kidney transplants (Table 2).

Table 2.

C-Statistics for 3-Year Mortality Prediction for Maintenance Dialysis Patients and Kidney Transplant Recipients.

Model	C-statistic (95% CI)
Maintenance dialysis	0.70 (0.69-0.70)
Kidney transplant recipients	0.72 (0.69-0.75)
Deceased donor transplant	0.68 (0.64-0.71)
Living donor transplant	0.71 (0.64-0.78)

Note. CI = confidence interval.

Predictive Model Calibration

In maintenance dialysis patients, the 3-year observed mortality risk was slightly lower than the predicted risk (33.3% vs 35.9%). However, after we recalibrated the intercept, the predicted 3-year mortality risk was almost identical to the observed (33.5% vs 33.3%). Figure 1 demonstrates the observed and predicted mortality risk in deciles based on predicted probability ranking in maintenance dialysis patients after intercept recalibration. Figure 2a and 2b demonstrates the calibration plot before and after intercept recalibration. In kidney transplant recipients, the 3-year observed mortality risk (6.2%) was similar to the predicted mortality risk (5.8%) (Figure 3). Figure 4 demonstrates the calibration plot for kidney transplant recipients. Supplementary Figures 3 to 5 demonstrate the observed and predicted probability rankings and the calibration plots for recipients of a living donor kidney transplant and recipients of a deceased donor kidney transplant. In accordance with ICES privacy policies, cell sizes less than or equal to 5 cannot be reported; therefore, we could not report the observed versus predicted mortality risk by deciles of predicted probability for recipients of a living donor kidney transplant. Supplementary Figure 5 shows calibration plots before and after intercept recalibration in recipients of a deceased donor kidney transplant.

Figure 1.

Observed and predicted mortality probability in maintenance dialysis patients after intercept recalibration.

Figure 2.

Calibration plot comparing the observed and predicted mortality probability for maintenance dialysis patients before (a) and after (b) intercept recalibration.

Figure 3.

Observed and predicted mortality probability in kidney transplant recipients.

Figure 4.

Calibration plot comparing the observed and predicted mortality probability for kidney transplant recipients.

Sensitivity Analysis

To allow all individuals to be followed for a maximum of 3 years, we restricted our accrual period from January 1, 2004, to December 31, 2013. We included 19 866 maintenance dialysis patients and 4006 kidney transplant recipients. During 3-years of follow-up, 33.8% (n = 6712) of maintenance dialysis patients died and 6.2% (n = 247) of kidney transplant recipients died. The C-statistics were comparable with what was found in our primary analysis. For example, the C-statistic for maintenance dialysis patients was 0.70 (95% CI: 0.69-0.71) and 0.72 (95% CI: 0.68-0.75) for kidney transplant recipients (Supplementary Table 4). The calibration was also comparable. For example, the 3-year observed mortality risk in dialysis patients was 33.8% versus a predicted mortality risk of 35.9%, while for kidney transplant recipients the observed mortality risk was 6.2% versus a predicted risk of 5.7%.

Dialysis vs. Kidney Transplant- Estimated Survival in Ontario Risk Calculator

We developed a Canadian Web site (http://dialysisvstransplant.ca) and an iOS application called Dialysis vs. Kidney Transplant- Estimated Survival in Ontario (Figure 5). The Web site provides absolute and relative probabilities of dying (or surviving), provides visual representation of risk, is user friendly, and provides resources for health care professionals and patients about kidney transplantation and living donation in Ontario. The risk calculator is intended to be used by doctors, nurses, social workers, and/or patient educators with ESKD patients who do not have contraindications to transplant.

Figure 5.

Screenshot of the Dialysis vs. Kidney Transplant- Estimated Survival in Ontario risk calculator (Web site).

Discussion

We found the discriminative and predictive ability of the iChoose Kidney risk calculator to estimate mortality in Canadian maintenance dialysis patients was similar to the US population. After minor modifications, the predictive accuracy improved in our Canadian cohort. Our results suggest the Canadian version of iChoose Kidney (Dialysis vs. Kidney Transplant- Estimated Survival in Ontario) may be a valuable resource for clinical educators to use with ESKD patients and their families to help them make an informed decision about whether they should pursue kidney transplantation.

The C-statistics we found were comparable with the iChoose Kidney validation study conducted in the US population (dialysis cohort, n = 331 930 and kidney transplant recipient cohort, n = 28 739).¹⁰ For example, the C-statistic for maintenance dialysis patients in our external validation study was 0.70 (95% CI: 0.69-0.70) versus a C-statistic of 0.70 (95% CI: 0.70-0.71) in the US study.¹⁰ Similarly, the C-statistic for kidney transplant recipients in our study was 0.72 (95% CI: 0.69-0.75) compared with a C-statistic of 0.70 (95% CI: 0.69-0.72) in the US study.¹⁰ Of note, the C-statistics found in our study are slightly higher compared with other widely used prediction tools in the kidney transplant population and other populations. For example, both the Kidney Donor Risk Index and the Fracture Risk Assessment Tool have C-statistics of 0.62.^22,23

After we recalibrated the intercepts in 2 of our models, iChoose Kidney was also adequately calibrated in our Canadian ESKD patients with similar observed and predicted mortality risks. However, even prior to recalibration, the estimates were comparable. For example, in kidney transplant recipients, the 3-year observed mortality risk (6.2%) was similar to the predicted mortality risk (5.8%).

Like many places, in Ontario kidney transplant education could be improved, with patients reporting that one of the main barriers to kidney transplantation is a lack of effective education.⁷ The Dialysis vs. Kidney Transplant- Estimated Survival in Ontario risk calculator may help improve kidney transplant education with better knowledge about the prognosis with different treatment options. The Dialysis vs. Kidney Transplant- Estimated Survival in Ontario Web site and iOS application are similar to the US version which was designed using best practices to communicate risk (eg, use of simple language, displaying mortality risk visually in both absolute and relative terms).^9,24,25 A randomized controlled trial of the US iChoose Kidney tool supports the use of the tool to improve knowledge about survival in transplantation versus dialysis. Specifically, 470 patients presenting for a transplant evaluation from 3 US transplant centers were randomized to receive standard transplant education (control) versus standard transplant education plus iChoose Kidney (intervention).²⁶ Transplant knowledge improved from pre- to immediately post-evaluation to a greater extent in the intervention group than control. Furthermore, 95% of surveyed transplant nephrologists and surgeons indicated that there would be a benefit to using this tool in their regular clinical practice.²⁶

Given the survival benefit of kidney transplantation over dialysis varies by patient characteristics, large prospective studies should be conducted that incorporate other factors associated with mortality.^27,28 For example, the inclusion of factors such as a previous cancer diagnosis may improve the predictive ability of the risk calculator. Gander et al added dialysis modality (ie, peritoneal dialysis, in-center hemodialysis, and home hemodialysis) and more detailed dialysis categories (ie, <6 months, 6-12 months, 1-2, 2-3, 3-5, 5-7, 7-10, 10-14, 14+ years) to the original iChoose Kidney model and found good calibration and similar C-statistics to the original model.²⁹ However, it is also important to consider data availability, simplicity, and accuracy before adding additional risk factors that may improve the model’s predictive ability. Given the survival advantage of preemptive kidney transplantation (ie, transplant without any prior exposure to dialysis),³⁰ a future study should use Ontario data to examine the predictive accuracy of the risk calculator when categorizing kidney transplants as preemptive versus nonpreemptive. Gander et al found that C-statistics for recipients of a preemptive transplant were comparable with recipients of a nonpreemptive transplant.²⁹ Last, providing mortality estimates over a longer duration (eg, 5- and 10-year mortality risk) would provide patients with additional information to further guide informed decision-making about treatment options.

Our study has several strengths. First, to our knowledge, this is the only validated risk calculator to predict mortality in Canadian (Ontario) dialysis patients that has been translated into a format (Web site, iOS application) that can be used by health care providers to help ESKD patients make more informed treatment decisions.^31,32 Several other risk prediction models for mortality have been validated in the ESKD population but have not been translated into a format that is easily accessible to patients.^33-35 Second, concerns about selection bias in our study are minimal with universal health care benefits allowing for the inclusion of all Ontario dialysis patients and kidney transplant recipients. Last, administrative databases allowed us to accurately capture mortality.^16,17

Limitations of our study are recognized. First, our data sources precluded the inclusion of ethnicity and serum albumin. Nonetheless, the predictive ability of iChoose Kidney in our Canadian cohort was similar to the US cohort. Second, the relatively small number of events did not allow us to provide 1-year mortality estimates. Similarly, the number of events prevented us from examining the predictive accuracy of the models by year of cohort entry to ensure accuracy over time. Third, our results may not be generalizable to other Canadian provinces. However, given the US iChoose Kidney risk calculator performed relatively well in the Ontario population despite differences in patient characteristics and health care systems, it is probable that this tool would generalize to other provinces where characteristics would be expected to be more similar. It is important to note that some of the differences observed in patient comorbidities between the US population and our Canadian population may have been due to the US study capturing comorbidities using the CMS 2728 Medical Evidence Report which has been found to undercapture comorbidities.^36,37 Fourth, mortality risk estimates assume that all treatment options are immediately available to a patient. However, we know that the average wait time for a deceased donor kidney transplant in Ontario is 5 years.³⁸ Given that it is important for the patient to be informed about wait times and the impact that patient characteristics (eg, blood group and panel-reactive antibodies) can have on these wait times, on the Web site and iOS application we provide the following information: In Ontario, the average wait time for a deceased donor kidney transplant is between 3-6 years. Wait times can vary based on blood type and antibody levels. Future studies should examine the impact the inclusion of these variables has on the predictive accuracy of the risk calculator; previous studies suggest both blood group (through its impact on transplant access) and panel-reactive antibodies are associated with mortality.^39,40 Last, data availability prevented us from performing a sensitivity analysis restricting to patients on the transplant waitlist; as a result, selection bias (ie, transplant patients are often healthier than dialysis patients) is present when comparing mortality estimates in dialysis versus kidney transplantation. However, restricting our analysis in this way is not without limitations as patients may not be on the waitlist for reasons unrelated to their health (eg, may not have been educated about transplant). Given these limitations, the Dialysis vs. Kidney Transplant- Estimated Survival in Ontario risk calculator should not be used by clinicians to decide who is eligible for a transplant.

In summary, despite differences in patient characteristics and health care systems, the iChoose Kidney risk calculator was able to adequately predict mortality in maintenance dialysis patients and kidney transplant recipients from Ontario, Canada. After minor modifications, the predictive accuracy improved. The Dialysis vs. Kidney Transplant- Estimated Survival in Ontario risk calculator may be a useful resource to help Canadian ESKD patients decide whether they should pursue kidney transplantation.

Supplemental Material

Supplementary_Materials – Supplemental material for Predicting 3-Year Survival in Patients Receiving Maintenance Dialysis: An External Validation of iChoose Kidney in Ontario, Canada

Supplemental material, Supplementary_Materials for Predicting 3-Year Survival in Patients Receiving Maintenance Dialysis: An External Validation of iChoose Kidney in Ontario, Canada by Vivian S. Tan, Amit X. Garg, Eric McArthur, Rachel E. Patzer, Jennifer Gander, Pavel Roshanov, S. Joseph Kim, Greg A. Knoll, Seychelle Yohanna, Megan K. McCallum and Kyla L. Naylor in Canadian Journal of Kidney Health and Disease

Footnotes

Authors’ Note

Parts of this material are based on data and/or information compiled and provided by CIHI. However, the analyses, conclusions, opinions, and statements expressed in the material are those of the author(s), and not necessarily those of CIHI. Parts of this material are based on data and information provided by Cancer Care Ontario (CCO). The opinions, results, view, and conclusions reported in this article are those of the authors and do not necessarily reflect those of CCO. No endorsement by CCO is intended or should be inferred.

Author Contributions

REP developed iChoose Kidney. KLN and AXG approached REP to validate her tool in the Canadian setting. VT, KLN, and AXG participated in its design and coordination. EM provided analytic and statistical support. VT and KLN drafted the article. All authors read and approved the final article.

Ethics Approval and Consent to Participate

Institute for Clinical Evaluative Sciences (ICES) is a prescribed entity under section 45 of Ontario’s Personal Health Information Protection Act (PHIPA). Section 45 authorizes ICES to collect personal health information, without consent, for the purpose of analysis or compiling statistical information with respect to the management of, evaluation or monitoring of, the allocation of resources to or planning for all or part of the health system. Projects conducted under section 45, by definition, do not require review by a Research Ethics Board. This project was conducted under section 45, and approved by ICES’s Privacy and Compliance Office. ICES is a designated prescribed entity under section 45 of the PHIPA. Participant informed consent was not required for this study.

Consent for Publication

Not applicable.

Availability of Data and Material

The data set from this study is held securely in coded form at the Institute for Clinical Evaluative Sciences (ICES). While data sharing agreements prohibit ICES from making the data set publicly available, access can be granted to those who meet prespecified criteria for confidential access, available at . The full data set creation plan is available from the authors upon request.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Drs Kim and Knoll have received investigator-initiated research grants from Canadian Institutes of Health Research and Astellas Canada. Dr Garg received an investigator-initiated grant from Astellas for a Canadian Institutes of Health Research study in living kidney donors. All other authors have no conflicts of interest to declare.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is funded in part by the Canadian Institutes of Health Research (CIHR) SPOR Networks in Chronic Disease (Canadians Seeking Solutions and Innovations to Overcome Chronic Kidney Disease [Can-SOLVE CKD]). The opinions, results, and conclusions are those of the authors and are independent from the funding source. Dr Naylor is supported by the Canadian Institutes of Health Research (CIHR) Fellowship. Dr Garg is supported by the Dr Adam Linton Chair in Kidney Health Analytics. This research was made possible by infrastructure support from the Lilibeth Caberto Kidney Clinical Research Unit. This study was supported by the Institute for Clinical Evaluative Sciences (ICES) Western site. ICES is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Core funding for ICES Western is provided by the Academic Medical Organization of Southwestern Ontario (AMOSO), the Schulich School of Medicine and Dentistry (SSMD), Western University, and the Lawson Health Research Institute (LHRI). No endorsement by ICES, AMOSO, SSMD, LHRI, or the MOHLTC is intended or should be inferred. The research was conducted by members of the ICES Kidney, Dialysis and Transplantation team, at the ICES Western facility, who are supported by a grant from the Canadian Institutes of Health Research (CIHR).

ORCID iD

Pavel Roshanov

Supplemental Material

Supplemental material is available for this article online.

References

Canadian Institute for Health Information. Canadian Organ Replacement Register Annual Report: Treatment of End-Stage Organ Failure in Canada, 2001 to 2010. Ottawa, ON: Canadian Institute for Health Information; 2011.

Rana

Gruessner

Agopian

et al . Survival benefit of solid-organ transplant in the United States. JAMA Surg. 2015;150:252-259.

Ortiz

Aronen

Koskinen

et al . Health-related quality of life after kidney transplantation: who benefits the most? Transpl Int. 2014;27:1143-1151.

Canadian Institute for Health Information. Canadian Organ Replacement Register Annual Report: Treatment of End-Stage Organ Failure in Canada, 2003 to 2012. Ottawa, ON: Canadian Institute for Health Information; 2014.

Parvez

Abdel-Kader

Song

et al . Conveying uncertainty in prognosis to patients with ESRD. Blood Purification. 2015;39:58-64.

Davison

SN.

End-of-life care preferences and needs: perceptions of patients with chronic kidney disease. Clin J Am Soc Nephrol. 2010;5:195-204.

Getchell

McKenzie

Sontrop

et al . Increasing the rate of living donor kidney transplantation in Ontario: donor- and recipient-identified barriers and solutions. Can J Kidney Heal Dis. 2017;4:1-8.

Stacey

Légaré

Lewis

et al . Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2017;4:1-346.

Reyna

VF.

A theory of medical decision making and health: fuzzy trace theory. Med Decis Mak. 2008;28:850-865.

10.

Patzer

Basu

Larsen

et al . iChoose Kidney: a clinical decision aid for kidney transplantation versus dialysis treatment. Transplantation. 2016;100:630-639.

11.

Benchimol

Smeeth

Guttmann

et al . The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) Statement. PLoS Med. 2015;12:1-22.

12.

Moons

Altman

Reitsma

et al . Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162:W1-W73.

13.

Canadian Institute for Health Information. Canadian Organ Replacement Register Annual Report: Treatment of End-Stage Organ Failure in Canada, 2004 to 2013. Ottawa, ON: Canadian Institute for Health Information; 2015.

14.

Lam

McArthur

Kim

et al . Validation of kidney transplantation using administrative data. Can J Kidney Heal Dis. 2015;2:20.

15.

Ontario Ministry of Finance. Ontario Population Projections Update 2012-2036. Toronto, Canada: Government of Ontario; 2013.

16.

Quinn

Laupacis

Austin

PPC

et al . Using administrative datasets to study outcomes in dialysis patients: a validation study. Med Care. 2010;48:745-750.

17.

Iron

Zagorski

Sykora

Manuel

DG.

Living and Dying in Ontario: An Opportunity for Improved Health Information. ICES Investigative Report. Toronto, ON, Canada: Institute for Clinical Evaluative Sciences; 2008.

18.

Statistics Canada. Census Profile, 2016 Census. http://www12.statcan.gc.ca/census-recensement/2016/dp-pd/prof/details/Page.cfm?Lang=E&Geo1=PR&Code1=24&Geo2=&Code2=&Data=Count&SearchText=Quebec&SearchType=Begins&SearchPR=01&B1=All&GeoLevel=PR&GeoCode=24. Published 2017. Accessed December 10, 2017.

19.

Harrell

Jr.

Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York, NY: Springer-Verlag; 2001.

20.

Austin

Steyerberg

EW.

Graphical assessment of internal and external calibration of logistic regression models by using loess smoothers. Stat Med. 2014;33:517-535.

21.

Janssen

KJM

Moons

KGM

Kalkman

et al . Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61:76-86.

22.

Rao

Schaubel

Guidinger

et al . A comprehensive risk quantification score for deceased donor kidneys: the kidney donor risk index. Transplantation. 2009;88:231-236.

23.

Kanis

Oden

Johnell

et al . The use of clinical risk factors enhances the performance of BMD in the prediction of hip and osteoporotic fractures in men and women. Osteoporos Int. 2007;18:1033-1046.

24.

Lipkus

IM.

Numeric, verbal, and visual formats of conveying health risks: suggested best practices and future recommendations. Med Decis Mak. 2007;27:696-713.

25.

Elwyn

Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006;333:417-410.

26.

Patzer

McPherson

Basu

et al . Effect of the iChoose Kidney decision aid in improving knowledge about treatment options among transplant candidates: a randomized controlled trial. Am J Transplant. 2018;18:1954-1965.

27.

Tonelli

Wiebe

Knoll

et al . Systematic review: kidney transplantation compared with dialysis in clinically relevant outcomes. Am J Transplant. 2011;11:2093-2109.

28.

Wolfe

Ashby

Milford

et al . Comparison of mortality in all patients on dialysis, patients on dialysis awaiting transplantation, and recipients of a first cadaveric transplant. N Engl J Med. 1999;341:1725-1730.

29.

Gander

Basu

McPherson

et al . iChoose Kidney for treatment options: updated models for shared decision aid. Transplantation. 2018;102:e370-e371. doi:10.1097/TP.0000000000002301.

30.

Meier-Kriesche

Port

Ojo

et al . Effect of waiting time on renal transplant outcome. Kidney Int. 2000;58:1311-1317.

31.

Dialysis vs. Kidney Transplant. Estimated Survival in Ontario. http://dialysisvstransplant.ca. Published 2018. Accessed April 4, 2018.

32.

Dialysis vs. Kidney Transplant. App Store Preview. https://itunes.apple.com/ca/app/dialysis-vs-kidney-transplant/id1291112519?platform=ipadpreserveScrollPosition=true%23platform/ipad#platform/ipad. Published 2018. Accessed April 4, 2018.

33.

van Walraven

Austin

Knoll

Predicting potential survival benefit of renal transplantation in patients with chronic kidney disease. CMAJ. 2010;182:666-672.

34.

MacHnicki

Pinsky

Takemoto

et al . Predictive ability of pretransplant comorbidities to predict long-term graft loss and death. Am J Transplant. 2009;9:494-505.

35.

Jassal

Schaubel

Fenton

SS.

Predicting mortality after kidney transplantation: a clinical tool. Transpl Int. 2005;18:1248-1257.

36.

Malas

Wish

Moorthi

et al . A comparison between physicians and computer algorithms for form CMS-2728 reporting. Hemodial Int. 2017;21:117-124.

37.

Longenecker

Coresh

Klag

et al . Validation of comorbid conditions on the end-stage renal disease medical evidence report: the CHOICE study. Choices for healthy outcomes in caring for ESRD. J Am Soc Nephrol. 2000;11:520-529.

38.

Trillium Gift of Life Network. Kidney Donation and Transplant. https://www.giftoflife.on.ca/resources/pdf/TGLN_Kidney_Brochure_WEB.pdf. Published 2015. Accessed December 12, 2017.

39.

Glander

Budde

Schmidt

et al . The “blood group O problem” in kidney transplantation-time to change? Nephrol Dial Transplant. 2010;25:1998-2004.

40.

Sapir-Pichhadze

Tinckam

Laupacis

Logan

Beyene

Kim

SJ.

Immune sensitization and mortality in wait-listed kidney transplant candidates. J Am Soc Nephrol. 2016;27:570-578.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.43 MB