Multicriteria Decision-Making Methods for Optimal Treatment Selection in Network Meta-Analysis

Abstract

Background

Network meta-analysis exploits randomized data to compare multiple interventions and generate rankings. Selecting an optimal treatment may be complicated when multiple conflicting outcomes are evaluated in parallel.

Design

The present study suggested the incorporation of multicriteria decision-making methods in network meta-analyses to select the best intervention when multiple outcomes are of interest by creating partial and absolute rankings with the TOPSIS, VIKOR, and PROMETHEE algorithms. The TOPSIS and VIKOR techniques represent distance-based methods for compromise intervention selection, whereas the PROMETHEE analysis method allows the definition of preference and indifference thresholds. In addition, the PROMETHEE technique allows a variety of modeling options by selecting alternative preference functions. Different weights may be applied to outcomes objectively with the entropy method as well as subjectively with the analytic hierarchy process, enabling the individualization of treatment choice depending on the clinical scenario.

Results

Visualization of decision analysis may be performed with multicriteria score-adjusted scatterplots, while league tables may be constructed to depict the PROMETHEE I partial ordering of interventions. A simulated study was performed assuming equal weights of outcomes, and the TOPSIS, VIKOR, and PROMETHEE II methods were compared using a similarity coefficient, indicating a high degree of agreement among methods, especially with higher numbers of interventions.

Conclusions

Multicriteria decision analysis provides a flexible and computationally direct way of selecting compromise interventions and visualizing treatment selection in network meta-analyses. Further research should provide empirical data about the implementation of multicriteria decision analysis in real-world network meta-analyses aiming to define the most suitable method depending on the clinical question.

Highlights

Multicriteria decision-making methods can be implemented in network meta-analysis to indicate compromise interventions.

The TOPSIS, VIKOR, and PROMETHEE methods can be used for optimal treatment selection when conflicting outcomes are evaluated.

The weights of outcomes can be defined objectively or subjectively, reflecting the priorities of the decision maker.

Graphical Abstract

This is a visual representation of the abstract.

Keywords

TOPSIS VIKOR PROMETHEE weight ranking order

Network meta-analysis represents an extension of the conventional pairwise meta-analysis that enables the simultaneous comparison of multiple interventions. To achieve this, data from randomized controlled trials with 2 or more arms are pooled, exploiting both direct and indirect evidence.¹ This process offers greater precision in effect estimates and allows the ranking of treatments regarding specific outcomes of interest.² For this purpose, in Bayesian network meta-analysis, the interventions are ordered according to their surface under the cumulative ranking curve (SUCRA) ratings, which are derived by the distribution of rank probabilities.³ Correspondingly, in the frequentist framework, the P-score has been proposed as a ranking tool, expressing the degree of certainty that a treatment is better than its alternatives.⁴

Decision making in health care is multifactorial, since selecting an optimal treatment plan relies on the interplay of various factors, such as efficacy, safety, acceptability, and patient-related outcomes.⁵ Nonetheless, discrepancies are anticipated among the ranking lists of different outcomes, precluding the direct draw of conclusions concerning the most appropriate intervention. As a result, a synthesis of the rankings estimated by network meta-analyses is needed to identify treatments providing the optimal balance in the investigated outcomes of interest. In this line, cluster analysis has been proposed as an initial method to detect patterns among different treatments in regard to different outcomes,⁶ although computational complexity issues along with the lack of direct implications for decision making have limited its widespread use in network meta-analyses. Moreover, a partial ordering approach has been also suggested that schematically illustrates the ordering relations but without indicating the most desirable treatment alternative.⁷

Health economic models have been recently incorporated into network meta-analyses aiming to combine the generated estimates of relative efficacy with cost data and investigate the relative cost-effectiveness of interventions.⁸ In this context, several models can be applied, such as the Markov multistate and continuous-time semi-Markov models, and estimate cost-effectiveness using measures including the incremental cost-effectiveness ratio per quality-adjusted life-year gained or the incremental net monetary benefit. However, the implementation of such methodologies may be limited by their computational complexity in terms of assumptions and parameter definition, as well as by the fact that long-term projections are performed using short-term data derived by trials.^9,10 Interestingly. Chaimani et al.¹¹ proposed the Probability of Selecting a Treatment to Recommend (POST-R) as a novel measure to generate rankings in network meta-analysis using a Markov chain approach, which also takes into account additional information, such as the amount of confidence in evidence, clinical experience, and cost of treatments. Nonetheless, such an approach may be based on expert opinions, introducing a significant degree of subjectivity.

The present study aims to provide a computationally direct way of generating rankings in network meta-analysis; to achieve this, multicriteria decision-making methods are incorporated to define compromise interventions by taking into account multiple outcomes. To this end, 3 different approaches are described and implemented in both empirical and simulated data exploiting the estimated SUCRA or P-score rankings of treatments. In addition, an objective and a subjective method for assigning weights to outcomes according to the clinical scenario are discussed.

In the first part, the methodology of applying the 3 decision-making algorithms in network meta-analyses is described. Subsequently, these methods are implemented in 2 real-world meta-analyses, using 2 approaches for determining weights to different outcomes. Finally, the degree of concordance among the 3 proposed decision-making methods is tested through a simulation study.

Materials and Methods

Multicriteria Decision Analysis

TOPSIS method

The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) approach¹² aims to select the most appropriate intervention as the one with the maximum distance from the positive ideal point and the minimum distance from the negative ideal point. These points represent hypothetical interventions, with coordinates being the maximum and minimum SUCRA or P-score values of the data set for each outcome, respectively. The basis for calculations is the decision matrix:

X = [\begin{matrix} \begin{matrix} x_{11} \\ x_{21} \end{matrix} \begin{matrix} x_{12} & \dots \\ x_{22} & \dots \end{matrix} \begin{matrix} x_{1 n} \\ x_{2 n} \end{matrix} \\ \begin{matrix} ⋮ \\ x_{m 1} \end{matrix} \begin{matrix} ⋮ & \dots \\ x_{m 2} & \dots \end{matrix} \begin{matrix} ⋮ \\ x_{mn} \end{matrix} \end{matrix}]

(1)

where n represents the number of outcomes and m the number of interventions.

At first, a normalized matrix is constructed with the vectorial procedure:

r_{ij} = \frac{x_{ij}}{\sqrt{\sum_{i = 1}^{m} x_{ij}^{2}}}

(2)

where $x_{ij}$ is the original SUCRA or P-score value of interventions for each outcome and $r_{ij}$ is its respective normalized value. Subsequently, a weighted normalized decision matrix ( $v_{ij}$ ) is created as follows:

v_{ij} = w_{j} r_{ij}

(3)

where $w_{j}$ represents the weight of the j^th outcome. In the next step, the positive ( $v_{j}^{+}$ ) and negative ( $v_{j}^{-}$ ) ideal interventions are determined using the equations (4) and (5).

v_{j}^{+} = {v_{1}^{+}, v_{2}^{+}, \dots, v_{n}^{+}} = {ma x_{ij} (v_{ij}^{+})}

(4)

v_{j}^{-} = {v_{1}^{-}, v_{2}^{-}, \dots, v_{n}^{-}} = {mi n_{ij} (v_{ij}^{-})}

(5)

Then, the distances of each point of the above ideals ( $D_{i}^{+} and D_{i}^{-}$ ) are estimated using the following equations.

D_{i}^{+} = \sqrt{\sum_{j = 1}^{n} {(v_{ij} - v_{j}^{+})}^{2}}

(6)

D_{i}^{-} = \sqrt{\sum_{j = 1}^{n} {(v_{ij} - v_{j}^{-})}^{2}}

(7)

The last step consists of estimating the score ( $S_{i}$ ) of each intervention using the equation (8).

S_{i} = \frac{D_{i}^{-}}{D_{i}^{+} + D_{i}^{-}}

(8)

Then, interventions are ranked according to their S score, with S closer to 1 indicating a better intervention. The TOPSIS analysis was performed in R (package “TOPSIS”¹³).

VIKOR method

The Vise Kriterijumska Optimizacija I Kompromisno Resenje (VIKOR) method¹⁴ aims to determine a set of compromise interventions to guide decision makers when contradictory outcomes are present. The methodology is based on the L^p metric and creates rankings according to the amount of proximity to an ideal solution. First, the decision matrix (1) is used, and for each outcome, the best ( $f_{i}^{*}$ ) and worst ( $f_{i}^{-}$ ) performance are determined. In case the j^th outcome is beneficial, then

f_{i}^{*} = max_{i} x_{ij}, f_{i}^{-} = min_{i} x_{ij}

(9)

while if the j^th outcome is nonbeneficial, then

f_{i}^{*} = min_{i} x_{ij}, f_{i}^{-} = max_{i} x_{ij},

(10)

In the next step, the S_i and R_i values are calculated with equations (11) and (12).

S_{i} = \sum_{i = 1}^{n} w_{j} \frac{| f_{i}^{*} - x_{ij} |}{| f_{i}^{-} - x_{ij} |}

(11)

R_{i} = max_{j} w_{j} \frac{| f_{i}^{*} - x_{ij} |}{| f_{i}^{-} - x_{ij} |}

(12)

where $w_{j}$ is the weight of the j^th outcome. Then, the Q_i value is estimated with the equation (13).

Q_{i} = v \frac{(S_{i} - S^{*})}{(S^{-} - S^{*})} + (1 - v) \frac{(R_{i} - R^{*})}{(R^{-} - R^{*})}

(13)

where $S^{*} = min_{i} S_{i}$ , $S^{-} = max_{i} S_{i}$ , $R^{*} = min_{i} R_{i}$ , $R^{-} = max_{i} R_{i}$ .

The intervention with the lowest $S_{i}$ value corresponds to the intervention with the maximum group utility of the majority. On the other hand, the $R_{i}$ value expresses the maximum regret across outcomes and should therefore be minimized. The $Q_{i}$ value takes into account both values. The parameter $v \in [0, 1]$ expresses the weight used by the decision maker to combine the $S_{i}$ and $R_{i}$ values, with $v$ corresponding to the weight of the maximum group utility strategy and 1 − $v$ to the weight of the individual regret. Typically, $v$ is set to the value of 0.5 to compromise the $S_{i}$ and $R_{i}$ values.

Subsequently, the interventions are ranked according to their Q values, and the intervention with the lowest Q is proposed as the compromise solution if 2 conditions are satisfied. Specifically, the plausibility of the C1 condition of acceptable advantage is satisfied if

Q (A ″) - Q (A') \geq \frac{1}{m - 1}

(14)

where $A ″$ is the intervention at the second position and $A'$ the intervention at the first position of the $Q$ ranking, while $m$ is the number of interventions. The C2 condition of acceptable stability is satisfied in case the $A'$ intervention has the minimum $S$ and/or $R$ value. If the C2 condition is not satisfied, both interventions ${A', A ″}$ are proposed as compromise solutions. In case the C1 condition is not satisfied, the compromise solution is the set ${A', A ″, \dots, A^{(M)}}$ , for the maximum $M$ that the following relation applies:

Q (A^{(M)}) - Q (A') < \frac{1}{m - 1}

(15)

The analysis with the VIKOR method was applied using R (package “MCDM”¹⁵).

PROMETHEE method

The Preference Ranking Organization Method for Enrichment of Evaluations (PROMETHEE) algorithms were introduced by Brans and Vincke¹⁶ to rank a set of interventions with conflicting outcomes. The PROMETHEE is a family of outranking methods, with PROMETHEE I providing partial and PROMETHEE II complete rankings. A preference function is introduced, while thresholds of preference ( $p$ ) and indifference ( $q$ ) may be applied. Specifically, an evaluation matrix is created as the decision matrix (1). Then, the preference function is defined using 1 of the following generalized criteria: usual, U-shape, V-shape, level, linear, and Gaussian.¹⁷ The present study uses the linear unicriterion preference function, which is commonly implemented and is defined as follows:

P (d) = {\begin{matrix} 0, if | d | \leq q \\ \frac{| d | - q}{p - q}, if q < | d | \leq p \\ 1, if | d | > p \end{matrix}

(16)

where $d$ is the SUCRA or P-score difference between 1 interventions, $p$ the preference threshold and $q$ the indifference threshold. In the next step, the overall preference index is estimated using equation (17).

Π (α, β) = \sum_{j = 1}^{n} w_{j} P_{j} (a, β)

(17)

where $Π (α, β)$ is the overall preference intensity of $α$ over $β$ with regard to all outcomes, $w_{j}$ is the weight of the j^th outcome, and $P_{j} (a, β)$ is the preference function of $α$ over $β$ with regard to the j^th outcome. Then, the outranking flows are calculated. The positive outranking flow $φ^{+} (a)$ indicates the superiority of $α$ over the other interventions, whereas the negative outranking flow $φ^{-} (a)$ indicates the superiority of all other interventions over $α$ .

φ^{+} (a) = \frac{1}{m - 1} \sum Π (α, x)

(18)

φ^{-} (a) = \frac{1}{m - 1} \sum Π (x, a)

(19)

According to the PROMETHEE I approach, $α$ is preferred to $β$ if

{\begin{matrix} φ^{+} (a) > φ^{+} (β) and φ^{-} (a) < φ^{-} (β), or \\ φ^{+} (a) = φ^{+} (β) and φ^{-} (a) < φ^{-} (β), or \\ φ^{+} (a) > φ^{+} (β) and φ^{-} (a) = φ^{-} (β) \end{matrix}

(20)

Alternatively, $α$ is indifferent to $β$ if

φ^{+} (a) = φ^{+} (β) and φ^{-} (a) = φ^{-} (β)

(21)

while $α$ is incomparable to $β$ if

{\begin{matrix} φ^{+} (a) > φ^{+} (β) and φ^{-} (a) > φ^{-} (β), or \\ φ^{+} (a) < φ^{+} (β) and φ^{-} (a) < φ^{-} (β) \end{matrix}

(22)

For the PROMETHEE II method, the net outranking flow ( $φ (α)$ ) is estimated with equation (23).

φ (α) = φ^{+} (a) - φ^{-} (a)

(23)

As a result, a complete ranking can be obtained by ordering interventions according to their net outranking flows, with higher values indicating better interventions. The PROMETHEE I and II analyses were performed in R (package “PROMETHEE”¹⁷).

Estimation of Weights

Entropy method

The entropy method¹⁸ is an objective approach for weight assignment, expressing the degree of information uncertainty. The entropy is the measure of the disorder of a system; hence, information entropy reflects the amount of useful information that can be derived. In this line, outcomes with high dispersion of the measured values (SUCRAs or P-scores) are supposed to provide more information and are thus attributed with higher weights. For the estimation of weights, the decision matrix (1) is first normalized.

r_{ij} = \frac{x_{ij} - min_{i} x_{ij}}{max_{i} x_{ij} - min_{i} x_{ij}}

(24)

Then, the entropy value ( $e_{j}$ ) of the j^th outcome is estimated with equation (25).

e_{j} = - \frac{\sum_{i = 1}^{m} f_{ij} \ln f_{ij}}{\ln m}

(25)

where $f_{ij} = \frac{r_{ij}}{\sum_{i = 1}^{m} r_{ij}}$ .

Finally, the entropy weights ( $w_{j}$ ) are calculated with the following equation.

w_{j} = \frac{1 - e_{j}}{\sum_{j = 1}^{n} 1 - e_{j}}

(26)

A sample R script for entropy weight estimation is provided in Supplementary Appendix 1.

Analytic hierarchy process

The analytic hierarchy process¹⁹ is a subjective method for determining weights, which is based on a hierarchy including the decision goal, the interventions, and the outcomes used for their assessment. Priorities are defined by constructing a pairwise comparison system; to achieve this, verbal evaluations are converted to numerical integer values ranging from 1 to 9, with higher values indicating the stronger importance of one outcome over another. As a result, a pairwise comparison matrix (A) is created as the following.

A = [\begin{matrix} \begin{matrix} C_{11} & C_{12} \\ C_{21} & C_{22} \end{matrix} \\ \begin{matrix} ⋮ & ⋮ \end{matrix} \\ \begin{matrix} C_{n 1} & C_{n 2} \end{matrix} \end{matrix} \begin{matrix} \begin{matrix} \dots & C_{1 n} \\ \dots & C_{2 n} \end{matrix} \\ \begin{matrix} ⋱ & ⋮ \end{matrix} \\ \begin{matrix} \dots & C_{nn} \end{matrix} \end{matrix}]

(27)

where $n$ is the number of outcomes. Then, a normalized matrix ( $Y_{ij}$ ) is generated by dividing each element by the sum of its column.

Y_{ij} = \frac{C_{ij}}{\sum_{i = 1}^{n} C_{ij}}

(28)

Subsequently, the weights ( $w_{ij}$ ) are estimated by averaging the elements of normalized pairwise matrix rows.

w_{ij} = \frac{\sum_{j = 1}^{n} Y_{ij}}{n}

(29)

The next step is to test the consistency of the matrix. Specifically, the maximum eigenvalue ( $λ_{\max}$ ) of the nonnormalized pairwise matrix is estimated with equation (30).

λ_{\max} = \frac{1}{n} \sum_{i = 1}^{n} \frac{{(Aw)}_{i}}{w_{i}}

(30)

Then, the consistency index (CI) is calculated using the maximum eigenvalue as follows.

CI = \frac{λ_{\max} - n}{n - 1}

(31)

Finally, the consistency ratio (CR) is estimated by dividing the CI by the random index (RI) that corresponds to the CI of a randomly generated pairwise comparison matrix.

CR = \frac{CI}{RI} \times 100 %

(32)

Values of CR less than 10% are acceptable, indicating consistency of the matrix.²⁰

Similarity Coefficient

The similarity of rankings generated by the TOPSIS, VIKOR, and PROMETHEE II methods was quantified using the similarity coefficient proposed by Sałabun and Urbaniak.²¹ This coefficient is considered to be appropriate for decision-making purposes, as the positions at the top of the ranking exert a greater impact on the estimated similarity. The similarity coefficient (WS) is calculated by equation (33).

WS = 1 - \sum_{i = 1}^{n} (2^{- R_{xi}} \frac{| R_{xi} - R_{yi |}}{max {| 1 - R_{xi} |, | N - R_{xi} |}})

(33)

where $R_{xi}$ and $R_{yi}$ are the positions of the i^th element in the $x$ and $y$ ranking, respectively, while $N$ is the total number of interventions. The $WS$ coefficient ranges from 0 to 1, with values closer to 1 indicating higher similarity. An R script for the calculation of the $WS$ coefficient is reported in Supplementary Appendix 2.

Published Data Sets

Antidepressants for major depressive disorder

In a network meta-analysis of 522 double-blinded randomized controlled trials, the relative efficacy and acceptability of 22 treatments (21 antidepressants and placebo) were evaluated following a Bayesian approach.²² A total of 116,477 patients were included, while the odds ratio was chosen as the effect measure. Efficacy was defined as the response rate, quantified by the proportion of patients with a reduction of ≥50% of the total depression score, while acceptability referred to treatment discontinuation due to any reason. Interventions were ranked according to their SUCRA values (Table 1).

Table 1

SUCRA Values for Efficacy and Acceptability of Antidepressants for Major Depressive Disorder

Intervention	SUCRA (%)
Intervention	Efficacy	Acceptability
Agomelatine	51.6	91.9
Amitriptyline	98.6	66.9
Bupropion	41.4	61.6
Citalopram	30.4	69.7
Clomipramine	29.4	7.9
Desvenlafaxine	27.7	34.1
Duloxetine	83.0	29.5
Escitalopram	56.9	79.5
Fluoxetine	29.2	87.4
Fluvoxamine	58.1	28.9
Levomilnacipran	44.5	18.0
Milnacipran	64.6	62.3
Mirtazapine	85.7	55.1
Nefazodone	55.3	69.0
Paroxetine	70.1	67.4
Placebo	0.0	50.7
Reboxetine	13.6	19.7
Sertraline	55.2	64.2
Trazodone	30.8	22.2
Venlafaxine	74.3	40.8
Vilazodone	45.2	25.1
Vortioxetine	54.6	47.9

SUCRA, surface under the cumulative ranking curve.

Multicriteria decision analysis was performed using the TOPSIS, VIKOR, and PROMETHEE I and II methods. For the PROMETHEE analysis, the preference SUCRA threshold was set at 5% and the indifference threshold at 1%. A 2-dimensional plot reflecting the TOPSIS $S$ scores of interventions was constructed as a visualization tool. The script for the plot generation is available in Supplementary Appendix 3. The PROMETHEE I partial rankings were illustrated schematically in a league table. The analysis was conducted for equal and entropy weights. The similarity of methods was assessed with the $WS$ similarity coefficient. The R code for the major depressive disorder analysis is provided in Supplementary Appendix 4.

Pharmacologic treatments for acute bipolar depression

A total of 42 studies were pooled in a network meta-analysis comparing the effects of 21 interventions.²³ A frequentist model was applied, while summary odds ratios and standardized mean differences were reported. The outcomes of interest were the following: response rate (50% reduction in the total depression score), remission rate (Montgomery-Åsberg Depression Rating Scale <10 or Hamilton Depression Rating Scale <7), reduction in depression severity (change in rating score), acceptability (discontinuation out due to any reason), and tolerability (treatment-emergent affective switch). The estimated P-scores were used to rank the pharmacologic interventions (Table 2).

Table 2

P-Score Values for Multiple Outcomes of the Acute Bipolar Depression Data Set

Intervention	P-Score
Intervention	Response	Remission	Depression Severity	Acceptability	Tolerability
Aripiprazole	0.80	0.86	0.31	0.88	0.21
Carbamazepine	0.46	0.76	0.19	0.33	0.54
Cariprazine	0.58	0.53	0.70	0.70	0.53
Divalproex	0.28	0.40	0.85	0.54	0.62
Escitalopram	0.39	0.44	0.47	0.45	0.64
Fluoxetine	0.19	0.13	0.87	0.27	0.67
Gabapentin	0.87	0.70	0.01	0.57	0.52
Imipramine	0.29	0.39	0.70	0.42	0.16
Lamotrigine	0.60	0.56	0.31	0.57	0.52
Lithium	0.71	0.67	0.42	0.63	0.49
Lurasidone	0.31	0.48	0.82	0.59	0.60
Moclobemide	0.46	0.29	0.75	0.45	0.31
Olanzapine/Fluoxetine	0.31	0.33	0.48	0.39	0.47
Olanzapine	0.62	0.52	0.65	0.47	0.39
Paroxetine	0.79	0.70	0.39	0.63	0.28
Phenelzine	0.39	0.60	0.65	0.71	0.42
Quetiapine	0.46	0.42	0.54	0.52	0.29
Sertraline	0.54	0.37	0.43	0.37	0.86
Tranylcypromine	0.01	0.03	0.55	0.08	0.71
Venlafaxine	0.12	0.15	0.39	0.02	0.49
Ziprasidone	0.96	0.88	0.26	0.78	0.65

Absolute rankings were obtained with the TOPSIS, VIKOR and PROMETHEE II algorithms. For the PROMETHEE II analysis, the preference P-score threshold was set at 0.05 and the indifference threshold at 0.01. The similarity of rankings was evaluated using the $WS$ similarity coefficient. The outcomes weights were determined subjectively with the analytic hierarchy process, examining two different scenarios. In the first scenario, the decision-maker places more impact on the efficacy of interventions, while in the second one more importance is placed on the treatment acceptability and tolerability. Specifically, pairwise comparison matrices were constructed using a scale of relative importance. The R code for the acute bipolar depression analysis is provided in Supplementary Appendix 5.

Simulated Data

The simulation study consisted of 1,000 simulated data sets of SUCRA values, ranging from 0% to 100% for different scenarios of available interventions and outcomes. Specifically, the number of alternative interventions was set to be 10, 15, or 25, while the evaluated outcomes were considered to be 2, 4, 6, or 8. As a result, 12 different scenarios were tested, and overall, 12,000 simulated data sets were created. SUCRA values were generated using the uniform distribution for each intervention and outcome. For each case, multicriteria decision analysis was applied with the TOPSIS, VIKOR, and PROMETHEE II methods, obtaining absolute rankings of the interventions. For the PROMETHEE II analysis, the linear preference function was applied, while the preference and indifference SUCRA thresholds were set at 5% and 1%, respectively. Equal weights of the outcomes were hypothesized. The similarity of rankings was evaluated using the $WS$ ranking coefficient. A sample R script for the simulation process is available in Supplementary Appendix 6.

Results

Antidepressants for Major Depressive Disorder

The relationship between the SUCRA values for efficacy and acceptability is depicted in Figure 1. The outcomes of the absolute rankings generated by the TOPSIS, VIKOR, and PROMETHEE II methods are presented in Table 3. All methods indicated amitriptyline as the best intervention. More specifically, TOPSIS indicated as top interventions those with the highest S score, namely, amitriptyline (S: 0.821, rank 1), followed by mirtazapine (S: 0.715, rank 2) and paroxetine (S: 0.710, rank 3). Correspondingly, the VIKOR method indicated as best the following interventions with the lowest Q values: amitriptyline (Q: 0.004, rank 1), paroxetine (Q: 0.099, rank 2), and milnacipran (Q: 0.183, rank 3). It should be noted that both assumptions of the VIKOR algorithm were satisfied, as the difference between the Q values of paroxetine and amitriptyline was greater than 1/21, while amitriptyline had the lowest S and R values. The PROMETHEE II method demonstrated that the top treatments with the highest net Φ were amitriptyline (net Φ: 0.369, rank 1), escitalopram (net Φ: 0.286, rank 2), and paroxetine (net Φ: 0.274, rank 3). The similarity coefficient was 0.963 for the TOPSIS–VIKOR comparison, 0.949 for the VIKOR–PROMETHEE II comparison, and 0.965 for the TOPSIS–PROMETHEE II comparison.

Figure 1

Scatterplot of surface under the cumulative ranking curve (SUCRA) values for efficacy and acceptability. Colors indicate the TOPSIS S score of interventions.

Table 3

TOPSIS, VIKOR, and PROMETHEE II Rankings of Antidepressants Using Equal and Entropy-Defined Weights^a

Intervention	Equal Weights								Entropy Weights
	TOPSIS		VIKOR				PROMETHEE II		TOPSIS		VIKOR				PROMETHEE II
	S	Rank	S	R	Q	Rank	Net Φ	Rank	S	Rank	S	R	Q	Rank	Net Φ	Rank
Placebo	0.286	19	0.745	0.500	0.919	21	−0.262	17	0.356	16	0.703	0.416	0.674	17	−0.222	17
Agomelatine	0.677	5	0.238	0.238	0.193	5	0.238	4	0.744	2	0.198	0.198	0.052	2	0.282	3
Amitriptyline	0.821	1	0.149	0.149	0.004	1	0.369	1	0.779	1	0.174	0.174	0.004	1	0.347	1
Bupropion	0.512	13	0.470	0.290	0.429	11	−0.036	15	0.554	11	0.452	0.242	0.284	10	−0.014	14
Citalopram	0.490	14	0.478	0.346	0.514	13	0.036	12	0.563	10	0.442	0.288	0.334	11	0.086	10
Clomipramine	0.213	21	0.851	0.500	0.993	22	−0.405	21	0.169	21	0.876	0.584	1.000	22	−0.421	21
Desvenlafaxine	0.294	18	0.704	0.360	0.691	16	−0.286	19	0.300	18	0.701	0.402	0.655	16	−0.266	18
Duloxetine	0.571	10	0.451	0.371	0.530	14	0.083	10	0.484	14	0.499	0.434	0.550	14	0.030	12
Escitalopram	0.678	4	0.285	0.211	0.188	4	0.286	2	0.727	3	0.262	0.176	0.070	3	0.306	2
Fluoxetine	0.549	11	0.379	0.352	0.452	12	0.060	11	0.638	8	0.324	0.293	0.256	9	0.125	9
Fluvoxamine	0.452	15	0.580	0.375	0.627	15	−0.012	14	0.391	15	0.609	0.438	0.633	15	−0.050	15
Levomilnacipran	0.333	17	0.714	0.440	0.812	19	−0.274	18	0.276	19	0.742	0.514	0.820	20	−0.304	19
Milnacipran	0.652	6	0.349	0.176	0.183	3	0.179	7	0.650	7	0.349	0.206	0.168	6	0.165	7
Mirtazapine	0.715	2	0.284	0.219	0.199	6	0.238	4	0.662	5	0.310	0.256	0.201	8	0.202	6
Nefazodone	0.627	7	0.356	0.220	0.250	7	0.202	6	0.660	6	0.342	0.183	0.135	5	0.224	5
Paroxetine	0.710	3	0.290	0.146	0.099	2	0.274	3	0.709	4	0.291	0.170	0.083	4	0.268	4
Reboxetine	0.139	22	0.861	0.431	0.903	20	−0.429	22	0.140	22	0.861	0.502	0.890	21	−0.425	22
Sertraline	0.605	8	0.385	0.220	0.271	8	0.119	8	0.627	9	0.376	0.192	0.171	7	0.127	8
Trazodone	0.259	20	0.759	0.415	0.808	18	−0.298	20	0.231	20	0.771	0.484	0.805	19	−0.308	20
Venlafaxine	0.589	9	0.427	0.304	0.419	10	0.119	8	0.525	12	0.458	0.355	0.426	13	0.079	11
Vilazodone	0.361	16	0.668	0.398	0.720	17	−0.202	16	0.314	17	0.690	0.464	0.723	18	−0.220	16
Vortioxetine	0.521	12	0.485	0.262	0.400	9	0.000	13	0.506	13	0.492	0.306	0.390	12	−0.012	13

Amitriptyline emerged as the best intervention (bold text).

The partial ordering of alternatives according to the PROMETHEE I method is illustrated in Figure 2, suggesting that agomelatine is indifferent to mirtazapine, while sertraline is incomparable to venlafaxine. The entropy method assigned a weight of 0.416 and 0.584 to the efficacy and acceptability outcomes, respectively. The entropy-weighted analysis provided similar outcomes, with amitriptyline ranking first with all methods.

Figure 2

League table demonstrating the partial ordering of antidepressants according to the PROMETHEE I method.

Pharmacologic Treatments for Acute Bipolar Depression

The pairwise comparison matrices are presented in Table 4. In the first scenario, the outcome of response is considered of moderate importance compared with remission, of strong importance compared with depression severity, of very strong importance compared with acceptability, and of extreme importance compared with tolerability. On the other hand, in the second scenario, the outcome of acceptability is deemed to be of moderate importance compared with tolerability, of strong importance compared with response, of very strong importance compared with remission, and of extreme importance compared with depression severity. As a result, in the first scenario, higher weights are assigned to the outcomes of response ( $w = 0.522$ ) and remission ( $w = 0.238$ ), while in the second scenario, more impact is given to acceptability ( $w = 0.505)$ and tolerability ( $w = 0.262)$ . The matrices are considered to be consistent, since their $CRs$ are estimated below 10%.

Table 4

Determination of Weights from the Pairwise Comparison Matrix of the Analytic Hierarchy Process

Criteria	Response	Remission	Depression Severity	Acceptability	Tolerability	Weight
Scenario 1
Response	1	3	6	7	9	0.522
Remission	$\frac{1}{3}$	1	2	5	7	0.238
Depression severity	$\frac{1}{6}$	$\frac{1}{2}$	1	3	5	0.137
Acceptability	$\frac{1}{7}$	$\frac{1}{5}$	$\frac{1}{3}$	1	3	0.068
Tolerability	$\frac{1}{9}$	$\frac{1}{7}$	$\frac{1}{5}$	$\frac{1}{3}$	1	0.035
Consistency ratio						5.1%
Scenario 2
Response	1	3	5	$\frac{1}{5}$	$\frac{1}{3}$	0.136
Remission	$\frac{1}{3}$	1	2	$\frac{1}{7}$	$\frac{1}{5}$	0.060
Depression severity	$\frac{1}{5}$	$\frac{1}{2}$	1	$\frac{1}{9}$	$\frac{1}{7}$	0.037
Acceptability	5	7	9	1	3	0.505
Tolerability	3	5	7	$\frac{1}{3}$	1	0.262
Consistency ratio						4.2%

The results of the TOPSIS, VIKOR and PROMETHEE II rankings are exhibited in Table 5. In both scenarios, tranylcypromine ranked first by all methods, with VIKOR assumptions being satisfied in both cases. Specifically, in both scenarios, tranylcypromine achieved the highest TOPSIS S score (scenario 1: 0.924, scenario 2: 0.897), the lowest VIKOR Q value (scenario 1: 0.000, scenario 2: 0.000), and the highest PROMETHEE II net Φ value (scenario 1: 0.175, scenario 2: 0.179). Therefore, it can be assumed that tranylcypromine may be the most appropriate intervention irrespective of whether the clinical management plan gives priority to efficacy or acceptability. For the first and second scenario, the similarity coefficient was 0.998 and 0.999 for the TOPSIS-VIKOR comparison, 0.978 and 0.977 for the VIKOR-PROMETHEE II comparison, and 0.979 and 0.978 for the TOPIS-PROMETHEE II comparison, respectively.

Table 5

TOPSIS, VIKOR, and PROMETHEE II Rankings of Interventions for Acute Bipolar Depression Placing More Importance on Efficacy (Scenario 1) and Acceptability (Scenario 2)^a

Intervention	Scenario 1								Scenario 2
	TOPSIS		VIKOR				PROMETHEE II		TOPSIS		VIKOR				PROMETHEE II
	S	Rank	S	R	Q	Rank	Net Φ	Rank	S	Rank	S	R	Q	Rank	Net Φ	Rank
Aripiprazole	0.166	19	0.855	0.432	0.865	19	−0.164	20	0.054	21	0.944	0.504	1.000	21	−0.185	21
Carbamazepine	0.463	13	0.602	0.246	0.519	13	−0.052	15	0.609	5	0.447	0.182	0.344	5	0.070	5
Cariprazine	0.421	14	0.549	0.311	0.558	14	−0.033	13	0.289	17	0.646	0.399	0.705	17	−0.066	16
Divalproex	0.696	4	0.307	0.148	0.242	4	0.109	4	0.461	10	0.460	0.305	0.489	9	0.044	8
Escitalopram	0.584	8	0.432	0.208	0.380	8	0.035	9	0.533	7	0.435	0.252	0.415	7	0.064	6
Fluoxetine	0.823	3	0.155	0.098	0.102	3	0.170	2	0.721	3	0.250	0.147	0.187	3	0.163	2
Gabapentin	0.122	20	0.858	0.470	0.908	20	−0.156	19	0.367	15	0.658	0.322	0.627	15	−0.062	15
Imipramine	0.685	5	0.348	0.153	0.272	5	0.100	5	0.474	8	0.569	0.262	0.507	10	0.015	9
Lamotrigine	0.377	16	0.622	0.322	0.613	16	−0.077	16	0.386	14	0.596	0.322	0.590	14	−0.042	14
Lithium	0.274	17	0.701	0.382	0.723	17	−0.103	17	0.319	16	0.661	0.357	0.668	16	−0.088	17
Lurasidone	0.655	7	0.355	0.164	0.288	6	0.067	7	0.413	12	0.508	0.334	0.551	12	−0.001	11
Moclobemide	0.562	9	0.399	0.246	0.401	9	0.052	8	0.466	9	0.546	0.252	0.481	8	0.007	10
OFC	0.670	6	0.360	0.164	0.291	7	0.080	6	0.559	6	0.444	0.217	0.381	6	0.054	7
Olanzapine	0.389	15	0.565	0.333	0.591	15	−0.049	14	0.449	11	0.571	0.264	0.510	11	−0.028	12
Paroxetine	0.203	18	0.769	0.426	0.809	18	−0.129	18	0.269	19	0.755	0.357	0.724	18	−0.125	20
Phenelzine	0.561	10	0.479	0.208	0.407	10	0.003	12	0.269	18	0.673	0.404	0.728	19	−0.097	18
Quetiapine	0.530	11	0.477	0.246	0.446	11	0.007	11	0.398	13	0.613	0.293	0.567	13	−0.036	13
Sertraline	0.469	12	0.484	0.290	0.497	12	0.012	10	0.629	4	0.324	0.205	0.297	4	0.112	4
Tranylcypromine	0.924	1	0.065	0.052	0.000	1	0.175	1	0.897	1	0.105	0.056	0.000	1	0.179	1
Venlafaxine	0.837	2	0.191	0.079	0.101	2	0.131	3	0.818	2	0.184	0.138	0.139	2	0.124	3
Ziprasidone	0.067	21	0.927	0.519	1.000	21	−0.181	21	0.255	20	0.747	0.445	0.817	20	−0.099	19

OFC, olanzapine/fluoxetine combination.

Tranylcypromine emerged as the best treatment in both scenarios (bold text).

Simulated Data

The similarity of absolute ranking methods for various scenarios of different numbers of interventions and outcomes is schematically presented in Figure 3. Specifically, a high degree of similarity was observed for the TOPSIS-VIKOR comparison, with a median $WS$ similarity coefficient ranging from 0.917 to 0.993 in case of 2 outcomes, from 0.911 to 0.977 in case of 4 outcomes, from 0.890 to 0.943 in case of 6 outcomes, and from 0.871 to 0.930 in case of 8 outcomes. Regarding the comparison of VIKOR and PROMETHEE II, the median similarity coefficient ranged from 0.917 to 0.979 for 2 outcomes, from 0.912 to 0.955 for 4 outcomes, from 0.884 to 0.935 for 6 outcomes, and from 0.856 to 0.925 for 8 outcomes. Correspondingly, for the comparison of TOPSIS and PROMETHEE II, the median similarity coefficient ranged from 0.909 to 0.980 for 2 outcomes, 0.912 to 0.973 for 4 outcomes, from 0.897 to 0.972 for 6 outcomes, and from 0.919 to 0.973 for 8 outcomes. Overall, the inspection of boxplots indicated a trend toward greater similarity of methods with an increasing number of interventions and a lower number of outcomes.

Figure 3

Boxplots of simulated data displaying the similarity of TOPSIS, VIKOR, and PROMETHEE II methods for a different number of interventions and outcomes.

Discussion

Network meta-analysis constitutes an important tool for decision making in health care, as it enables the ranking of interventions preserving randomization. Network meta-analyses typically implement a univariate approach, although multiple outcomes are often evaluated in parallel. Therefore, decision making requires the identification of compromise solutions, taking into account the treatment rankings across various outcomes. To this end, the present study suggested the application of the TOPSIS, VIKOR, and PROMETHEE II methods in network meta-analysis to obtain absolute multicriteria rankings based on the estimated SUCRA or P-score values across multiple outcomes. The results of these methods were compared in real-world meta-analyses, as well as in simulated data sets, suggesting an overall high degree of agreement. It should be stated that higher similarity coefficients were obtained with a higher number of interventions and a lower number of outcomes. As a result, more than 1 method may be applied by reviewers of network meta-analyses with a low interventions-to-outcomes ratio to test the stability of treatment rankings.

A new graphical tool was proposed, especially in cases in which 2 outcomes were assessed. Specifically, a scatterplot of SUCRA values was constructed, coloring points according to their TOPSIS $S$ score, providing direct information about the best compromise intervention. Similar plots can be created using the VIKOR $Q$ or the PROMETHEE II net outranking flow values. A sample script is provided in the Appendix for replication of the plot in R. Previous research has suggested the visualization of multiple outcomes with the use of heatmaps,^24–26 while the Kilim plot²⁷ has been recently proposed, adding information about statistical significance; however, these approaches remain descriptive without indicating a compromise intervention.

The implementation of PROMETHEE I in network meta-analysis may serve as a useful decision-making tool, as partial ordering is able to reveal complex relationships among interventions. This approach may overcome the limitations of absolute rankings, allowing the identification of interventions that are indifferent or even incomparable to others.¹⁶ An analogous method of partial ranking has been suggested for the resolution of conflicting rankings, proposing the construction of Hasse diagrams.⁷ The present methodology extends this approach, suggesting the visualization of partial ordering in a league table, which is less complicated, especially when the number of the compared interventions is large.

The results of decision analysis may be highly dependent on the applied outcome weights. Apart from equal-weight analysis, the objective entropy method was proposed, as it provides a direct way of assigning superior weights to outcomes containing more information than others.²⁸ Alternatively, the subjective analytical hierarchical process allows the decision maker to place greater importance on certain outcomes depending on the clinical scenario and preference. The process is flexible and computationally simple and allows the use of verbal judgments as well as the evaluation of consistency.²⁹ In this context, network meta-analyses may include multiple sensitivity analyses assigning superior weights to different outcomes, enabling the individualized choice of the most appropriate treatment for specific patient populations.

The incorporation of decision analysis methods in network meta-analysis may be complicated by the inherent limitations of the metrics used for treatment ranking. Specifically, the SUCRA and P-score values include no information about statistical uncertainty; hence, they should be interpreted in the context of the confidence intervals of the effect measure. The quality of evidence is also not taken into account, and the possibility of bias should be considered.³⁰ The generalizability of the comparison of the 3 methods may be limited by the use of the uniform distribution for the generation of SUCRA values in the simulation study; therefore, their degree of agreement remains to be confirmed in real-world network meta-analyses. It is also important to note that the possibility of choosing from various decision analysis methods with different normalization algorithms and outcome weights may increase the degrees of freedom during the analysis, increasing the risk of selective reporting; hence, the exact plan for implementation of specific multicriteria algorithms should be prespecified in the protocol of network meta-analyses.

In conclusion, the TOPSIS, VIKOR, and PROMETHEE multicriteria decision analysis methods can be incorporated in network meta-analysis, contributing to optimal treatment selection when multiple conflicting outcomes are simultaneously evaluated. The weights of different outcomes may be defined objectively or subjectively to reflect the priorities of the decision maker. Future research should be directed toward testing alternative weighting methods, providing empirical data about the utility of multicriteria methods in network meta-analysis, and determining the most appropriate algorithm depending on the clinical question.

Supplemental Material

sj-docx-1-mdm-10.1177_0272989X221126678 – Supplemental material for Multicriteria Decision-Making Methods for Optimal Treatment Selection in Network Meta-Analysis

Supplemental material, sj-docx-1-mdm-10.1177_0272989X221126678 for Multicriteria Decision-Making Methods for Optimal Treatment Selection in Network Meta-Analysis by Ioannis Bellos in Medical Decision Making

Footnotes

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. The author received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Ioannis Bellos

Data Availability Statement

The codes used in the analysis are provided as supplementary material.

References

Watt

Tricco

Straus

, et al. Research techniques made simple: network meta-analysis. J Invest Dermatol. 2019;139:4-12.e1.

Mbuagbaw

Rochwerg

Jaeschke

, et al. Approaches to interpreting and choosing the best treatments in network meta-analyses. Syst Rev. 2017;6:1–5.

Salanti

Ades

Ioannidis

JPA

. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol. 2011;64:163–71.

Rücker

Schwarzer

Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med Res Methodol. 2015;15:1–9.

Thokala

Devlin

Marsh

, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. 2016;19:1–13.

Chaimani

Higgins

JPT

Mavridis

, et al. Graphical tools for network meta-analysis in STATA. PLoS One. 2013;8:e76654.

Rücker

Schwarzer

Resolve conflicting rankings of outcomes in network meta-analysis: partial ordering of treatments. Res Synth Methods. 2017;8:526–36.

López-López

Sterne

JAC

Thom

HHZ

, et al. Oral anticoagulants for prevention of stroke in atrial fibrillation: systematic review, network meta-analysis, and cost effectiveness analysis. BMJ. 2017;359:j5058359.

Marques

EMR

Dennis

Beswick

, et al. Choice between implants in knee replacement: protocol for a Bayesian network meta-analysis, analysis of joint registries and economic decision model to determine the effectiveness and cost-effectiveness of knee implants for NHS patients-The KNee Implant Prostheses Study (KNIPS). BMJ Open. 2021;11(1):e040205.

10.

Khoo

Zhou

Teng

, et al. Network meta-analysis and cost-effectiveness analysis of new generation antidepressants. CNS Drugs. 2015;29:695–712.

11.

Chaimani

Porcher

Sbidian

, et al. A Markov chain approach for ranking treatments in network meta-analysis. Stat Med. 2021;40:451–64.

12.

Hwang

C-L

Yoon

Multiple attribute decision making methods and applications a state-of-the-art survey. Lect Notes Econ Math Syst. 1981;186:58–191.

13.

Mosalman Yazdi

. topsis: TOPSIS method for multiple-criteria decision making (MCDM). R package version 1.0. 2013.

14.

Opricovic

Tzeng

GH.

Compromise solution by MCDM methods: a comparative analysis of VIKOR and TOPSIS. Eur J Oper Res. 2004;156:445–55.

15.

Blanca

MCDM: Multi-Criteria Decision Making Methods for Crisp Data. R package version 1.2. 2016.

16.

Brans

Vincke

A preference ranking organisation method. Manage Sci. 1985;31:647–56.

17.

Ishizaka

Resce

Mareschal

Visual management of performance with PROMETHEE productivity analysis. Soft Comput. 2017;22:7325–38.

18.

Wang

Liu

, et al. Application of the entropy weight and TOPSIS method in safety evaluation of coal mines. Procedia Eng. 2011;26:2085–91.

19.

Saaty

RW.

The analytic hierarchy process—what it is and how it is used. Math Model. 1987;9:161–76.

20.

Alonso

Lamata

Consistency in the analytic hierarchy process: a new approach. Int J Uncertain Fuzziness Knowledge-Based Syst. 2006;14:445–59.

21.

Sałabun

Urbaniak

A new coefficient of rankings similarity in decision-making problems. Comput Sci. 2020;12138:632.

22.

Cipriani

Furukawa

Salanti

, et al. Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. Lancet. 2018;391:1357–66.

23.

Bahji

Ermacora

Stephenson

, et al. Comparative efficacy and tolerability of pharmacological treatments for the treatment of acute bipolar depression: a systematic review and network meta-analysis. J Affect Disord. 2020;269:154–84.

24.

Bellos

Pergialiotis

Papapanagiotou

, et al. Comparative efficacy and safety of oral antihypertensive agents in pregnant women with chronic hypertension: a network metaanalysis. Am J Obstet Gynecol. 2020;223:525–37.

25.

Veroniki

Straus

Fyraridis

, et al. The rank-heat plot is a novel way to present the results from a network meta-analysis including multiple outcomes. J Clin Epidemiol. 2016;76:193–9.

26.

Pillinger

McCutcheon

Vano

, et al. Comparative effects of 18 antipsychotics on metabolic function in patients with schizophrenia, predictors of metabolic dysregulation, and association with psychopathology: a systematic review and network meta-analysis. Lancet Psychiatr. 2020;7:64.

27.

Seo

Furukawa

Veroniki

, et al. The Kilim plot: a tool for visualizing network meta-analysis results for multiple outcomes. Res Synth Methods. 2021;12:86–95.

28.

Kumar

Singh

Singh Bilga

, et al. Revealing the benefits of entropy weights method for multi-objective optimization in machining operations: a critical review. J Mater Res Techol. 2021;10:1471–92. DOI: 10.1016/j.jmrt.2020.12.114

29.

Emrouznejad

Marra

The state of the art development of AHP (1979–2017): a literature review with a social network analysis. Int J Prod Res. 2017;55:6653–75.

30.

Mbuagbaw

Rochwerg

Jaeschke

, et al. Approaches to interpreting and choosing the best treatments in network meta-analyses. Syst Rev. 2017;6:1–5.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB