Developing a new multidimensional model for selecting strategic plans in balanced scorecard

Abstract

The main motivation of this research is to develop an innovative multidimensional model through multi attribute decision making (MADM) methods for strategic plans selection process in the Balanced Scorecard (BSC). The current study adopted MADM analytical methods including AHP, ELECTRE, BORDA, TOPSIS and SAW to rank the initiatives / strategic plans in BSC. Then the results of those methods were compared against each other in order to find a robust model for selecting strategic plans. The correlation coefficient between methods indicated that multidimensional and ELECTRE methods with 0.944 are the best performing and AHP with negative correlation (–0.455) is the worst performing method for selecting strategic plans in BSC. The high correlation demonstrates that the model can be a useful and effective tool to finding the critical aspects of evaluation criteria as well as the gaps to improve company performance for achieving desired level. Developing multidimensional model is the core model for the selection of strategic plans. This study addresses the problem and issues of group decision making process for selecting strategic plans in BSC. It has numerous contributions that particularly includes; 1) Determination of the explicit criteria sub-criteria and criteria to improve ranking strategic plans in BSC, 2) Adopting MADM analytical methods including AHP, ELECTRE, BORDA, TOPSIS and SAW for the selection of strategic plans decision problem in BSC, 3) Developing multidimensional model to address the selection of strategic plans problems in BSC. The proposed model will provide an approach to facilitate strategic plans decision problem in BSC.

Keywords

Balanced scorecard model MADM group decision making strategic management

1 Introduction

Balancing operational and long term strategic objectives is always difficult for companies. In fact, without appropriate strategic direction even brilliant processes will not lead to success. In addition, in the world even excellent strategy is useless without strong execution plan [1]. Strategic planning is fundamental for organizational success. It helps an organization to achieve its competitive advantage in the complex business environment.

Senior executives and directors focus on the current complex situation of a company and display big picture through clear understating of future image. They require more important information of the company rather than just focusing on financial indicators. The review of the strategic planning and performance can create the broader view for the company to accomplish its organizational goals and strategic objectives [2].

Balanced scorecard is an appropriate instrument for transforming strategy into action. It was initiated by Kaplan and Norton [3]. The BSC includes four dimensional perspectives that include customers’ perspectives, internal processes, financial processes, and learning and development perspectives [4, 5]. Every perspective has its own indicators for evaluating organizational performance to achieve its objectives. Some other attributes are used for the evaluation of company development in accessing long-term objectives. To determine the leading and lagging attributes the BSC helps the managers in their companies [6].

In addition, the balanced scorecard has speedily became the dominant character in business management research studies including such as information systems, operations management and organization studies, etc. It is the most trending tool among companies because of its simplicity. It is an amazing tool for those directors and managers who have to work through keep track of a few key indicators instead of heaps of statistics [7 –9].

The critical literature review has revealed that some mathematical model is developed for improving and prioritizing strategic plans in the BSC based on MADM methods [10 –12]. However, it is shown several problems which makes it unable to deliberate the empirical investigation and expert evaluation in the organizational performance. Furthermore, there is a lack of ability to prioritize strategic plans for achieving organizational short and long term objectives and translating strategy into action [13, 14]. In addition, lack of adopting MADM analytical methods comprising AHP, ELECTRE, BORDA, TOPSIS and SAW to rank the strategic plans in BSC is existed. The next challenge would be comparing the results of those methods against each other, in order to find a robust model for choosing strategic plans.

There are two primary aims for this study: 1. To investigate and adopt MADM analytical methods comprising AHP, ELECTRE, BORDA, TOPSIS and SAW to rank the strategic plans in BSC. 2. To compare the results of MADM methods against each other in order to find a robust model for choosing strategic plans. In general, adopting MADM analytical methods can present better results for choosing strategic plans in BSC [15, 16]. Since the relative importance of group decision could reduce different individual opinion preferences on a single collective preference, adopting the MADM methods will solve this problem with multiple criteria.

The remaining part of this paper proceeds in different sections. The second section has described the literature of BSC and MADM methods. In the section three and four proposed methods and case study are discussed respectively. Finally, in the section five, discussion, conclusion and future direction in this theme are provided.

2 Literature review

BSC is an innovative framework for translating strategy into action (designing operative strategies); The BSC framework provides a journey to an organization to obtain the vast perspective for taking correct strategic decisions that has an impact on financial processes, customers’ preferences, internal processes and practical learning for employees. This framework measures the financial and non-financial events, external objectives and goals, internal improvements and competencies, short-term as well as long-term sustainable goals, past events, outcomes, and ongoing actions as indications of future growth and progress [2 , 17].

The four aspects of BSC are portrayed briefly as follows:

Financial: This aspect generally comprises the conventional financial key performance measures, which are typically related to profitability. This perspective measurement criteria are normally cash flow, profit, economic value added (EVA), return on invested capital (ROIC), and ROI.

Customer: This perspective is the origin of business profits; thus increasing customer satisfaction is the goal retained by companies. In this perspective, management recognizes the anticipated market segments and target customers for operational departments and screens the performance of operational units in these target segments. Some examples of the core or genetic measures are customer satisfaction, new customer acquisition, customer retention, market share in targeted segments and market position.

Internal business process: The aspect enhances internal business process through fulfilling customers and ensures shareholders through transcending at internal environment of business. For setting the objectives and measures, the first step is analyzing corporate value-chain. A conventional operating process ought to be fine-tuned to recognize the customer and financial aspects objectives. Then a comprehensive internal business-process value chain would be constructed that can meet current and future needs. A usual enterprise internal value chain comprises of three essential business processes: operation, innovation and after-sale services.

Learning and growth: This aspect construct an applicable design that provide support to other perspectives and create sustainable development in an organization and constant improvements through people, organizational procedures and systems. This aspect highlights employee performance measurement, such as training and skills, employee satisfaction, continuity, since employee growth is an intangible asset to enterprises that will contribute to business growth. In the other three aspects, there is mostly a gap between the actual and target system, procedure capabilities and human resources. Through learning and growth, enterprises can alleviate this gap. The indicators consist expenses on training, turnover rate of workers and lead time for introducing innovation to a market expenditures on new technologies [7, 17].

The framework of the BSC is exhibited in Fig. 1 [3].

Fig. 1

Balanced Scorecard model [18].

2.1 Decision making methods

Usually, there are two appropriate methods for decision making. 1) Modeling method, 2) Trial and error method. As the decision maker (DM) and board of directors faces practical problems for informed decision making. So, they adopt practical application of trial and error method for selecting the best alternative and conduct trail to observe the particular results. If the chances of decision error are high and cause problems for organization then decision makers chose another modified alternative. However, in modeling method, the decision makers construct model for practical problems and resolve factors that are affecting organization [19].

Subsequently there are two type of criteria for decision making, multi-attributes and other is multi-objectives. Multi-criteria decision making (MCDM) problems are further categories into two groups:

Multi-objective decision making (MODM)

Multi-attribute decision making (MADM)

There are major differences between MODM and MADM. MODM is focusing on continuous decision problems, essentially with numerous objective functions in mathematical programming. On the other hand, MADM concentrates on problems with various alternatives in discrete decision problems [20].

2.2 Multi attribute decision making (MADM)

MADM is considered as prominent area of operational research, preparing a useful evaluation framework for multiple conflicting criteria. MADM was recognized as the most popular part of MCDM [21]. The viewpoints of DMs and their opinions are regularly engaged in the process of decision making. George Miller (Psychologist) also recognized that an individual’s has strong visualization to deal with concurrent or similar information [22, 23]. MADM is widely used decision methodologies in different areas such as business, sciences, government and engineering worlds that could help to improve quality of decision making process and make it more efficient, explicit and rational.

There are several MADM applications in business and engineering,, that utilize more manufacturing systems flexibilities [24], design of layout [25], integrated manufacturing systems [26], and the evaluation for technological intervention, investment and other financial decisions [27]. The succinct of several MADM methods is described as follows.

2.2.1 Analytical hierarchy process (AHP)

Saaty [28] introduced the AHP, one of widely used MADM method. The AHP assesses multi-criteria tangible and intangible attributes in a systematic manner and it structures a decision making issue into a different hierarchy levels of criteria to resolves decision-making problems [8, 29].

2.2.2 Elimination Et choice translating reality (ELECTRE)

ELECTRE was introduced by Roy [30] as a technique for resolving deficiencies in existing decision making and offers accurate solution methods. It is a perspective for decision aid. Further philosophy is explained by Roy in a lengthy conservation. The focus is on the method itself which is referred to in particular as ELECTRE. It has develop through the different versions of numerical style (I, II, II, IV, V, IS, A). All the versions have same primary concepts but their applications and operational are different from one another [31].

2.2.3 Technique for order-preference by similarity to ideal solution (TOPSIS)

Yoon & Hwang [32] design TOPSIS as an alternative method of the ELECTRE solution method. Actually, the gist of this method refers to chosen alternative having shorter distance from the ideal solution and has longest distance from the negative solution in a geometrical aspect.

2.2.4 Simple additive weighting (SAW)

This method is a simple one and known as scoring method which is highly applied multi-attribute decision making technique. This method is considered as a weighted average scoring method to calculate alternatives through multiplying the scaled value for each alternative with its criteria weight. The weights are allocated by DMs that is represent importance of each criteria [33].

2.2.5 Borda

Since several alternatives are remarked, the Borda method is a suitable procedure in multi-person decision making. This method supports DM to categorize alternatives as per attributes and then according to the complex structure of the matrix, the group conformity, design as “n” class that can be obtain for “m” alternative that is further solve by zero-one programming model. Eventually all alternatives will be ranked [34, 35].

In previous studies, some mathematical programming and MADM methods such as AHP methods have been applied for group decision making problems [6 , 36–39]. However, AHP has two major problems. In BSC, due to complex nature in the process of decision-making, AHP cannot deal with criteria interrelationship in the model. The other issue is subjectivity of AHP which can be a weakness of the method. Dodangeh et al. [20] introduced an integrated goal programming (GP) with TOPSIS for group decision making problems in BSC for choosing strategic plans.

However, in the GP there is a technical problem which DMs need to determine preemptive priority order of the goals. Indeed, priori determined goals input is unable to produce an applicable solution. In this sense, DMs may be satisfied eventually through generating iterative solution. Therefore, it would be costly and inefficient.

The majority of researches conducted on BSC, adopted two or three methods of MADM for ranking strategic plans methods instead of using five MADM methods for modeling and then compare the analysis of statistical results [12 , 40]. Furthermore, the previous studies are subjective, and DM cannot allocate numbers to preferences. To the best of our knowledge there is no study to investigate and adopt MADM analytical methods including AHP, ELECTRE, BORDA, TOPSIS and SAW and to rank and analyze the strategic plans as well as compare the results of the MADM methods against each other in order to find a robust model for choosing strategic plan in BSC.

3 Materials and methods

MADM is defined by Hwong & Yoon [32] as follows. It is a decision making process to prioritize preferable decisions (such as assessing, prioritization and choices) between different alternatives by usually opposite and multiple attribute. In the MADM, group decision panel encounter the common steps that need to be addressed particularly are:

1) Multiple attributes, 2) Alternatives, 3) Attributes weight 4) Dimensionless units, 5) Quality of attributes and 6) Opinions of decision makers.

There are different MADM methods which are categorized as in the following groups:

Compensatory methods: for instance, if a product has good quality but high cost, indeed high cost will be compensated through high quality. In this category there are different models such as MDS, ELECTRE, MRS, TOPSIS, linear assignment, SAW, etc.

Non compensatory methods: in this category, different attributes do not compensate each other. For example, if you want to get a driving license, you need to pass tree non compensatory attributes including normal eye test, driver knowledge test and driving test that you will need to pass all three attributes and they do not compensate each other’s [41]. These models are consisting ELIMINATION, DOMINANCE, LEXICOGRAPH, PERMUTATION, etc.

3.1 Expert panel and strategic plans weights

There are several objectives for MADM modelling problem that can be identified via decision makers. All MADM methods require information to determine the objectives relative importance. Weights of objective can be assigned to an objective directly through scientific methods or by a decision maker group. These weights determine each objective relative importance.

Usually expert groups are categorized based on their different organizational levels, such as knowledge and work experience, etc. So, expert group idea’s weight could be different in special subject. In this sense, every opinions has its own weight assigning depending on their experience and knowledge and regarding to that. In this study we use hierarchical objectives for determining strategic plans weights in BSC, as it can be illustrated in Fig. 2.

Fig. 2

Objectives hierarchy in BSC.

For this to happen, the study determines the weights of aspects and sub aspects in BSC applying opinion of expert. The geometric average method utilized for calculating final weights of sub aspects (financial, customer, internal processes and learning and growth). The calculation method is adopted for BSC can be seen as follows.

${TW}_{Cij} = \sqrt{W_{Ci} . W_{Cij}}$ (1)

TW_Cij: Final objective weights

W_Ci: Weights of aspect

W_Cij: Objective weights

TW_Cij: objective final weights are equal the weight of strategic plan [42].

3.2 A New multidimensional model

There are different steps to model the MADM analytical methods including AHP, ELECTRE, BORDA, TOPSIS and SAW to rank and analyze the strategic plans in BSC, which is described as follows. The modelling process is systematically illustrated in Fig. 3.

Fig. 3

Modelling Process.

Step 1: Separate methods in two groups; the methods which rank Strategic plans in different priorities/grades (each alternative is ranked in different priorities/grades) and the methods that place some Strategic plans in the same priorities/grades. In the first group, list the ranking alternatives in own columns and in the second group, write all kind of ranking which is decomposed to them [43 –45].

Step 2: Multidimensional matrix has ranking that includes rows “m” as strategic plans and considered “n” as a columns in the MADM methods. This matrix is consist of ordinal numbers (rating of strategic plans) based on each method in each column. After decomposing ELECTRE method into 48 modes, in this study m and n are 12 and 52 respectively.

Step 3: Consider the biggest number (rate/rank) inside the matrix, so subtract each cell in this matrix from the biggest number. Then biggest numbers will be zero and remaining will be further modified.

Step 4: Add another new column at the end of matrix and compute the sum of each row and put them in the corresponding place. So, the biggest number of this column is represented the best strategic plans and the others will be followed by it.

3.3 Correlation coefficient

Correlation is primary method to determine existing relationship among variables exists and concerned with finding with its magnitude and particular direction. It is one of the most common useful statistics. It uses statistics and illustrates the degree of relationship among two variables. The symbol of correlation is “r”. $r = \frac{n \sum xy - (\sum x) (\sum y)}{\sqrt{[n \sum x^{2} - {(\sum x)}^{2}] [n \sum y^{2} - {(\sum y)}^{2}]}}$ (2)

Where:

n: Number of pairs of scores

∑xy : Sum of the products of paired scores

∑x : Sum of x scores

∑y : Sum of y scores

∑x² : Sum of squared x scores

∑y² : Sum of squared y scores

The measurement of r is always between – 1.0 and +1.0. However, the negative correlation means there is negative relationship among variables and positive means the positive relationship among variables. The data is ranked as ordinal numbers for finding correlation. Spearman’s rho correlation coefficient is used [46].

4 Results

In this study, the BSC four perspectives are remarked as the skeleton for developing the strategic plans through breaking down the balanced scorecard model. A case study was conducted in a manufacturing company. An expert panel was formed to determine strategic plans and their weights including managing director, chief financial officer, chief sales officer, chief financial officer, chief operating and supply chain officer, were applied. Then the expert panel involved for developing the BSC model through adopting the MADM methods and a consensus reached by them. The BSC four perspectives model is illustrated in Table 1.

Table 1
Balanced Scorecard model for the case study

Financial

Objectives Measures Target Initiatives

Increasing revenue 0.79 0.91 I1-Marketing Research

Increasing profit 0.82 0.92 I2- Marketing

Cost reduction 0.10 0.05 I3- ABC

Customer

Increasing customer satisfaction 0.81 0.95 I4-After sales Services

Increasing Market share 0.60 0.75 I5-Marketing Research

Increasing added value for customers 0.75 0.90 I6-Value Engineering

Internal Processes

Objectives Measures Target Initiatives

On time delivery 0.64 0.85 I7-Time &Motion Study

Product development 0.64 0.72 I8-QFD

Continuous improvement 0.74 0.86 I9-TQM

Learning &Growth

Increasing employees satisfaction 0.66 0.82 I10-Increasing personnel salary

Increasing employees productivity 0.54 0.72 I11-Personnel evaluation and reward system

Increasing informational skills 0.71 0.86 I12-MIS

Financial
Objectives	Measures	Target	Initiatives
Increasing revenue	0.79	0.91	I1-Marketing Research
Increasing profit	0.82	0.92	I2- Marketing
Cost reduction	0.10	0.05	I3- ABC
Customer
Increasing customer satisfaction	0.81	0.95	I4-After sales Services
Increasing Market share	0.60	0.75	I5-Marketing Research
Increasing added value for customers	0.75	0.90	I6-Value Engineering
Internal Processes
Objectives	Measures	Target	Initiatives
On time delivery	0.64	0.85	I7-Time &Motion Study
Product development	0.64	0.72	I8-QFD
Continuous improvement	0.74	0.86	I9-TQM
Learning &Growth
Increasing employees satisfaction	0.66	0.82	I10-Increasing personnel salary
Increasing employees productivity	0.54	0.72	I11-Personnel evaluation and reward system
Increasing informational skills	0.71	0.86	I12-MIS

For finding the multidimensional rank, we extracted the results of five papers in which data (criteria and alternatives) were similar. After that, by using their rank of alternatives, multidimensional matrix was made, and the precise rank was determined.

After separating methods into two groups, the first group was formed based on AHP, BORDA, TOPSIS and SAW. In other words, these methods have ranked strategic plans in different priorities/grades (each alternative is ranked in different grades) as calculated in Table 2.

Table 2

Strategic plans ranking in MADM methods

Rank	AHP method	BORDA method	TOPSIS method	SAW method
1	A1	A1	A1	A1
2	A9	A4	A2	A7
3	A2	A5	A6	A3
4	A7	A3	A3	A4
5	A3	A7	A12	A6
6	A11	A11	A9	A9
7	A8	A2	A7	A12
8	A4	A12	A4	A10
9	A5	A9	A10	A11
10	A12	A10	A11	A8
11	A6	A8	A8	A2
12	A10	A6	A5	A5

As described in the step one of a new multidimensional model, the second group was ranked based on ELECTRE method, as shown in Table 3.

Table 3

ELECTRE method ranking

Rank	ELECTRE method
1	A1
1	A3
2	A12
3	A6
3	A7
3	A9
4	A4
5	A10
5	A11
6	A2
6	A8
7	A5

Alternatives (strategic plans/initiatives) 1, 3 and 6, 7, 8 and 10, 11 and 2, 8 were each assigned the same rank/grade, while the others were assigned different grades. In step two of the methodology, the multidimensional ranking matrix had 12 rows as strategic plans, and 52 columns for the MADM methods.

The rank of ELECTRE method was expanded into 48 different modes; so the main matrix had 12 rows as alternatives and (48 + 4) columns as ranked alternatives. Following step three and four of methodology: rankings were calculated based on the multidimensional model. Eventually, the multidimensional method for ranking strategic plans is shown in Table 4.

Table 4

Correlations of methods

Correlation Coefficient	Multidimensional	AHP	Borda	TOPSIS	SAW	ELECTRE
Multidimensional	1	– 0.46	0.42	0.266	0.08	0.944^**
AHP	– 0.455	1	0.28	0.224	0.4	– 0.308
Bord	0.42	0.28	1	0.559	0.41	0.51
TOPSIS	0.266	0.224	0.559	1	0.38	0.308
SAW	0.084	0.399	0.413	0.378	1	0.168
ELECTRE	0.944^**	– 0.31	0.51	0.308	0.17	1

Finally, the correlation coefficient between methods (Multidimensional, AHP, Borda, TOPSIS, SAW and ELECTRE) was tested, and the correlation analysis results are presented in Table 5. As you can see, interestingly there was a significant positive correlation between the multidimensional method and ELECTRE method, with 0.944, being the best methods. On the other hand, the AHP method has a negative correlation coefficient of –0.455, is the worst method.

Table 5

Final rank

Result
Rank	Multidimensional method
1	A1
2	A3
3	A12
4	A7
5	A9
6	A6
7	A4
8	A11
9	A10
10	A2
11	A8
12	A5

5 Discussion, conclusion and future works

The Balanced Scorecard is a model for translating strategy into action by assessing various business perspectives performance measurement. Several pieces of research and studies have developed models for evaluating operational strategies in BSC [13–15 , 47]. However, selecting the best strategic plans is a complex decision making process, which requires various analytical methods to overcome this crucial problem. Companies cannot execute all the initiatives and strategic plans with time constraints and limited budget and resources. So, they need to have a multidimensional model for selecting high priority strategic plans [48]. The proposed model solves these issues by developing a multidimensional decision making methodology that combines group decision making methods, comprising AHP, ELECTRE, BORDA, TOPSIS, and SAW. This study has clearly revealed that strategic plans decision can be improved in several ways by adopting multidimensional model, which was presented via the case study. The model can solve the challenge of comparing the results of those methods against each other in order to find a robust model for selecting strategic plans in BSC. The correlation coefficient between methods has indicated that multidimensional and ELECTRE with 0.944 are the best methods and AHP with a negative correlation of –0.455 is the worst method. In this sense, the multidimensional model seems logical regarding the strategic plans implementation decision in BSC. For instance, marketing research and ABC analysis have the highest execution priorities, which lead to increased revenue and cost reduction strategy. Applying the multidimensional model for group strategic plans decision issue in this study has several particular benefits and contributions comprising: 1) Solving the strategic plans selection problem through determining clear and explicit criteria and sub-criteria in BSC; 2) Adopting MADM analytical methods including AHP, ELECTRE, BORDA, TOPSIS and SAW for the strategic plans decision problem in BSC; 3) Developing multidimensional model for choosing strategic plans in BSC. Furthermore, the evidence from this study shows that there are numerous implications for practice and managers. Insightful information can also be extracted from the various MADM analytical methods (Table 5) to identify the solid methods for choosing imitative and strategic plan in BSC. Strategic decision makers and managers are able to evaluate strategic plans through multidimensional and ELECTRE methods. They are confident which MADM methods are solid and best for choosing strategic plans in BSC. Also, they know what methods to avoid for choosing intuitive and strategic plans according to our comparison of the MADM analytical methods.

In conclusion, the case study has demonstrated the application of the multidimensional model. The analysis results of the multidimensional model have been verified by experts and highlighted as acceptable. It has shown that the model can be a useful and effective tool for finding the critical aspects of evaluation criteria as well as the gaps to improve company’s performance for achieving desired level.

To boost this area of study and to assist further combine strategic planning discussion into the decision making modeling area, future research may adopt numerous other methods such as OWA operators [49 –51]. Although, some methods have been developed with a variety of formal modeling techniques, they may be limited due to different reasons. MCDM methods and decision support tools and methodologies can assist organizations and managers make more effective decisions.

References

Kaplan

and Norton

, MASTERING the Management System, Harvard Business Review 86(1) (2008), 62–77.

Goodspeed

S.W.

, Tanslating strategy into action: The balanced scorecard. (2003), Medical University of South Carolina - College of Health Professions: United States – South Carolina.

Kaplan

and Norton

, The Balanced Scorecard - Measures that drive performance, Harvard Business Review 70(1), 71–79.

Najmi

, Zarei

and Hosaini

, BSC & EFQM Interaction or Conflict. in first conference of performance management. (2001), Tehran, Iran: university of Tehran.

Kaplan

and Norton

, The balanced scorecard: translating strategy into action, 1996: Harvard Business Press.

Guo

and Yu

, The comprehensive evaluation of railway freight enterprises’ performance based on the balanced scorecard card and AHP. (2011).

Lee

A.H.I.

, Chen

W.-C.

and Chang

C.-J.

, A fuzzy AHP and BSC approach for evaluating performance of IT department in the manufacturing industry in Taiwan, Expert Systems with Applications 34(1) (2008), 96–107.

Leung

, Lam

and Cao

, Implementing the balanced scorecard using the analytic hierarchy process & the analytic network process, Journal of the Operational Research Society 57(6) (2005), 682–691.

Akkermans

and Oorschot

, Relevance assumed: a case study of balanced scorecard development using system dynamics, Journal of the Operational Research Society 56(8) (2004), 931–941.

10.

Daniel

and Talaei-Khoei

, Can fuzzy Multi-Criteria Decision Making improve Strategic planning by Balanced Scorecard? in Pacific Asia Conference on Information Systems, PACIS 2016 - Proceedings. (2016), 215.

11.

Umayal Karpagam

P.L.

and Suganthi

, Performance measurement of organisations: A review of balanced scorecard technique, International Journal of Business Performance Management 14(2) (2013), 129–148.

12.

Bentes

A.V.

, et al., Multidimensional assessment of organizational performance: Integrating BSC and AHP. Journal of Business Research (2011).

13.

Zhao

and Li

, Evaluating the performance of thermal power enterprises using sustainability balanced scorecard, fuzzy Delphic and hybrid multi-criteria decision making approaches for sustainability, Journal of Cleaner Production (2015).

14.

Zhang

and Wu

, The application research of multi-dimension decision-making of bank’s microfinance performance evaluation using rough set and grey fuzzy approach, International Journal of Applied Decision Sciences 8(4) (2015), 323–338.

15.

Jiang

and Liu.

, A multi-criteria group decision making model for performance evaluation of commercial banks, in Proceedings - 2013 10th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2013. (2013).

16.

Gwo-Hshiung

, Tzeng

G.-H.

and Huang

J.-J.

, Multiple attribute decision making: Methods and applications. 2011: CRC Press.

17.

Lee

and Seo

K.K.

, A Hybrid Multi-Criteria Decision-Making Model for a Cloud Service Selection Problem Using BSC, Fuzzy Delphi Method and Fuzzy AHP, Wireless Personal Communications 86(1) (2016), 57–75.

18.

Kaplan

and Norton

, The Balanced Scorecard, Translating Strategy into Action. (1990).

19.

Ghodsypour

, Multiple Objective Decision Making. (2003), Tehran, Iran: Amir Kabir University Publication.

20.

Dodangeh

, Yusuff

R.B.M.

and Jassbi

, Using Topsis Method with Goal Programming for Best selection of Strategic Plans in BSC Model, Journal of American Science 6(3) (2010), 136–142.

21.

Triantaphyllou

, Multi-criteria decision making methods: a comparative study. 2000: Springer Netherlands.

22.

Doyle

and Green

, Efficiency and cross-efficiency in DEA: derivations, meanings and uses, Journal of the Operational Research Society 45(5) (1994), 567–578.

23.

Saaty

, Validation examples for the Analytic Hierarchy Process and the Analytic Network Process. (2004).

24.

Wabalickis

, Justification of FMS with the analytic hierarchy process, Journal of Manufacturing Systems 7(3) (1988), 175–182.

25.

Cambron

and Evans

, Layout design using the analytic hierarchy process, Computers & Industrial Engineering 20(2) (1991), 211–229.

26.

Putrus

, Accounting for intangibles in CIM justification, CIM Review 6(2) (1990), 23–29.

27.

Boucher

and MacStravic

, Multiattribute evaluation within a present value framework and its relation to the analytic hierarchy process, The Engineering Economist 37(1) (1991), 1–32.

28.

Saaty

T.L.

, The analytic hierarchy process (1980), McGraw-ill, New York.

29.

Ishizaka

, Balkenborg

and Kaplan

, Does AHP help us make a choice" An experimental evaluation, Journal of the Operational Research Society (2010).

30.

Roy

, The outranking approach and the foundations of ELECTRE methods, Theory and Decision 31(1) (1991), 49–73.

31.

Huang

and Chen

, Using the ELECTRE II method to apply and analyze the differentiation theory, Proc Eas Asia Soc Trans Stud 5 (2005), 2237–2249.

32.

Yoon

and Hwang

C.L.

, Multiple Attribute Decision Making : An Introduction, Quantitative Applications in the Social Sciences, ed. U.o.I. Michael S. Lewis-Beck. 1995 Sage Publications, Inc. 83.

33.

Asgharpour

M.J.

, Multiple Criteria Decision Making. 6 ed. (2008), Tehran: university of Tehran Press.

34.

García-Lapresta

J.L.

, Martínez-Panero

and Meneses

L.C.

, Defining the Borda count in a linguistic decision making context, Information Sciences 179(14) (2009), 2309–2316.

35.

Anisseh

, et al., 360 Degree personnel performance appraisal using the MADM models and presenting a model for overall ranking. in IEEM 2007: 2007 IEEE International Conference on Industrial Engineering and Engineering Management. (2007).

36.

Al-Hedaithy

S.A.

, An analytic approach to developing strategic performance measures in small organizations: A comparative study of the development process in a small government agency versus a small private company. (2000), The George Washington University: United States – District of Columbia.

37.

Huang

H.-C.

, Designing a knowledge-based system for strategic planning: A balanced scorecard perspective, Expert Systems with Applications 36(1) (2009), 209–218.

38.

Yuan

F.-C.

and Chiu

, A hierarchical design of case-based reasoning in the balanced scorecard application, Expert Systems with Applications 36(1) (2009), 333–342.

39.

Figueira

, Greco

and Ehrgott

, Multiple Criteria Decision Analysis: State of the Art Surveys. 2005. Springer New York.

40.

Yüksel

İ.

and Dağdeviren

, Using the fuzzy analytic network process (ANP) for Balanced Scorecard (BSC): A case study for a manufacturing firm, Expert Systems with Applications 37(2) (2010), 1270–1278.

41.

Korhonen

, Moskowitz

and Wallenius

, Multiple criteria decision support - A review, European Journal of Operational Research 63(3) (1992), 361–375.

42.

Dodangeh

, Modeling of Fuzzy Balanced Scorecard, in Department of Industrial Management. (2006), IAU, Science & Research Branch: Tehran 130.

43.

Dodangeh

, Mojahed

and Nasehifar

, Ranking of Strategic Plans in Balanced Scorecard by Using Electre Method, International Journal of Innovation, Management and Technology 1(3) (2010), 269–274.

44.

Dodangeh

, et al., Priority of strategic plans in BSC model by using of Group Decision Making Model. in 2008 IEEE International Conference on Industrial Engineering and Engineering Management, IEEM 2008. (2008).

45.

Dodangeh

, Anisseh

and Mousakhani

, Priority of strategic plans in BSC model by using Borda method. in 2008 International Conference on Wireless Communications, Networking and Mobile Computing, WiCOM 2008. (2008).

46.

Healey

J.F.

and Prus

S.G.

, Statistics : a tool for social research, (2009), Toronto: Nelson Education.

47.

Xian

, Sun

and Xu

, A new fuzzy comprehensive evaluation model with balance supply chain scorecard based on fuzzy PCA, Journal of Computational Information Systems 10(9) (2014), 3913–3920.

48.

Hermawan

A.F.

and Anshari

, Performance measurement of project management by using FANP Balanced Score Card, Journal of Theoretical and Applied Information Technology 83(2) (2016), 262–269.

49.

Merigó

J.M.

and Gil-Lafuente

A.M.

, The induced generalized OWA operator, Information Sciences 179(6) (2009), 729–741.

50.

Merigó

J.M.

, Fuzzy decision making with immediate probabilities, Computers & Industrial Engineering 58(4) (2011), 651–657.

51.

Merigó

J.M.

, Probabilities in the OWA operator, Expert Systems with Applications 39(13) (2012), 11456–11467.