Abstract
Since the emergence of digital revolution, many governments have sought to use the internet for the benefits of communication and information easiness to present public services. Due to citizen’s consciousness of internet, the online public services have increased rapidly. Allowing citizens to entrance governmental services, information and participate in governmental decision making process, called the electronic government. In recent years, various e-government websites have increased. Beside the basic lines for e-government websites design process, the evaluation approach also very important for increasing sites quality. The success of these sites depends largely on their quality, security and accessibility. This research provides a multi-criteria group decision making (MCGDM) technique rely on neutrosophic VIKOR method, for evaluating e-government websites. To represent preferences of decision makers about criteria significance weights and performance assessments, triangular neutrosophic numbers are used for representing linguistic variables. In this research, two algorithms are developed based on a neutrosophic linguistic approach. Depending on two algorithms and the VIKOR method, a general framework is proposed. A case study is presented to validate the proposed framework, and a comparative study between two algorithms is illustrated with detail.
Introduction
The concept of information and communication technologies (ICTs) has applied at an ascending rate, especially in private organizations. In order to enhance the interaction between citizen and governments, the online government services have been presented by governments [1]. We can refer to e-government as a utilization of technologies and internet for providing enhanced services to internal and external citizens or customers. Since the advantages of e-government became obvious and very important, the number of projects concerning e-government has raised widely since 1996 from 3 to more than 500 national actions [2]. There exists only one channel for online offering of services and products to private and public organizations. This channel is e-government website or called portals. Benjamin and Whitley [3] noted that, “website can’t justify itself simply by being only a website, but it should has goals and seeks to achieve them”. It is important to evaluate e-government websites periodically and asking ourselves an important question, “do these sites can provide governmental services in the best form”. According to Wood et al. [4], the key element for evaluating e-government websites is a multi-dimensional strategy. The evaluation strategy is a very important part for developing websites and increasing their quality. Since website evaluation criteria are complex, conflict, vague and subject to alter interpretation, our research proposed a group decision making framework relied on neutrosophic VIKOR for compromising multiple, conflict criteria and dealing efficiently with vague and inconsistent information which exist usually in web evaluation problems. The VIKOR method presented especially for multi-criteria decision making problems with conflicting and non-measurable criteria [5, 6]. VIKOR helps decision makers to rank, select from a set of alternatives and obtain final decision by compromising solution. To deal with vague and inconsistent information, we present VIKOR in neutrosophic environment. A generalization of classical, fuzzy and intuitionistic fuzzy sets, called neutrosophic set [7]. Neutrosophic set deal with vague and inconsistent information, by considering all aspects of decision making process. The main concepts and definitions of neutrosophic set presented in [8] with detail.
This research is organized as follows: in Section 2 a literature review of related researches are presented. In Section 3 the VIKOR approach is presented in neutrosophic environment and the proposed framework is illustrated with detail. In Section 4 an application example for e-government website evaluation is presented and a comparative study also implemented on the same example. Section 5 concludes the paper with future directions of research topic.
Literature review
A survey regarding the literature of e-government website evaluation and the VIKOR approach presented in this section. Some drawbacks of the current approaches are also discussed.
Kumar and Best defined e-government as the utilization of information and communication technologies for enhancing services delivery to private and public organizations [9]. E-government has three sectors as: Government to Government (G2G), Government to Citizen (G2C), Government to Business (G2B). The fourth sector that has been identified by some observers and considered as a subset of G2G [10] is Government to Employee (G2E). Layne and Lee [11] defined e-government as an evolutionary event, providing basic information through the web to public. Different aspects may be used for evaluating e-government. For example, Gupta and Jana [12], aggregated the measurement approaches of e-government performance into hard, soft and hierarchy measure. Padilla [13], observed that website contents differ and depend on different functions and audience. Various criteria were used to evaluate the quality of website. Henriksson et al. [14] evaluate e-government website using several criteria: Security, Privacy, Usability, Content, and Citizen participation. Barnes and Vidgen [15] developed eQual instrument for evaluating e-government websites. A more comprehensive criteria for evaluating websites, proposed by Eschen-felder et al. [16]. These criteria divided into two major categories: Ease of use, Content. The content criteria were divided into sub-criteria: Content, currency, metadata, service, privacy, accuracy, and website orientation. The second criteria also divided into five sub-criteria: Feedback technique, Links, Accessibility, Navigability and Design.
For determining different strategies of Turkey’s e-government, Kahraman et al. [17] combined the crisp and fuzzy AHP with SWOT technique in fuzzy environment. Also Cristina et al. [18] proposed the g-Quality approach, for evaluating quality of Brazilian websites. A model based on fuzzy analytic hierarchy process, proposed by Zhong et al. [19] to evaluate security of e-government system. Another framework based on fuzzy multi-criteria decision making, proposed by Syamsuddin and Hwang for evaluating security of e-government. Tavana et al. [20] proposed a fuzzy framework based on ANP and TOPSIS, for estimating readiness of e-government according to citizen relationship management. Based on UTA II approach, a global evaluation model of e-government proposed by Siskos et al. [21]. For estimating the usability and credibility of e-government websites, Hung and Benyoucef proposed an empirical study [22].
One of the best techniques for disbanding multi-criteria decision making problems with conflicting and incommensurable criteria is VIKOR approach. The VIKOR approach applied in fuzzy environment by many researchers. A fuzzy VIKOR approach with triangular fuzzy numbers for weights and attribute values, proposed by Opricovic [5, 6]. An expanded VIKOR approach for decision making problem, proposed by Sayadi et al. [23]. Ju and Wang introduced an expanding VIKOR approach for multiple criteria group decision making problems [24]. A hesitant fuzzy linguistic VIKOR approach, developed by Liao et al. [25]. A novel expansion of VIKOR approach under fuzzy environment presented by Qin et al. [26]. VIKOR approach utilized in a wide range of problems over the last decade [27]. For deducing the preference order of pit mining equipment, a VIKOR approach, proposed by Bazzazi et al. [28]. An inclusive VIKOR approach for selecting materials, proposed by Jahan et al. [29]. For supplier selection problem a fuzzy VIKOR approach developed by Shemshadi et al. [30]. To improve domestic airline service quality, a developed VIKOR approach utilized by Liou et al. [31]. For solving insurance company problem, Yuenur and Demirel [32] proposed an expanded VIKOR approach in fuzzy environment. A fuzzy VIKOR approach for estimating quality of hospital services in Taiwan, proposed by Chang [33]. To estimate the sensitivity of water supply to climate alteration and variability in South Korea, a VIKOR model presented by Kim and Chung in fuzzy environment [34]. For selecting a suitable site of waste management problem, the VIKOR approach used by Liu et al. [35]. It was noted from previous studies, the fast expansion of the VIKOR approach and its ability to solve different MCDM problem. The VIKOR approach is suitable for e-government website evaluation; because it can compromise solutions to problem with conflicting criteria. Burmaoglu and Kazancoglu used the analytic hierarchy process and VIKOR approach in fuzzy environment for evaluating e-government website [36]. In this research, it is the first time to apply VIKOR approach to e-government website evaluation, under neutrosophic environment.
Gaps of previous VIKOR researches for evaluating website of e-government
The drawbacks of Burmaoglu and Kazancogl model for evaluating e-government website are as follows:
The fuzzy theory Take into consideration only the truth membership degree, and fails to deal with indeterminacy and falsity membership degree. The linguistic values say good, very-good, etc. should be accompanied by a confirmation degree (i.e. the sureness, indeterminacy and falsity degrees for the provided linguistic variables, for example when we compares a car according to speed criterion, and the result was that the car was very-good according to speed criterion, then the decision maker or expert should say that, he/she is 80%sure, 10%indeterminate and 10%not sure that the car is very good with respect to speed criterion), but they didn’t do this in their model. For example, by comparing criteria 1 according to second criteria, it takes very good as a linguistic variable and its confirmation degree according to decision making opinion is absolutely agreed (i.e. the decision maker/expert considers the statement is always truth and there doesn’t exist any indeterminacy or falsity degrees and this absolutely doesn’t exist in reality). They only take the linguistic value only in their model. Also decision making process in real life problem has three forms: agree, not sure, disagree. This shows that their fuzzy model failed to represent reality efficiently.
Our proposed model overcame the previous drawbacks and a comparison analysis between two researches presented in the last section.
New neutrosophic linguistic VIKOR framework for MCGDM
The neutrosophic set concept was first proposed by Smarandache in 1995 and it represent real world problem effectively by considering all aspects of decision making situations, (i.e. truthiness, indeterminacy and falsity) [37]. The linguistic variables [38] used by decision makers to express their opinion and preferences. Instead of numbers, a precipitated words used by Zadeh in [39]. Words or sentences can be used to define linguistic variable values. For example, to represent weight by a linguistic variable it can be represented as: very low, low, high, etc. A special neutrosophic set used commonly in various types of decision making problems is a triangular neutrosophic number
E-government website evaluation is a complex multi-criteria group decision making problem. So the neutrosophic VIKOR approach, as a convincing decision technique can be used to solve this type of problems. In this section, two algorithms rely on neutrosophic linguistic methods are advanced and a general framework also illustrated to address this type of problems. In e-government website evaluation problem, let D
r
be a committee of decision makers where r = 1, 2, …, E, C
j
represent criteria where j = 1, 2, …, n, and A
i
represent alternatives, where i = 1, 2, …, m. The criteria can be classified as cost and benefit criteria so, let us use C
c
, C
b
to represent cost and benefit criteria. Suppose that neutrosophic value which transformed from the linguistic variables, by D
r
decision makers for i
th
alternative and j
th
criterion is
Algorithm 1
The first algorithm of VIKOR approach relies on linguistic variables and in this algorithm we make a deneutrosophication process (for obtaining crisp values) in advanced stage (after obtaining aggregated neutrosophic ratings of alternatives). The steps with more details are as follows:
The aggregated neutrosophic weight of each criterion calculates as follows:
The aggregated neutrosophic ratings of alternatives calculate by:
Where | | means the absolute value of deneutrosophic value. After deneutrosophication process of
For all cost criteria, i.e. j ∈ C
c
;
Where
The Q (A1) has an acceptable benefit:
Where Benf is the benefit of alternative, A2 is the alternative with the second position of ranking process, A1 is the alternative with the first position of ranking process and m is the number of alternatives. The Q (A1) has a stability state i.e. if it’s alsobest ranked in S
i
and/or R
i
.
If second condition not satisfied, then a set of compromising solutions includes A1, A2. If first condition not satisfied, then a set of compromising solutions includes A1, …, A
m
.
The second algorithm of VIKOR approach relies on linguistic variables and in this algorithm we make a deneutrosophication process (for obtaining crisp values) in late stage (after obtaining neutrosophic values of S i , R i , Q i ). The steps with more details are as follows: Step from 1 to 5 as in Algorithm 1.
For the cost criteria

A general framework of proposed methodology.
Until now all values are triangular neutrosophic numbers.
The general framework of neutrosophic VIKOR presented in Fig. 1.

The hierarchy for selecting superior e-government website.
To test the efficiency and effectiveness of proposed frame work for the e-government website evaluation domain, we presented in this section a case study which includes scenario, implementation methods and data analysis.
E-government website evaluation is a complex, vague and time consuming process. At the same time, it’s a very important process for developing, increasing quality of websites and achieving citizen’s satisfaction. Nowadays, the main objectives of all countries are to offer sustainable, accessible and seamless services for their citizens via e-government websites. Our case study includes the evaluation process of various e-government websites, which contains five countries as follows: Singapore, Finland, Canada, Hong Kong, and Australia.
A group of four decision makers was formed, to select the best governmental website and sufficient criteria should be listed by decision makers to do a proper analysis. The evaluating criteria were as follows: Physical effort, Cognitive effort, Tolerance, Reach, Trust, Error prevention.
Algorithm 1
For disbanding this problem, the neutrosophic VIKOR framework was used via applying the following steps:
Criteria weights
Criteria weights
The neutrosophic ratings of alternatives with respect to each criterion presented in Table 2.
Neutrosophic rating of alternatives according to each criterion
The aggregated neutrosophic decision matrix
Each linguistic variable is a triangular neutrosophic number. For criteria weights, the linguistic variables are as follows: Very low (VL) = (0.0, 0.0, 0.25), Low (L) = (0.0, 0.25, 0.5), Medium (M) = (0.25, 0.5, 0.75), High (H) = (0.5, 0.75, 1), Very High (VH) = (0.75, 1,1).
For rating alternatives with respect each criterion, the linguistic variables are: Very Poor (VP) = (0.0, 0.0, 0.17), Poor (P) = (0.0, 0.17, 0.33), Medium Poor (MP) = (0.17, 0.33, 0.5), Fair(F) = (0.33, 0.5, 0.67), Medium Good (MG) = (0.5, 0.67, 0.83), Good (G) =) = (0.67, 0.83, 1), Very Good (VG) = (0.83, 1,1). The previous linguistic variables are used by decision makers to clear their preferences. The very important part is the confirmation degree of linguistic variable according to each decision maker. The confirmation degree can be as follows: Absolutely Sure (AS) = (1, 0, 0); the first parameter(1) is the truth degree, second parameter(0) is indeterminacy degree and finally the falsity degree. Very Strongly -Sure(VSS) = (0.9, 0.1, 0.1), Equally-Sure(ES) = (0.5, 0.5, 0.5), Absolutely Not Sure(ANS) = (0.0, 0.0, 1), Very Strongly Not Sure (VSN) = (0.3, 0.75, 0.7) and Strongly Not Sure (SNS) = (0.4, 0.65, 0.6). After determining linguistic variables and their confirmation degree according to each decision maker, the weights of criteria presented in Table 1. Each weight value is the linguistic variable concatenated with the confirmation degree.
Calculate the crisp weight value by using Equation (3) and then make a normalization process. The values of weights are as in Fig. 3 and equals the following: w1 = 0.11, w2 = 0.38, w3 = 0.13, w4 = 0.28, w5 = 0.03, w6 = 0.08.

Criteria weights values.
The crisp decision matrix
The ascending order for ranking alternatives

The separation of alternatives from the best value.
For making a comparative analysis and study the effect of last deneutrosophication process on ranking of alternatives, we will solve the same problem by using Algorithm 2 through the following steps:
Steps from 1 to 3 are the same in the previous algorithm. Step 4. The neutrosophic values of
Values of normalized neutrosophic difference
Values of normalized neutrosophic difference

The separation of alternatives from the best value.
From Fig. 5, it’s obvious that Singapore is the alternative with the largest separation degree from the best value and Australia is the alternative with the minimum separation degree.
The ascending order for ranking alternatives
It’s obvious that, the early deneutrosophication VIKOR approach is the easier. However the comparative study illustrated that, the first alternative is the worst e-government website according to two algorithms. The obtained values of R, Q from the first alternative are identical but from second algorithm, values are different but not very large. From second algorithm, the separation of alternatives from the best value is smaller than in Algorithm 1. We also concluded that, different deneutrosophication method led to different ranking of alternatives and choosing the suitable method also depends on context.
Comparative analysis with other research
By comparing our model with Burmaoglu and Kazancoglu model for evaluating e-government website, the results are as follows:
Our model which relies on neutrosophic VIKOR approach has a great representation of reality than Burmaoglu and Kazancoglu model. Because in our model we take into consideration all aspects of decision making process (i.e. agree state, not sure state and disagree state), but Burmaoglu and Kazancoglu concerned only on agree or truthiness state. By taking all aspects of decision making in our model, then we can efficiently represent vague, inconsistent and incomplete information than Burmaoglu and Kazancoglu model. Each linguistic variable in our model supported by the confirmation degree according to decision maker’s opinion and it’s a very important part for representing reality. But Burmaoglu and Kazancoglu doesn’t. In our model, the evaluation process of e-government website is performed by two algorithms. And a comparison between two algorithms results illustrated with detail.
Our model is the first for evaluating e-government website in neutrosophic environment.
Conclusions and future works
All e-government efforts are critically based on the accessibility of its website. The evaluation process of e-government websites is very important for developing performance and quality of websites. In our research the proposed model for evaluating e-government website based on VIKOR approach in neutrosophic environment. The weight of criteria and rating of alternatives represented by linguistic variables, the degree of truthiness, indeterminacy and falsity of each linguistic variable also determined according to each decision maker opinion. The linguistic variables and its degree of sureness presented by triangular neutrosophic numbers. Two algorithms are presented for evaluation process. The posterior and prior deneutrosophication in two algorithms affects results of ranking alternatives. Our proposed framework deals efficiently with vague, inconsistent and incomplete information by considering all aspects of decision making process. In the future we will consider the feedback and interdependence between criteria. We also will use other method for evaluation process, such as TOPSIS and ANP.
