A new method for ranking the usefulness of negative online reviews based on combined weighting method and improved TOPSIS

2.1 Online reviews and their usefulness

Online reviews refer to the comments made by consumers on the price, quality, performance and other aspects of goods or services after completing their consumption behaviors, which are published on the Internet platforms for other consumers to refer to and learn from when consuming online. Due to the asymmetry of online shopping information, potential consumers cannot fully understand the relevant information of the product before purchasing. The product and service information contained in online reviews is very precious, which can determine the purchase decision of potential consumers to a certain extent [7].

With the rapid development of e-commerce, the number of online reviews is increasing and there is a lot of noise, so the effectiveness is difficult to judge, which greatly affects the accuracy of obtaining user needs from product reviews. Therefore, the usefulness of online reviews has become a hot topic for many scholars. Usefulness means that online reviews can reduce the uncertainty of consumers in consumption process and provide a reference for purchasing decisions [5]. Singh et al. [8] proposed that the usefulness of reviews refers to the ranking of product reviews based on helpful votes. Hu et al. [9] pointed out that identifying useful reviews can reduce the time for consumers to find the information they need, provide potential consumers with more valuable information, and promote businesses to create diversified services.

The research on the usefulness of online reviews can be divided into two categories: influencing factor research and usefulness prediction. Existing studies on the factors affecting the usefulness of online reviews mostly focus on such factors as review polarity, product type, review length, review timeliness, review text characteristics, and reviewer characteristics. Liu et al. [10], starting from factors such as the characteristics of the reviewer, the length and star rating of the review, consumer perceived usefulness and the credibility of the review, used regression analysis to study the most influential factors. Lee et al. [11] showed that the number of reviews has a great impact on the usefulness of online reviews of experiential products, but has no significant impact on the usefulness of search-based product reviews. Research by Min et al. [12] showed that the number and quality of reviewers’ social networks have a significant impact on the usefulness of reviews. Zhang et al. [13] built a model based on the information adoption theory and found that the number of valid reviews, negative reviews, and uploaded pictures have a significant positive impact on the usefulness of reviews. Cai et al. [14] used linear regression to study the influence of the emotional intensity of the review text on the usefulness of the review in negative online reviews. In terms of usefulness classification methods, Sun et al. [15] studied the influence of the amount of information on the usefulness of reviews, and took into account the difference between search products and experience products, and proposed different classification thresholds to classify useful reviews. Felbermary et al. [16] used sentiment lexicon to extract sentiment attributes, and used random forest to build a classification model to measure the role of sentiment dimension in the usefulness of reviews. Chen [17] proposed an online product review credibility classification model based on DDAG-SVM based on the three-dimensional index characteristics of review content, reviewer characteristics and merchant characteristics. Feng et al. [18] built a model for predicting the usefulness of online reviews based on the text characteristics of online reviews and fused information gain, and used gradient descent algorithm to solve the model parameters based on a large amount of online review data.

2.2 Negative online reviews

Online reviews can be divided into positive online reviews, neutral online reviews and negative online reviews according to their valence. Negative online reviews refer to reviews posted by consumers on negative information about products on Internet to inform others of their dissatisfaction experience [19]. Compared with positive and neutral reviews, negative reviews expressing consumer complaints contain more accurate information, which can help consumers avoid purchase risks to a greater extent, improve the quality of consumer decision-making, and have a greater impact on consumers [20].

The impact of negative online reviews on the perception and behavior of potential consumers mainly depends on the information characteristics of the online negative reviews and the characteristics of the review recipients. The information characteristics of online negative reviews mainly include the number, intensity, quality, emotional expression of negative reviews and so on. From the perspective of emotional intensity, Cai et al. [14] found that strong negative emotions reduce the usefulness of negative reviews, and moderate negative emotions can increase the usefulness of negative reviews. Zhang et al. [21] analyzed the impact of negative online reviews on product sales from three aspects: the number of negative online reviews, the professional ability of review publishers, and product prices. Yang et al. [22] compared reviews of different product types from the dimension of review length. The study found that the number of negative reviews of search products was significantly more than that of positive reviews and neutral reviews, and the length of positive reviews and negative reviews of experiential products was significantly greater than that of neutral reviews. Ren et al. [23] extracted the three emotions of anger, fear, and sadness in negative online reviews, and found through empirical research that compared with search products, the emotion of anger in negative online reviews has a greater negative impact on the perceived usefulness of experience products. Zhao et al. [24] showed that when the content of negative online reviews is related to the service level of the business, and the business has a higher ability to improve their service, the business’s apology for the negative reviews will be more beneficial. Relevant research on receiver characteristics mainly involves receiver gender, perception of information, involvement, professionalism, search motivation and so on. For example, Lee et al. [25] found that consumers with a high degree of involvement are more likely to be affected by the quality of reviews, while those with a low degree of involvement are more likely to be affected by the number of negative reviews.

The above-mentioned scholars have studied online reviews and their usefulness from different focuses and angles and have achieved certain research results. However, the existing researches on online reviews mostly focus on online reviews as a whole or on positive online reviews. The degree of influence of each influencing factor on the usefulness of reviews will vary with the differences in product types and review polarity. This is also one of the reasons for the inconsistent research results on the impact of certain influencing factors on the usefulness of reviews.

3.1 Fuzzy analytic hierarchy process

The traditional analytic hierarchy process (AHP) requires the elements in the judgment matrix to be exact numbers. That is, the experts must have a clearer understanding of each index used in the judgment process and the various objects involved in the comparison. However, in the process of handling actual problems, it is more difficult to use precise values to assign values to the comparison judgment matrix. Experts prefer to use vague concepts such as “about” and “approximate” with the help of the fuzzy set theory proposed by Zadeh in 1965 [26], Chang [27] integrated the concept of fuzzy set into the analytic hierarchy process to form the fuzzy analytic hierarchy process (FAHP). This method can increase the rationality of the judgment matrix under the premise of fully considering the vagueness of personal judgment [28]. Therefore, this paper uses FAHP to calculate the weight of each index.

The triangular fuzzy numbers(TFNs) is based on the fuzzy number. A triangular fuzzy number(TFN), denoted as s ˜ = ( p , r , t ) is composed of membership functions [26] as follows: u s ˜ ( x ) = { x - p r - p for p ≤ x ≤ r t - x t - r for r ≤ x ≤ t 0 otherwise(1) The language terms used in this paper and the corresponding TFN [29] are shown in Table 1.

The steps of FAHP based on TFNs are described below.

Step 1: Decision-making experts construct comparative judgment matrices based on TFNs. The elements of the comparison judgment matrix represent the relative importance of the two evaluation indices at the same level in the minds of experts. If there are n evaluation indices at a certain level, the fuzzy judgment matrix made by a single expert at this level is shown in Equation (2): S ˜ = [ 1 s ˜ 12 . . . s ˜ 1 n s ˜ 21 1 . . . s ˜ 2 n . . . . . . . . . . . . s ˜ n 1 s ˜ n 2 . . . 1 ](2) where s ˜ kl is a TFN, which represents the result of comparing the relative importance of the index k and the index l in the opinion of experts.

In order to reduce individual differences, a questionnaire on the relative importance of evaluation indices will be issued to experts, and the scoring results will be processed according to Equation (3) as the element values of the m dimensional matrix set S ˜ m = { s ˜ klm } . p kl = min m = 1 , 2 , . . . , M ( p klm ) r kl = ∏ m = 1 M r klm M t kl = max ( t klm ) m = 1 , 2 , . . . , M }(3)

Step 2: Defuzzification of TFN. We use Equation (4) to defuzzify the TFN s ˜ kl = ( p kl , r kl , t kl ) to make it an accurate value [29]: ( s k l ∂ ) β = [ β × p k l ∂ + ( 1 − β ) × t k l l ] , 0 ≤ β ≤ 1 , 0 ≤ ∂ ≤ 1 p k l ∂ = ( r k l − p k l ) × ∂ + p k l t k l ∂ = t k l − ∂ × ( t k l − r k l ) }(4) where ∂ indicates the preference display of the decision maker. The uncertainty is maximum when ∂=0 and the stability gradually increases as ∂ increases. β indicates the attitude of the decision maker towards risk. β = 1 indicates highly pessimistic where β = 0 indicates highly optimistic.

Step 3: Construction of a comparison matrix (5) based on exact values:

S = ( 1 ( s 12 ∂ ) β . . . ( s 1 n ∂ ) β ( s 21 ∂ ) β 1 . . . ( s 2 n ∂ ) β . . . . . . . . . . . . ( s n 1 ∂ ) β ( s n 2 ∂ ) β . . . 1 )(5)

Step 4: Normalization of the defuzzified matrix: N kl = ( s kl ∂ ) β ∑ k = 1 m ( s kl ∂ ) β(6) where k = 1, 2, . . . , m ; l = 1, 2, . . . , n

Step 5: Calculation of the weights of indices and sub-indices: w j ′ = N th rootvalue ∑ N th root(7)

Step 6: Checking the Consistence Index (C . I) of the matrix: ( C . I ) = γ max - n n - 1(8)

Step 7: Calculation of Consistence Ratio (C . R): ( C . R ) = C . I R . I(9) where R . I represents the random index, and its value varies with the change of the matrix size n. The value of C . R ≤ 0.1 indicates the matrix to be consistent.

The concept of “information entropy”, as a measure of uncertainty and information, was proposed by Shannon. If the amount of information of the plan to be evaluated is larger, the uncertainty is smaller, and the entropy is smaller; otherwise, the entropy value is larger. According to the characteristics of entropy, the entropy value can be calculated to determine the degree of dispersion of an evaluation index. The greater the degree of dispersion of the index, the greater the influence of the index on the comprehensive evaluation. The index weight value obtained by the entropy method is based on the information reflected by the degree of variation of the index data of each measurement, which can effectively reduce the interference of human factors in the index weighting [30]. The specific implementation steps of the entropy method are as follows:

Step 1: Standardize each index by using the Range method to avoid the inconsistency of different measurement indices in terms of order of magnitude and dimension: y ij = { x ij - min ( x ij ) max ( x ij ) - min ( x ij ) when x ij is a positive index max ( x ij ) - x ij max ( x ij ) - min ( x ij ) when x ij is a negative index(10) where i represents the object to be evaluated and j represents the evaluation index. x _ij and y _ij represent the original and standardized index values respectively.

Step 2: Calculate the information entropy E _j of each index y _ij : E j = ln 1 n ∑ i = 1 n [ ( y ij / ∑ i = 1 n y ij ) ln ( y ij / ∑ i = 1 n y ij ) ](11)

Step 3: Calculate the weight ω j o of each indicator y _ij : w j o = ( 1 - E j ) / ∑ j = 1 m ( 1 - E j )(12)

The traditional TOPSIS method judges the degree to which the object to be evaluated is close to the ideal solution based on the Euclidean distance. However, since the Euclidean distance to the ideal solution may also be close to the Euclidean distance of the negative ideal solution, there are certain limitations of evaluation results. The virtual worst solution is used to replace the negative ideal solution, and the spatial projection distance and angle relationship among the evaluation plan, the ideal solution and the virtual worst solution are comprehensively considered. Using the combination of projection method and angle measurement method to evaluate the construction of a new relative fit can avoid these limitations [31]. Suppose that there are m reviews A₁, A₂, …, A _m containing n indices X₁, X₂, …, X _n . The data value of indices are x _ij  (i = 1, 2, …, m ; j = 1, 2, …, n) and the weight of the index X _j is ω _j , W = (ω₁, ω₂, …, ω _n )  ^T .

Step 1: Standardize all raw data values by using equation (13) to obtain standardized index values. According to the index weight ω _j , we use equation (14) to calculate the weighted standardization matrix. y ij = x ij ∑ i = 1 m x ij(13) F = ( f ij ) = ( w i y ij )(14)

Step 2: Determine the positive ideal solution F⁺ and the virtual worst solution F^*-. Define the maximum value of each evaluation index as a positive ideal solution f j + , and the minimum value of each evaluation index as a negative ideal solution f j - . In order to avoid the phenomenon of unreasonable ranking in the evaluation process, the virtual worst solution is adopted to replace the negative ideal solution [32]. F + = ( max 1 ≤ i ≤ m fij ) = ( f 1 + , f 2 + , . . . , f n + )(15) F - = ( min 1 ≤ i ≤ m fij ) = ( f 1 - , f 2 - , . . . , f n - )(16) f j * - = ( 2 f j - - f j + )(17) Step 3: Calculate the projection distance and included angle between each negative review and the positive ideal solution and the virtual worst solution. Regarding each index as a dimension, the weighted standardized index value, the positive ideal solution, and the virtual worst solution corresponding to each online negative review are regarded as a multi-dimensional vector in the space coordinate system. As shown in Fig. 1, in a multi-dimensional space, the orthographic projection distance t⁺ is the projection distance of F^*-F _i from the virtual worst solution in the direction of F^*-F⁺. The negative projection distance t^*- is the projection distance of F^*-F _i from the virtual worst solution in the F^*-F⁺ direction. The included angle θ⁺ formed by each online negative review and the positive ideal solution is called a positive included angle, and the included angle θ^*- formed by each online negative review and the virtual worst solution is called a negative included angle, as shown in Fig. 2. The calculation methods of t⁺ and t^*- are shown in Equations (18) and (19), and the calculation methods of θ⁺ and θ^*- are shown in Equations (20) and (21).

OPEN IN VIEWER

Fig. 1

Positive and negative projection distance diagram.

OPEN IN VIEWER

Fig. 2

Positive and negative included angle diagram.

t i + = ( F i - F + ) · ( F * - - F + ) | | F * - - F + | | , i = 1 , 2 , . . . , m(18) t i * - = ( F i - F * - ) · ( F + - F * - ) | | F + - F * - | | , i = 1 , 2 , . . . , m(19) θ i + = arccos ( ( Fi · F + ) / ( | | F i | | | | F + | | ) ) , i = 1 , 2 , . . . , m(20) θ i * - = arccos ( ( Fi · F * - ) / ( | | F i | | | | F * - | | ) ) , i = 1 , 2 , . . . , m(21)

Step 4: According to Equation (22), the projection distance and included angle are nondimensionalize: M new = M i ∑ i = 1 m M i , i = 1 , 2 , . . . , m(22) where M _i indicates t i + , t i * - , θ i + , θ i * - , which are denoted as t i + , t i * - , O i + , O i * - after nondimensionalization.

Step 5: From step 3, it can be seen that the smaller T i + is, the closer the projection distance of a certain online negative review is to the positive ideal solution; the smaller T i * - is, the closer the projection distance of a certain online negative review is to the virtual worst solution. When O i + is smaller, the vector formed by a certain online negative review is closer to the vector formed by the positive ideal solution; the smaller O i * - is, the vector formed by the online negative review is closer to the vector formed by the virtual worst solution. Combining space projection distance and vector angle relationship, when T i + and O i + are smaller, the solution is closer to the positive ideal solution, T i * - and O i * - are smaller, the solution is closer to the virtual worst solution. So, we define S i + and S i * - as S i + = μ T i + + λ O i + , i = 1 , 2 , . . . , m(23) S i * - = μ T i * - + λ O i * - , i = 1 , 2 , . . . , m(24) where μ, λ indicate decision-making experts preference for projection distance and angle, and satisfies 0 ≤ μ, λ ≤ 1, μ + λ = 1. Here we take μ = λ = 1.

Step 6: According to the projection angle coefficient P _i , comprehensively measure the degree to which each object to be evaluated is close to the positive ideal solution and the principle virtual worst solution. P _i can be calculated according to Equation (25). The larger the value, the more effective the online negative review. P i = S i * - S i + + S i * - , i = 1 , 2 , . . . , m(25)

4.3 Reviewer-related characteristics

6.1 Data acquisition and processing

We select JD.com Fresh Food platform as an example, and uses python to crawl the negative review data of the top 5 products in the five categories of fresh fruits, seafood, meat, frozen food, and vegetables from 2018 to 2020. Excluding invalid negative reviews and repeated reviews, a total of 6667 review data were obtained. First, use the TF-IDF method to extract keywords. The higher the TF-IDF value of a word, the greater its representative ability to the document, and it is more suitable to be a keyword for the document. Based on the existing stopwords list, our study adds “fruit”, “negative reviews”, “true” and other general evaluation words that often appear but do not have much meaning for product information. Then the keywords are extracted, and the top 150 words in word frequency ranking are selected as the extraction result according to opinions of experts. The keywords are represented by word vectors through the Word2Vec method, and each word is represented by a 100-dimensional vector. And in the Word2Vec processing, the cosine value between word vectors is calculated. The larger the cosine value between the vectors, the higher the correlation between the two words and the greater the similarity. As a result, the cluster categories shown in Table 3 and the corresponding attribute word database can be obtained. Due to space limitations, only a part of the word database is shown here.

After obtaining the attribute vocabulary, the word segmentation technology is used to obtain the words contained in the text comments of each fresh online negative review. By traversing the words and comparing with the attribute words in the attribute vocabulary, the product attribute mentioned in the negative review text can be extracted.

In order to obtain the index weight value, we designed a questionnaire and invited 56 consumers who often purchase fresh products on the online platform to fill it out. Due to the limited length of the article, we have attached the questionnaire survey results of three experts in the appendix, as shown in Table 13. The results of the questionnaire can be used to calculate the weight values of the various indicators that affect the helpfulness of the online negative reviews of fresh products as shown in Tables 4 to 8. The weight value in the brackets of each indicator feature corresponds to the weight value of the indicator’s helpfulness for online negative reviews of fresh commodities, that is, the weight value of the overall target.

In order to avoid the sparseness of the data display and the limitation of space, here we chose the cold drink and frozen food product in the negative online review data set of fresh e-commerce. The serial number of the product is 191-200. The quantitative results of 10 reviews are shown in Table 9. The final helpfulness ranking results are shown in Table 10.

Table 11 lists the top five negative reviews of one certain brand of dairy products on JD.com Fresh Food according to the model in this article. And Table 12 lists the top five online negative reviews shown to customers on the product page of JD.com Fresh Food. From Figs. 4 and 5, it can be seen that comparing the sorting rules of negative online reviews on JD.com Fresh Food, the text information, pictures, and video information contained in the top negative reviews ranked according to our model are more comprehensive and rich, and can provide potential consumers with more information about product details.

OPEN IN VIEWER

Fig. 4

Fitting diagram of the normal distribution of closeness P _i -value.

OPEN IN VIEWER

Fig. 5

Helpfulness ranking results of negative online reviews according to traditional TOPSIS and improved TOPSIS (partial).

In order to verify the effectiveness of the improved TOPSIS method based on the combined weighting method proposed in this paper, this study uses the traditional TOPSIS method to calculate the relative closeness of the obtained fresh e-commerce negative online reviews helpfulness index data, that is, the closeness between negative reviews and positive and negative ideal solutions. The closeness between the ideal solutions is compared with the data obtained by the improved TOPSIS method proposed in this paper, as shown in Tables 11-12.

It can be seen from the data normal distribution fitting graph in Fig. 4 that compared with the traditional TOPSIS model, the fit data distribution calculated by the improved TOPSIS model based on the combined weighting method proposed in this study is more concentrated, which means the data fitting effect is better. We ranked the helpfulness of the 6667 negative online reviews crawled according to the traditional TOPSIS, compared with the improved TOPSIS ranking proposed in this article, and randomly selected the ranking results of 30 comments, as shown in Fig. 5. The overall trend of the relative closeness of the usefulness of each comment obtained by the two schemes is consistent, but our model considers the angular relationship between each review and the ideal solution and the virtual worst solution in the calculation process, which avoids some extreme values.