ASA: A framework for Arabic sentiment analysis

Abstract

Sentiment analysis (SA), also known as opinion mining, is a growing important research area. Generally, it helps to automatically determine if a text expresses a positive, negative or neutral sentiment. It enables to mine the huge increasing resources of shared opinions such as social networks, review sites and blogs. In fact, SA is used by many fields and for various languages such as English and Arabic. However, since Arabic is a highly inflectional and derivational language, it raises many challenges. In fact, SA of Arabic text should handle such complex morphology. To better handle these challenges, we decided to provide the research community and Arabic users with a new efficient framework for Arabic Sentiment Analysis (ASA). Our primary goal is to improve the performance of ASA by exploiting deep learning while varying the preprocessing techniques. For that, we implement and evaluate two deep learning models namely convolutional neural network (CNN) and long short-term memory (LSTM) models. The framework offers various preprocessing techniques for ASA (including stemming, normalisation, tokenization and stop words). As a result of this work, we first provide a new rich and publicly available Arabic corpus called Moroccan Sentiment Analysis Corpus (MSAC). Second, the proposed framework demonstrates improvement in ASA. In fact, the experimental results prove that deep learning models have a better performance for ASA than classical approaches (support vector machines, naive Bayes classifiers and maximum entropy). They also show the key role of morphological features in Arabic Natural Language Processing (NLP).

Keywords

Arabic convolutional neural network deep learning long short-term memory machine-learning sentiment analysis

1. Introduction

Nowadays, the Internet is full of users’ opinions, comments and expressed sentiments in many fields. This is due to the explosion of big data generated by social networks websites [1], personal blogs, online communities and web forums and other Internet sources. Indeed, millions of users express online their opinions and send their comment about many topics such as varied products, companies services, used cars and transports, social or natural phenomena, political issues, health, education and technological advances.

Worldwide, such expressed opinions and comments constitute a valuable information mine. In fact, companies, governments and institutions are increasingly interested in mining those big data to improve their products.

Thus, sentiment analysis (SA) is a valuable tool to explore such raw big data and extract valuable information. In fact, many scientists were attracted by this field, and many research communities are working to improve this area and the tools offered. One of the objectives of SA is to analyse the public opinions on social networks. More specifically, SA aims to categorise opinions and comments as holding positive, negative, or neutral sentiments. Such analysis enables companies and institutions to get a good knowledge in order to enhance their strategies and services or better understand patterns and behaviours.

For instance, SA helps marketing managers to understand and monitor customer’s tendencies as well as opinions about products, services or other issues [2]. Governments and political actors are also interested in monitoring citizen’s sentiments about various political issues, their offered services, population health and more [3]. Globally, SA [4] aims to identify the attitude of an online user or a community regarding a topic, a service or a product. It helps also to evaluate the overall tonality of a document.

In our study, we are interested in using SA for Arabic language and more specifically for the classification of Arabic texts at the sentence level. A sentence can be a blog, a tweet, a comment or an online review. Therefore, our goal is to explore how SA can be improved to determine if an Arabic sentence is expressing a positive, negative or neutral sentiment. We choose the Arabic language for several reasons because Arabic language is well spread among various countries and used by millions of people across the world [5]. It is an important language as far as the historical, cultural and social aspects are concerned. Furthermore, the Arabic language raises important issues and challenges for the SA due to its complex structure and morphology.

Various research works have been conducted on SA during the last decade, and many of them have focused on Arabic [6]. We decided to contribute to enrich this field by building a corpora for the Arabic language, especially that there is a growing audience generating many Arabic tweets, comments and opinions.

In previous works, several SA systems were developed for modern standard Arabic (MSA) and some dialects (mainly Egyptian and Levant dialects), whereas, in this work, we focused on SA for both MSA and Moroccan Dialectal Arabic (MDA) microblogs.

However, we faced one major problem, which is the lack of annotated data. To overcome this problem, we started by using and evaluating the performance of the proposed framework with the available resources from MSA and dialects. Then we created and annotated our own data set.

It is worth mentioning that there are other challenges facing the analysis of Moroccan tweets. This is because, Moroccan users tend to use multiple languages and dialects in Twitter or Facebook. Therefore, a sentence in a Moroccan tweet may contain words from standard Arabic, Moroccan Arabic ‘Darija’, Moroccan Amazigh dialect ‘Tamazight’, French, Spanish, and English. Indeed, Moroccan users like to mix words from multiple languages in their casual communications. It reflects also the richness of their culture and their diverse origins. Therefore, analysing Moroccan tweets is so complex.

In addition to the specificity of Moroccan tweets, we should know that there are other classical challenges faced in any SA case.

Indeed, the majority of the text produced by the social websites has an unstructured or noisy nature. This is due to the lack of standardisation, spelling mistakes, missing punctuation, non-standard words, repetitions and more.

In this study, our goal is to provide an adapted and efficient framework for Arabic text preprocessing and SA. The contribution of this article is as follows:

First, we build a new rich Arabic corpus for SA, which combines standard Arabic and Moroccan dialect from various real sources.

Second, we implement and evaluate the performance of various machine learning (ML) for the Arabic language. Therefore, we separately test each of those three algorithms namely support vector machines (SVMs), naïve Bayes (NB) classifiers and maximum entropy (ME) when using tweets written in MSA or in MDA.

Third, we evaluate if deep learning models have good or higher performance for Arabic sentiment analysis (ASA) than other state-of-the-art approaches.

Finally, we analyse and evaluate the performance of the selected algorithms when applied to Arabic using different preprocessing techniques such as normalisation, stop words and stemming. It is worth to point out that the idea of stemming is commonly used for other languages (such as English, German and Spanish). In this respect, the experiments are conducted to prove if the differences in performance when changing the stemmer are statistically significant or not. Our goal is to determine the best combination of processing techniques and classification models recommended for Arabic.

The rest of this article is organised as follows: section 2 explains and surveys the different approaches used for sentence-level sentiment classification and summarises the related work. Section 3 describes our proposed system for classifying Arabic sentiment. Section 4 presents the results of our system and their interpretations. Finally, the main conclusions and future works are presented.

2. Related works

SA has become an important field, since it helps to extract valuable knowledge from various sources of big data on the Internet. In fact, the proliferation of social media, blogs, comments and feedbacks of online customers makes SA very important. Generally, the SA task can be achieved using one of those methods: either ML-based method or lexical-based method. In the following, we detail and compare those two methods.

ML methods are usually based on classification. In fact, the sentiment detection is categorised according to a binary result (i.e. positive or negative). For this reason, this method requires labelled data to train classifiers.[7].

The advantage of this learning approach [8] is the ability to adapt trained models for specific contexts and purposes. However, one of the constraints is the availability of labelled data. Indeed, it is difficult to apply this approach to new data, since it may be expensive or unfeasible to get labelled data for newly generated data.

On the contrary to machine-learning approach, lexical-based methods [4] are based on a predefined list of words. Each word is linked to a specific sentiment. Thus, different lexical methods may be applied according to different contexts.

From the revised literature, it was proved that the machine-learning approaches (NB, ME and SVM) are more suitable for the case of Twitter than the lexical-based approach [4,8,9].

Currently, many ASA models have been developed based on the machine-learning methods. Such models implement different classifier algorithms and offer various features. It is noticed that the proposed approaches represent documents based on the Bag of Words (BoW) model. The approaches tried to extend word features with different features such as n-grammes (e.g. bi-gramme and tri-gramme), stems and part-of-speech (POS) tags.

Duwairi and Qarqaz [10] built an SA model for Arabic Twitter and Facebook comments. In their model, the texts were represented as a set of word bigram features. They also investigated the effect of using term frequency (TF) and term frequency inverse document frequency (TF-IDF) weighting schemes with SVM, NB and K-nearest neighbour (KNN) classifiers.

Abdul-Mageed et al. [11] presented a new system called subjectivity and sentiment analysis system (SAMAR). The proposed system relies on an SVM classifier for different Arabic social media applications: Web forums, chat, Wikipedia Talk Pages and Twitter. They tested different features including word n-grammes, POS tagging and word stems. They compared also various stylistic features related to social media applications.

Duwairi [12] explored an SA approach for Arabic tweets written in Jordanian Arabic dialectical and MSA. Indeed, they worked to enhance the performance of the SVM and NB classifiers by transforming words in tweets from their dialect form to the MSA form.

Hammad and Al-awadi [13], in their published work, studied the SA on Arabic hotel reviews collected from Twitter, Facebook and YouTube. For their experiments, they tested different classifiers including NB, SVM, DT and back-propagation neural network (BPNN), ML classifiers with BoW, POS tag and stem features. They proved that in their case, the SVM classifier shows the best average accuracy, followed by NB, DT and finally BPNN.

Duwairi and El-Orfali [14] evaluated the effects of stemming, feature correlation and n-gramme models for Arabic text on SA: three classifiers, namely, SVM, NB and KNN classifiers are used to evaluate their model. The results demonstrated that preprocessing strategies on the reviews increases the performance of the classifiers.

Alomari et al. [15] proposed also an Arabic Jordanian twitter corpus. This later classified Tweets as either positive or negative. They tested and applied various supervised ML SA approaches to Arabic users in social media. The used Tweets are written in either MSA or Jordanian dialect. The conducted experiments aimed to evaluate the use of different weight schemes, stemming and n-gramme terms techniques and scenarios. The results provided the best scenario for each classifier. Through their experiments, they showed that SVM classifier using TF-IDF weighting scheme with stemming through bi-gramme features outperforms the naïve Bayesian classifier.

Rahab et al. [16] proposed an approach to annotate Arabic comments extracted from Algerian Newspapers websites as positive or negative classes. For this work, they created an Arabic corpus named SIAAC (Sentiment polarity Identification on Arabic Algerian newspaper Comments). They tested two well-known supervised learning classifiers which are SVMs and NB. For experiments, they used different parameters and various measures in order to compare and evaluate results (recall, precision and F-measure). In terms of precision, the best results were obtained using SVM and NB. It was proved that the use of bi-gramme increases the precision for the two models. Furthermore, SIAAC showed competitive results in comparison to OCA (opinion corpus for Arabic) [17].

The effect of applying stemming and n-gramme techniques for Arabic texts (tweets) on sentiment classification was presented by Khoja and Garside [18]. They also investigated the impact of feature selection on the performance of the classifier. For this reason, three classifiers: SVM, NB, and KNN are used. The conducted experimentations showed that the best results of performance are obtained when applying a hybrid representation, which includes tokens with tri-grammes. Experimentation results also revealed that the use of feature-selection technique significantly improved the accuracy of the three classifiers for the task of opinion classification. Regarding the classifiers, SVM outperformed the other classifiers when using all the features. However, when selecting the most relevant features by the SVM feature-selection technique, the classifiers SVM and NB provided the best results.

Asghar et al. [19] proposed a framework that analyses customer information and their sentiments in order to better assist decision makers. Indeed, in this study, the feedbacks received via a microblogging site such as Twitter are deeply analysed to assist both online users and companies in the process of product selection, product development, product promotion and other services. For this reason, the proposed framework is based on a set of theory-based Twitter SA system to classify user sentiments. It uses rough set theory (RST)-centric rule induction algorithms. Their solution improves previous RST-based algorithms (such as LEM2) and ensures a better and an efficient sentiment classification process.

Al-Shawakfa et al. [20] compared six Arabic root-based stemmers based on a unified testing process. For which, they built a corpus that contains 27.6 million words using 3823 tri-literal roots. The results of the comparison showed that the best algorithm was that proposed by Ghawanmeh et al. [21], which demonstrated an accuracy rate of 39%, followed by Khoja (34%), Alshalabi (32%) [22], Sonbol et al. (24%) [23], Taghva et al. (20%) [24] and finally Al-Shalabi et al. (14%) [25]. Their main observations on the algorithms on hand were as follows, having in mind that all accuracy results are based on their own generated corpus:

The algorithm suffers from the affix ambiguity problems: if these prefixes and/or suffixes were removed, wrong roots would be generated.

The algorithms are time-consuming when they are improved to obtain better accuracy results.

In addition, it is important to notice that the results of any root stemming vary according to the corpus used. Therefore, various research works may differ about the efficiency of the same stemmers. In addition, while some articles showed that Khoja stemmer is less efficient [20], other articles confirmed that Khoja has a good performance [26,27]. For this reason, we considered Khoja in our research.

Deep learning (DL) is a field that recently emerged as part of artificial neural network. Many studies evaluated or used DL models in order to recognise and understand data such as images, audio and text. DL models are trained by embedding vectors that capture syntactic and semantic properties of the words [28]. It is based on multiple levels of representations and abstractions. They correspond to a set of features, factors and concepts. In DL, higher-level concepts are defined from lower-level ones and the same lower-level concepts can help define many higher-level concepts [29].

Many studies developed DL models and showed that they can be of higher efficiency and accuracy for English SA. Some examples of those models are recursive auto encoders (RAEs) [30], recursive neural tensor networks (RNTNs) [31], long short-term memory (LSTM) networks [32], gated recurrent neural networks (GRNN) [33,34] and dynamic memory networks (DMN) [35].

However, limited work was done to investigate DL for ASA. For instance, the studies [36,37] showed that according to preliminary results, the DL models called RAE and LSTM performed better than baseline SVMs. They demonstrated that performance can increase by enhancing some aspects of RAE; for which, they used sentiment embeddings for input and text representation based on morphological tokenization.

Further improvements were achieved by exploring the space of morphology and orthography. Baly et al. [38] trained RNTN with stems, automatic discretization and marked letter repetition and achieved the best results on ArSenTB; that is the first Arabic sentiment Treebank.

We refer the readers to read Al-Ayyoub et al.’s survey [39] in order to have an overview about the studies that used and evaluated DL for ASA.

3. Our approach

The aim of this research is to develop an ASA framework that improves the accuracy of SA of Arabic text. This is achieved by first implementing DL models for Arabic text then combining and comparing various proven techniques for both pre-processing and classification phases.

The overall architecture of our proposed framework consists of two main phases: training and testing. In the training phase, the classifier learns from a set of labelled tweets. Afterwards, the classifier is capable of classifying new unlabeled tweets in the testing phase. Each phase consists of the following steps: tweets preparation, tweets pre-processing, features extraction/selection, and classification.

The general process of the proposed ASA model is as follows: first, the tweets data sets are collected and annotated. Second, the tweets are pre-processed. Third, the features representation model is constructed. Fourth, the ML classifier is trained and evaluated on unlabeled data. As a final step, we applied various DL models to classify the Arabic tweets as either positive or negative. Then, we measured their accuracy and efficiency. The following sub-sections explain the phases of the proposed ASA framework.

3.1. Data collection

Being aware of the challenges related to the Moroccan dialect and Arabic in general, we created a publicly available SA data set. It was prepared manually by collecting reviewers’ opinions from Hespress website against various published articles and from a combination of Arabic reviews and comments from Facebook, Twitter and YouTube. The collected corpus, called Moroccan Sentiment Analysis Corpus (MSAC) [40] is characterised using informal and non-standard vocabulary such as repeated letters and non-standard abbreviations, the presence of onomatopoeia (i.e. pff and hhh) and non linguistic content such as emoticons.

It is noticed that some comments are written in Arabic but with English characters and numbers which are called Franco-Arabic or Arabizi Language. Many Moroccans web users use numbers instead of Arabic characters like ‘5’ to replace ‘خ’ (kh) and ‘7’ to replace ‘ح’ (h) This fact makes the Moroccan dialect content unique. Our system focuses on the comments written in Arabic language and ignores comments written in Arabizi language. Furthermore, our data set contains comments written in different forms. It is interesting to mention that there are several different forms of the various Arabic languages spoken in Morocco. Arabic language was divided into high and low dialects: Moroccan Arabic, Hassaniyya Arabic, Judeo-Moroccan Arabic, Moroccan sign language, Tachelhit, Central Atlas Tamazight and Ghomara Senhaja de Srair.

MSAC is a multi-domain corpus consisting of the text covering a maximum vocabulary from sport, social and political domains. For our experiments, we used two different corpora: MSAC and another freely available data set that is generated from SemEval-2017 Task 4 [41]. The latter has the same size and contains tweets and comments related to various topics such as technology, politics and sports. The data sets have high quality, since they were written by professional reviewers. For our tests, we decided to extract 2000 reviews: 1000 positive and 1000 negative. They are available in MSA and Arabic dialect.

3.2. Tweets cleaning and annotation

3.3.1. Data cleaning

Data cleaning is an essential step to guarantee data quality. It improves also the detection of tweets polarity. To attain it, we decided to clean our collected corpus (MSAC) from spelling errors and slang words. Furthermore, we addressed the issues of abbreviations, replication of characters and removed duplicated tweets. We also processed this corpus in order to delete user-names, hash tags, URLs, punctuations and unwanted whitespaces.

3.2.2. Tweet annotation

In this step, we removed neutral tweets because we only studied positive and negative tweets. We categorised each tweet in the data set manually into two classes: positive and negative. Our final corpus contains about 1000 of positive tweets and 1000 of negative ones. The corpus covers various topics. Table 1 gives a description of this data set.

Table 1.

Description of the MSAC corpus.

Properties	Positive	Negative
Number of tweets	1000	1000
Total number of words	10189	13900
Average number of words per tweets	10.1	13.9

MSAC: Moroccan Sentiment Analysis Corpus.

3.3. Tweets pre-processing

Before extracting the feature vectors, it is required to pre-process the tweets. The pre-processing task involves two steps: first, the tokenization and second, the pre-processing mechanism (normalisation, stemming and stop words removal) is applied on the cleaned tweets.

3.3.1. Tokenization

The first phase of the preprocessing module consists in the tokenization. It consists of splitting the text into words (tokens) separated by whitespaces or punctuation characters. The result of this operation is a set of words. Our framework offers various types of tokenization including the Natural Language Toolkit (NLTK) library.

3.3.2. Normalisation

As another step of pre-processing, it is essential in many cases to normalise data sets before applying classification models. In text mining, this means that we convert all the forms of a word into a common form. In this study, we decided to normalise the used data sets in order to ensure a consistent form of the used Arabic tweets. In fact, we included in the framework our normalizer that respects a predefined list of rules. In more detail and in order to get common forms of texts, the added normalizer removes the ‘tatweel’ character ‘_’(e.g. using tatweel the word ‘رحــــــيم’ (mercy) may look like (‘رحيم’). It also removes the tashkeel (مُشْكِلة (problem) >مشكلة). The normalizer removes diacritics, punctuation, numbers and special characters. Furthermore, the normalizer addresses the repetitions; it looks for two or more repetitions of characters which expresses affirmation and accentuation and replace them with the character itself (جميييييل جداااا –> جميل جد). In addition to all that, it replaces آ, إ and أ with ا. Finally, it replaces the final letter ة with ٥ and ي with ى.

For our experiments, we decided to use the PyArabic, a specific Arabic language library for Python. In fact, the PyArabic offers basic functions to manipulate Arabic letters and text such as detecting Arabic letters, Arabic letters groups and characteristics, removing diacritics, tashkeel. Furthermore, it includes Tashaphyne library for word normalisation.

3.3.3. Removing the stop words

Stop words like pronouns, articles and prepositions and other words are used frequently in tweets. However, they are not significant for text mining or SA. That is why we decided to remove the stop words before applying classification models or DL models. Stop words are grouped into two categories: general and domain specific. The first category is available in the public domain which includes the standard stop words, it usually has a fixed size, the second category includes non-standard stop words which are generated inside information retrieval or text classification systems. The size of domain-specific stop lists depends on the contextual and statistical characteristics of the corpus, including the sparsity of the corpus, the size of the vocabulary, and the number of sub-domains [42]. Our goal is to increase the accuracy and enhance the efficiency of ASA. By deleting such words, we can better identify the most relevant words that help into classification. In this research, the list of stop words (called Arabic StopList) is manually established. For instance, those words have been removed: (‘في’ (in), ‘انت’ (you), ‘من’ (of) and so on)

3.3.4. Lexicon developments

People are increasingly using emoticons to express their opinions or emotions. Therefore, we took into consideration texts written in informal language in the tweets. For this reason, as we can see from Table 2, three types of lexicons were created: lexicon for social acronyms, lexicon for emoticons and lexicon for interjections. Then, we converted them to their positive or negative sentiment.

Table 2.

Lexicon developments.

Style tag	Typographical symbols	Sentiment
Acronyms’ lexicon	LOL, GRB, KK	Positive
	WTF	Negative
Emoticons’ lexicon	;), :>, :, :-, :-x, :p, :-p, ^_^, O, :), :)	Positive
	:(, :-(, :/, :’(, :<, 3, :), o.o	Negative
Injections’ lexicon	Wow, waw, Haha, hihi, hehe, Oy, Tyt, اھاھ	Positive
	Oh dear, no way	Negative

3.4. Stemming

Stemming [43] is among the most important pre-processing steps in mining opinions as well as a very common requirement for natural language processing (NLP). Stemming standardises words by reducing each word to stem, base or root form. The application of the derivation enables to reduce the corpus data set size into a small dimensional space.

In order to improve the efficiency of text categorization, it is important to use the stemming technique. In fact, the stemming consists of extracting the affixes (such as infixes, prefixes and suffixes) from a word. Two types of stemming approaches can be cited [44]: light stemming and root extraction. The goal of light stemming is to extract the stem of the word by deleting the identified prefixes and suffixes. On the contrary, the goal of root extraction is to extract the words root by removing all the types of the words affixes (including infixes, prefixes and suffixes). It is worth mentioning that the root and the stem of a word need not to be identical. For instance, the root of the almaktabatالمكتبات (the libraries) is ktbكتب (write) and the stem is maktabمكتب (office).

Our framework offers two stemming approaches for Arabic text. These approaches include light stemming methods (Light10) and root-extraction methods (KHOJA). Our main goal is to identify the approach that is more suitable for the Arabic sentiment classification.

3.4.1. Khoja Arabic stemmer

Khoja’s stemmer [18] works by removing the longest suffixes and the prefixes. It then compares the remaining words with verbal and nominal patterns for extracting the root. The stemmer uses several linguistic data files as a list of all diacritical characters, punctuation characters, specific articles, and 168 stops words. Figure 1 presents the result of Khoja stemmer with our ASA framework.

3.4.2. Light stemmer

Light stemmer works by removing prefixes and suffixes from words to output a stem. However, it does not deal with patterns or infix to convert words into their root form. Light10 is the most used Arabic light stemmer [44].

3.5. Feature extraction

After text pre-processing, the next step is feature extraction/selection. The goal of feature-selection technique is to find the most relevant features for the classification task by removing irrelevant, redundant and noisy data [45]. Feature selection has a second objective that is the reduction of both the dimensionality of the feature space and the processing time. Many text features are considered for SA [46] such as n-gramme models and POS. The latter is used to find adjectives that contain opinion information. An n-gramme is a contiguous sequence of n terms from a given sequence of text. An n-gramme of size 1 is referred to as a uni-gramme; an n-gramme of size 2 is a bi-gramme; an n-gramme of size 3 is a tri-gramme. N-grammes of larger sizes are referred to by the value of n and keeping the words with the highest score according to a predefined threshold (predetermined measure of the importance of the word). We used unigrams during our experiments because it provided the best performance.

In the feature extraction step, the text is transformed to a vector representation. In this model, the weight of the word (feature) is calculated according to the document containing that word. There are several weighting schemes such as: Boolean weighting, TF weighting, inverse document frequency (IDF) weighting, and TF-IDF.

In our framework, the feature vectors applied to the classifier consists of a binary weighting (presence). The weight of every token or word is determined using the binary model where a token is given a weight equals to 1 if it is present in the tweet under consideration. Otherwise, the token is given a weight equals to 0 if the token is absent from the tweet. This weighting was introduced originally in SA by Mohammad et al. [47]. TF-IDF can be used as an alternative to the binary model. However, for SA, the binary model has been widely used by several researchers; hence, we chose to use the binary model in our experiment.

3.6. Training and testing classifiers

Our framework offers three algorithms for Arabic sentiment classifications. The data is classified using three supervised ML algorithms that are NB, SVM, ME classifiers. Such data are also classified using DL technique. In the following, we explain those algorithms.

3.6.1. Maching learning

An NB classifier [48] is a probabilistic classifier which is based on the probability models. The main assumption in this approach is the independency of the features. NB constitutes one of the widely used techniques for text classification. It has been used in various research studies such as Duwairi et al. [49], Tripathy et al. [50] and Catal and Nangir [51]. This classifier can be applied in various fields such as personal email sorting, document categorization, language detection, sentiment detection as well as the detection of spams in emails or sexually explicit content. NB is a classifier that can ensure good results.

The SVM algorithm [52] is a linear classification/regression algorithm. The aim of the SVM is to identify a best hyper-plane that separates two classes of data with the largest possible margin. According to many other studies, SVM ensures very good performance and high accuracy in the case of SA and for various languages. In this research work [53], it has been proved that SVM ensured good results in the case of English language in comparison to other classifiers. In addition, Medhat et al. [54] confirmed that SVM showed good results for SA of reviews that are written in Chinese. In our experience, we implemented linear support vector classification (LinearSVC). It is worth mentioning that BernoulliNB and LogisticRegression can also be used instead of LinearSVC.

The ME classifier [55] is a probabilistic classifier which belongs to the class of exponential models. Unlike the NB classifier, the ME does not assume that the features are conditionally independent of each other. The ME [56] is based on the principle of ME and from all the models that fit our training data; it selects the one which has the largest entropy. The ME classifier consumes more time for training the model in comparison to NB. However, the ME is useful for various text classification problems. For instance, it can be used for language detection, SA as well as topic classification and more. We used generalised iterative scaling (GIS) algorithm. The other available algorithms are improved iterative scaling (IIS) and Limited memory Broyden-Fletcher-Goldfarb-Shanno (LM-BFGS) algorithm.

Figure 2 presents the result of classifying Arabic opinion with SVM using our framework ASA.

3.6.2. DL technique

DL [30] is a recent subfield of ML and an efficient outcome of artificial neural network. In the last years, many researchers have studied DL for SA.

LeCun et al. [57] demonstrated that DL constitutes a very efficient approach for computational modelling and Big data. That is why, DL is now used in many domains. The main known categories of DL are deep neural networks (DNNs) based on various hidden layers. Recurrent neural networks (RNN) and convolutional neural networks (CNNs), LSTM. The shallow learning is another type that is widely used. It is based on at most three layers levels and widely used for linear problems [58].

The idea behind DL is to mimic the human brain in order to obtain the best possible representation of the studied data sets [39]. For that, the models are based on a set of algorithms, a solid network architecture as well as many neurons and layers. They are used to represent and characterise the data using different abstractions. The architecture consists of input layer and an output layer with many hidden layers (encoders) between them. Such hidden layers execute complex computations in order to extract features from the raw data. The ultimate goal is to learn multiple levels of representation in order to find a model for high-level abstractions in a data set.

The DL network [59] consists of several layers of interconnected neuron elements. The first layer represents the input that matches the space of the feature. Then, the first layer is followed by several layers of nonlinearity. At the end, the last layer corresponds to the output space.

In general, DL models work as follows: the weighted combination $α = Σ wi i xi + b$ of the inputs is aggregated, then the output signal f (α) is communicated by the connected neuron. In their network, such models use both a nonlinear activation function (f) and a threshold. In fact, the activation threshold of the neuron is calculated using the bias b.

In this article, our aim is to improve the ASA using the DL models. Another goal is to enhance Arabic text mining by improving the performance outcomes based on the combination of both the tested DL models and various pre-processing techniques. For this, two DL models are proposed under the proposed DL framework, namely CNN and LSTM.

CNN is one powerful category of DNN that demonstrated high performance and very good results in different feature-learning problems in computer vision, speech recognition, and text analysis areas [60]. Therefore, Kim [61], Wang et al. [62] and Al Sallab et al. [36] incorporated CNN as a powerful solution. In this domain, Shin et al. [63] presented an integrated CNN, lexicon models and naïve concatenation.

CNN is a multi-layer neural network that uses one or more convolutional and pooling layers. Such layers are followed by one or more fully connected layers. In fact, the convolutional filter and max pooling layers are iterated to extract the most relevant feature representation from the input data. The size and weights of the convolution filters are used to determine the most significant features. Convolution filter is floated over the complete input data in order to extract similar features at different spatial locations. The activation function called Softmax is also used for the connected layer.

LSTMs are another type of RNNs [64]. Such networks are used for objects that happen recurrently like time series data. They can learn the relationships between elements in an input sequence. They demonstrated high accuracy in many works done in text mining and English SA [32]. They sequentially process data or tweet’s words in order to detect long- and short-term relations between the current word and previous ones. Given some input as a sequence of vectors, an LSTM combines the current word with the output representation of preceding words to produce an updated output. For that, LSTM uses a cell composed of input, output, forget and memory gates. After going through the whole data, the network returns a decision about each vector in the sequence. The produced output is then used to train a softmax layer for sentiment classification [65].

LSTMs were designed to reduce dependencies in long sequences and to overcome the problem of a vanishing gradient problem in RNNs. This is by using a memory-cell. Therefore, it conserves the operations state on earlier vectors in the sequence and prevents vectors from forgetting status through multiple iterations [66].

4. Experimental results and discussions

To develop and implement our framework, we used python with its text mining library, which contains different tools designed to assist in the preparation of text documents for mining tasks (tokenization, stop word removal, and stemming, among others). Python is also used to implement three different classifiers (NB, ME, and SVM) and to estimate their performance.

Furthermore, in order to train and evaluate DL models (LSTM and CNN), we used Keras [67] with TensorFlow libraries. We used also pre-trained word embeddings as input features for LSTM and CNN. Indeed, the embedding layer can be created either by using weights of pre-trained embeddings (word2vec) or by learning embeddings from the vocabulary. In our case, the learned embeddings is used.

Concerning the parameters, we decided to set the hyper parameters for the network architecture as those used by Oswal [68] and Nagy [69].

As mentioned above, the 10-fold cross-validation was used to test the classifiers. We applied the Arabic stemming algorithm included to reduce words to their common root or stem. In addition, the Arabic stop words list was also applied to the texts of the corpus to remove words without relevant meaning. The impact of using stemming in the text pre-processing was also analysed. Furthermore, we evaluated the use of two different stemming techniques (light stemming and root extraction).

The performance of the developed approach was evaluated based on the most common performance measures including accuracy and precision (P).

Table 3 presents the results achieved from different classifiers in terms of precision, accuracy, recall and F-measure.

Table 3.

ML results without pre-processing.

Feature	SVM	NB	ME	CNN	LSTM
Accuracy (%)	80	67	78.5	96	93
Precision (%)	80.3	70.1	78.6	95	94
Recall (%)	80	67.1	78.5	94	93
F-measure (%)	80	65.1	78.5	95	93

ML: machine learning; SVM: support vector machines; NB: naïve Bayes; ME: maximum entropy; CNN: convolutional neural networks; LSTM: long short-term memory.

From the experimental results shown in Table 3, we can compare the results of the tested ML classifiers. The results demonstrate that SVM had better results than NB classifier in almost all the evaluation measures. It reached 80% of accuracy and 80.3% of precision.

Through the experiment, NB showed less performance than SVM. In fact, the best performance outputs achieved by NB are 67% as accuracy and 70.1% as precision.

By comparing DL and single models, the experiments show that DL enhances the efficiency of classification in terms of precision and accuracy. In fact, the CNN and LSTM algorithms performed well and outperformed the single model (SVM, NB and ME). The CNN and LSTM algorithms ensure the highest accuracy with 96% and 93%, respectively. They demonstrated a precision of (95% and 94%), respectively. Indeed, SVM showed a lower accuracy (80%) and less precision (80.3%). Therefore, we can confirm that CNNs have dramatically improved the sentiment classification. One of the key differentiators between CNNs and traditional ML approaches is the ability for CNNs to learn complex feature representations.

By considering those outputs, we conclude that DL models are recommended for Arabic sentiment classification, as they ensure high accuracy and performance in comparison to other methods. However, this solution has a negative effect side, as it consumes more time for finishing the training phase than a single model.

Our conclusion from this study confirms the conclusions obtained in other studies for English language, which confirm that DL substantially improves the performance of sentiment classification [36,38,58].

4.1. The impact of preprocessing on ML classifiers

The objective of this experiment is to test the effect of the preprocessing on ASA.

To test the performance of our proposed pre-processing stages, we test the three selected ML in the following cases: (1) after applying our normalizer and removing the stop words, (2) after applying light stemmer and (3) after applying Khoja stemmer. In each of those stages 10-fold cross-validation method was used.

The purpose of the first experimentation is to test the effect of normalisation and removing stop words on Arabic sentiment classification, this is shown in Table 4. The goal of the last four experiments is to study the influence of stemming on the sentiment classification performance. The goal is to compare various stemming approaches including light stemmers and root-extraction stemmers. The results are shown in Tables 5 and 6.

Table 4.

ML results with normalisation and stop word removal.

Feature	SVM	NB	ME	CNN	LSTM
Accuracy (%)	83	69.8	82.5	99	97.2
Precision (%)	82.8	71.8	82.3	98	97
Recall (%)	83	69.9	82.2	98.8	98
F-measure (%)	82.9	69.7	82.2	99	97.5

ML: machine learning; SVM: support vector machines; NB: naïve Bayes; ME: maximum entropy; CNN: convolutional neural networks; LSTM: long short-term memory. The bold values show that normalization and removing stop word gives the best results (compared to table 3).

Table 5.

ML results using stemmed tweets.

Properties	SVM		CNN		LSTM
	Accuracy	Precision	Accuracy	Precision	Accuracy	Precision
Light 10	82	82.1	95.5	94.9	89.9	89
Khoja	80.8	80.8	93.1	92	88.3	87
No Stem	80	80.3	90	89.2	86.8	85

ML: machine learning; SVM: support vector machines; CNN: convolutional neural networks; LSTM: long short-term memory.

Table 6.

Performance of the classifiers on the second data set.

Features	SVM	NB	ME	CNN	LSTM
Without preprocessing	79	77.2	75	92.1	90
Normalisation and stop word removal	86	84.5	83.9	95.4	93
Light stemmer	88.6	81	80	89	88.8
Root stemmer	80	78.8	75	85	85.7

SVM: support vector machines; NB: naïve Bayes; ME: maximum entropy; CNN: convolutional neural networks; LSTM: long short-term memory.

4.1.1. The impact of stop words and normalisation

Table 4 shows the performances of the three classifiers NB, ME and SVM using normalisation and stop word removal on the ASA. The performance is measured in terms of accuracy, precision, recall and F-measure.

When we compare the results obtained with the results of ML without pre-processing (Table 3), we can clearly note that the best performance of all classifiers is achieved when applying the pre-processing stage. In fact, as it can be seen from Table 4, removing stop word gives the best results in the three classifiers. For this model based on removing stop words, the best result of accuracy in the three classifiers is provided by CNN (99%). The best result of accuracy without removing stop word is provided also by CNN (96%) as shown in Table 3.

Therefore, if we consider the results of models in terms of the best accuracy. It is noticed that when we apply pre-processing, the accuracy is increased by 3% for CNN. Furthermore, the accuracy gets higher by almost 4.2% for LSTM, by almost 3% for the SVM, by almost 2.8% for the NB and almost 3.8% for the ME. Those improvements are also present for precision, recall and F-measure.

Those results confirm that the pre-processing stage is important to have better accuracy and performance for ASA. Indeed, the preprocessing is good to reduce the noise in the text, and it is helpful to reduce or eliminate some distortions in the features space.

We observe that the number of features has decreased when applying the pre-processing from 10,189 to 8756 features for positive tweets. When applying the pre-processing, the number of features has also decreased from 13,900 to 10,172 features for negatives tweets. In fact, by applying more steps from the preprocessing stage, we can reduce the problem of features over-fitting. This is because more related features will converge together. The pre-processing stage enhances also the rate of the learning stage.

This last experimentation confirms the results obtained by the first experimentation. It shows that the SVM classifier outperforms the ME and NB when using normalisation and removing stop word. Therefore, we can conclude that the SVM’s algorithm outperform the NB and ME in all cases. Our results for ASA are thus aligned with the previous published researches [19] [70] for English SA.

This experimentation confirms also the results obtained in the first experimentation. It shows that DL improves classification performance. Indeed, the experiments demonstrated that CNN and LSTM model have higher accuracy than other traditional models (SVM, NB and ME).

4.1.2. The impact of stemming

In these experiments SVM, CNN and LSTM have been applied to a set of collected tweets, they were tested under three preprocessing conditions of no stemming, root stemmer (Khoja) and light stemmer (light 10). The results are shown in Table 5.

Table 5 shows that the best performance of the three classifiers was achieved when using stemming (whether root stemming or light stemming). The observation shows that CNN achieved the highest accuracy in 95.5% when using light 10 stemmer. In contrast, it reaches 90% without stemmer. Therefore, we can note clearly that using the stemming factor affects positively the performance of CNN classifier.

The same results were obtained by Al-Badarneh et al. [27] and Mustafa et al. [43], which demonstrated that stem is a better choice to use in classifying Arabic text. Our obtained result is consistent with the finding of Liu and Zhang [45] which showed that the pre-processing steps like stemming improves SA accuracy.

This proves that the stem is a better choice to use when classifying Arabic text. This is because, it makes the corpus data set smaller since many words that are generated from the same stem are represented as one word (their stem). This will enhance both the processing time and storage utilisation and help to achieve the highest level of accuracy.

For its importance, stemming is used in many areas such as data mining, information retrieval and some applications of NLP. In fact, it is commonly used in SA that is a popular application of NLP and text mining [71]. Stemming has many advantages, as confirmed by Al-Shammari and Lin [72] and Sembok et al. [73]. Indeed, it reduces the size of the index terms. Stemming also helps to improve information retrieval efficiency by reducing the words to common roots.

By comparing the stemming methods, and as can be seen from Table 5, the methods based on the light stemming approaches outperform the methods based on root-extraction techniques using SVM, CNN or LSTM.

In fact, we can see that light stemmer showed better results than Khoja. It achieved the best accuracy using CNN at 95.5% followed by LSTM with 89.9% and SVM with 82%.

The result shows also that the best results were achieved with CNN using the light10 stemmer with 94.9% precision followed by Khoja with 92%. This proves that light stemmer technique outperformed root stemmer technique in our case study.

To summarise, light stemming approaches show better performance than root-extraction methods. In fact, light stemming protects the meaning of information by removing only words suffix and prefix. However, the root-extraction approach extracts the root or the base of each word, and therefore, it loses some meaningful morphological information.

The experimental results demonstrated that DL techniques are a good choice for ASA, and they have higher accuracy and performance than classical ML used for SA.

4.2. Performance for the second data set

The same sets of experiments were carried out on the second data set (data extracted from SemEval-2017 task 4). Table 6 shows the accuracy of different classifiers for the various tweet representations.

From Table 6, we can clearly note that the best performance of all classifiers is achieved when both stop words and normalisation techniques are used. In fact, as it can be seen from Table 6, removing stop word and normalisation give the best results in the all classifiers. For this model, the best result of accuracy is provided by CNN (95.4%) while the best result of accuracy without removing stop words is 92.1%. Therefore, we can notice that removing stop word increased the performance for all algorithms used.

Concerning stemming methods, the observation shows that the best performance of all classifiers is achieved when using stemming. In fact, the best accuracy is achieved by CNN classifier when light stemmer is used at 89% followed by root stemmer at 85%. Therefore, this study proves that the use of stemming methods enhance the performance of all classifiers in our case of Moroccan Arabic tweets.

To summarise the results of Tables 4 and 5, we notice that for our corpus, all classifiers reach their best performance when using unigrams with removing stop words and stemming technique. In addition to that, the results confirm that DL models have higher performance if we compare them to other existing classifiers. Thus, we can conclude that DL methods are appropriate for Arabic sentiment classification and Moroccan Arabic tweets as a special case.

5. Conclusion and future work

The article proposes and details our framework ASA designed for Arabic tweets. The framework implements various techniques and efficient models for both Arabic text pre-processing and ASA. To develop our framework, we built a new Moroccan Arabic data set which consists of 2000 tweets/comments, with a good balance between negative and positive sentiments. The data used include informal structures, non-standard dialects and many spelling errors as they were coming from variety of users and real sources.

In this work, several aspects of data representations were investigated. The feature vectors of the tweets were pre-processed in several ways, and the effects of these on the classifiers accuracy were investigated. The experiments showed that stop words removal, normalisation and stemming slightly improved the performance of the classification. In addition, the experimental results showed that DL models (CNN and LSTM) are more efficient and demonstrated higher accuracy than ML (SVM, NB and ME) in the case of ASA. DL models performed better than traditional models in all the following cases: using uni-grammes, using stop words, without stop words, with stemming or without stemming. This study proved the important potential of DL for ASA in terms of accuracy and performance.

We think that the accuracy can be enhanced by finding the best parameters of LSTM and CNN, so as future work, we will study the effect of each parameter on DNN.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Ahmed Oussous

References

Oussous

Benjelloun

Lahcen

, et al. Big data technologies: a survey. J King Saud Univ-Comp Inform Sci 2018;30: 431–448.

Chen

Cai

Lai

, et al. A topic-based sentiment analysis model to predict stock market price movement using Weibo mood. Web Intel Agent Sys 2016;14(4): 287–300.

Duwairi

Ahmed

Al-Rifai

SY.

Detecting sentiment embedded in Arabic social media – a lexicon-based approach. J Intel Fuzzy Sys 2015;29(1): 107–117.

Boudad

Faizi

Thami

, et al. Sentiment analysis in Arabic: a review of the literature. Ain Shams Eng J 2018;9: 2479–2490.

Oussous

Lahcen

Belfkih

Improving sentiment analysis of moroccan tweets using ensemble learning. In: Tabii

Lazaar

Al Achhab

Enneya

(Eds.) Big Data, Cloud and Applications. Berlin: Springer International Publishing, 2018, pp. 91–104.

Al-Moslmi

Albared

Al-Shabi

, et al. Arabic senti-lexicon: constructing publicly available language resources for Arabic sentiment analysis. J Inform Sci 2018;44(3): 345–362.

Gonçalves

Araújo

Benevenuto

, et al. Comparing and combining sentiment analysis methods. In Proceedings of the first ACM conference on Online social networks, Boston, MA, 7–8 October, 2013, pp. 27–38. New York: ACM.

Godsay

The process of sentiment analysis: a study. Int J Comp Appl 2015;126(7): 26–30.

Mostafa

AM.

An evaluation of sentiment analysis and classification algorithms for Arabic textual data. Int J Comp Appl 2017;158(3): 29–36.

10.

Duwairi

Qarqaz

Arabic sentiment analysis using supervised classification. In: International conference on future internet of things and cloud (FiCloud), Barcelona, 27–29 August 2014, pp. 579–583. New York: IEEE.

11.

Abdul-Mageed

Kuebler

Diab

. SAMAR: a system for subjectivity and sentiment analysis of social media Arabic. In: Proceedings of the 3rd workshop on computational approaches to subjectivity and sentiment analysis (WASSA), ICC Jeju, Republic of Korea, 12 July 2012. pp.19–28.

12.

Duwairi

. Sentiment analysis for dialectical Arabic. In: 6th international conference on information and communication systems (ICICS), Amman, Jordan, 7–9 April 2015, pp. 166–170. New York: IEEE.

13.

Hammad

Al-awadi

. Sentiment analysis for Arabic reviews in social networks using machine learning. In: Latifi

. (Ed.), Information technology: new generations. Berlin: Springer International Publishing, 2016, pp. 131–139.

14.

Duwairi

El-Orfali

A study of the effects of preprocessing strategies on sentiment analysis for Arabic text. J Inform Sci 2014;40(4): 501–513.

15.

Alomari

ElSherif

Shaalan

. Arabic tweets sentimental analysis using machine learning, In: Benferhat

Tabia

Ali

(Eds.) Advances in Artificial Intelligence: From Theory to Practice. Berlin: Springer International Publishing, 2017, pp. 602–610.

16.

Rahab

Zitouni

Djoudi

. SIAAC: sentiment polarity identification on Arabic Algerian news article comments. In: Proceedings of the Computational Methods in Systems and Software Applied Computational (CoMeSySo 2017) Intelligence and Mathematical Methods. Berlin; Springer, 2017, pp. 139–149.

17.

Rushdi-Saleh

Martín-Valdivia

Ureña-López

, et al. OCA: opinion corpus for Arabic. J American Soc Inform Sci Technol 2011;62(10): 2045–2054.

18.

Khoja

Garside

Stemming Arabic text. Lancaster: Computing Department, Lancaster University, 1999.

19.

Asghar

Khan

, et al. RIFT: a rule induction framework for twitter sentiment analysis. Arabian J Sci Eng 2018;43(2): 857–877.

20.

Al-Shawakfa

Al-Badarneh

Shatnawi

, et al. A comparison study of some Arabic root finding algorithms. J American Soc Inform Sci Technol 2010;61(5): 1015–1024.

21.

Ghawanmeh

Al-Shalabi

Kanaan

, et al. An algorithm for extracting the root for the Arabic language. In: Proceedings of the 5th international business information management association conference, Cairo, Egypt, 13–15 December 2005.

22.

Alshalabi

Pattern-based stemmer for finding Arabic roots. Inform Technol J 2005;4(1): 38–43.

23.

Sonbol

Ghneim

Desouki

MS.

Arabic morphological analysis: a new approach. In: 3rd International Conference on information and communication technologies: from theory to applications (ICTTA), Damascus, Syria, 7–11 April 2008, pp. 1–6.

24.

Taghva

Elkhoury

Coombs

Arabic stemming without a root dictionary. In: International conference on information technology: coding and computing (ITCC’05), Las Vegas, NV, 4–6 April 2005, pp. 152–157.

25.

Al-Shalabi

Kanaan

Al-Serhan

. New approach for extracting Arabic roots. In: International Arab conference on information technology, Alexandria, Egypt, 21–23 December 2003.

26.

Oraby

El-Sonbaty

El-Nasr

MA.

Finding opinion strength using rule-based parsing for Arabic sentiment analysis. In: Castro

Gelbukh

González

(eds) Mexican international conference on artificial intelligence. Berlin: Springer, 2013, pp. 509–520.

27.

Al-Badarneh

Al-Shawakfa

Bani-Ismail

, et al. The impact of indexing approaches on Arabic text classification. J Inform Sci 2017;43(2): 159–173.

28.

Mikolov

Sutskever

Chen

, et al. Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th International Conference on Neural Information Processing Systems, Volume 2. Red Hook, NY: Curran Associates Inc., 5–10 December 2013, pp. 3111–3119.

29.

Rojas-Barahona

LM.

Deep learning for sentiment analysis. Lang Linguist Compass 2016;10(12): 701–719.

30.

Socher

Pennington

Huang

, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the conference on empirical methods in natural language processing, Edinburgh, 27–31 2011, pp. 151–161. Stroudsburg, PA: Association for Computational Linguistics.

31.

Socher

Perelygin

, et al. Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the conference on empirical methods in natural language processing, Seattle, WA, 18–21 October 2013, pp. 1631–1642. Stroudsburg, PA: Association for Computational Linguistics.

32.

Tai

Socher

Manning

CD.

Improved semantic representations from tree-structured long short-term memory networks, 2015, https://arxiv.org/abs/1503.00075

33.

Tang

Qin

Liu

. Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the conference on empirical methods in natural language processing, Lisbon, Portugal, 17–21 September 2015, pp. 1422–1432. Stroudsburg, PA: Association for Computational Linguistics.

34.

Baly

Hobeica

Hajj

, et al. A meta-framework for modeling the human reading process in sentiment analysis. ACM Trans Inform Sys 2016;35(1): 7.

35.

Kumar

Irsoy

Ondruska

, et al. Ask me anything: dynamic memory networks for natural language processing. In: Balcan

Weinberger

(Eds) Proceedings of The 33rd International Conference on Machine Learning, PMLR, New York, 2016, pp. 1378–1387.

36.

Al Sallab

Hajj

Badaro

, et al. Deep learning models for sentiment analysis in Arabic. In: Proceedings of the second workshop on Arabic natural language processing, Beijing, China, 26–31 July 2015, pp. 9–17. Stroudsburg, PA: Association for Computational Linguistics.

37.

Baly

Badaro

El-Khoury

, et al. A characterization study of Arabic twitter data with a benchmarking for state-of-the-art opinion mining models. In: Proceedings of the third Arabic natural language processing workshop, Valencia, 3 April 2017, pp. 110–118. Stroudsburg, PA: Association for Computational Linguistics.

38.

Baly

Hajj

Habash

, et al. A sentiment treebank and morphologically enriched recursive deep models for effective sentiment analysis in Arabic. ACM Trans Asian Low-Res Lang Inform Proces 2017;16(4): 23.

39.

Al-Ayyoub

Nuseir

Alsmearat

, et al. Deep learning for Arabic NLP: a survey. J Comput Sci 2018;26: 522–531.

40.

https://github.com/ososs/Arabic-Sentiment-Analysis-corpus

41.

http://alt.qcri.org/semeval2017/task4/index.php?id=results

42.

Makrehchi

Kamel

MS.

Extracting domain-specific stopwords for text classifiers. Intel Data Anal 2017;21(1): 39–62.

43.

Mustafa

Eldeen

Bani-Ahmad

, et al. A comparative survey on Arabic stemming: approaches and challenges. Intel Inform Manage 2017;9(2): 39–67.

44.

El-Defrawy

El-Sonbaty

Belal

NA.

A rule-based subject-correlated Arabic stemmer. Arabian J Sci Eng 2016;41(8): 2883–2891.

45.

Liu

Zhang

A survey of opinion mining and sentiment analysis. In: Aggarwal

Zhai

(Eds) Mining Text Data. Boston, MA: Springer, 2012, pp. 415–463.

46.

Pang

Lee

Vaithyanathan

. Thumbs up? sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing - Volume 10, Stroudsburg, PA, 6 July 2002, pp. 79–86. Stroudsburg, PA: Association for Computational Linguistics.

47.

Mohammad

Alwada’n

Al-Momani

Arabic text categorization using support vector machine naïve Bayes and neural network. GSTF J Comput (JOC) 2018;5(1): 108–115.

48.

Saloot

Idris

Mahmud

, et al. Hadith data mining and classification: a comparative analysis. Artificial Intel Rev 2016;46(1): 113–128.

49.

Duwairi

Alfaqeh

Wardat

, et al. Sentiment analysis for Arabizi text. In: 7th International Conference on Information and Communication Systems (ICICS), Irbid, Jordan, 5–7 April 2016, pp. 127–132. New York: IEEE.

50.

Tripathy

Agrawal

Rath

SK.

Classification of sentiment reviews using n-gram machine learning approach. Exp Sys Appl 2016;57: 117–126.

51.

Catal

Nangir

A sentiment classification model based on multiple classifiers. Appl Soft Comput 2017;50: 135–141.

52.

Zhang

Law

. Sentiment classification of online reviews to travel destinations by supervised machine learning approaches. Exp Sys Appl 2009;36(3): 6527–6535.

53.

Wiebe

, et al. Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP. In: Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP, Suntec, Singapore, 2–7 August 2009.

54.

Medhat

Hassan

Korashy

Sentiment analysis algorithms and applications: a survey. Ain Shams Eng J 2014;5(4): 1093–1113.

55.

El-Halees

AM.

Arabic text classification using maximum entropy. IUG J Nat Stud 2015;15(1): 157–167.

56.

Da Silva

Hruschka

Jr.

Tweet sentiment analysis with classifier ensembles. Dec Supp Sys 2014;66(1): 170–179.

57.

LeCun

Bengio

Hinton

Deep learning. Nature 2015;521(7553): 436–444.

58.

Deng

Deep learning: methods and applications. Found Trends Sig Proc 2014;7(3–4): 197–387.

59.

Vieira

Pinaya

Mechelli

Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: methods and applications. Neuro Biobehav Rev 2017;74: 58–75.

60.

Yuan

Xiang

Task-specific word identification from short texts using a convolutional neural network. Intel Data Anal 2018;22(3): 533–550.

61.

Kim

Convolutional neural networks for sentence classification, 2014, https://arxiv.org/abs/1408.5882

62.

Wang

Jiang

Luo

. Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, The COLING 2016 Organizing Committee, Osaka, Japan. 11–17 December 2016, pp. 2428–2437.

63.

Shin

Lee

Choi

. Lexicon integrated CNN models with attention for sentiment analysis. In: Proceedings of the 8th workshop on computational approaches to subjectivity sentiment and social media analysis, Copenhagen, Denmark, 7–11 September 2017, pp. 149–158. Stroudsburg, PA: Association for Computational Linguistics.

64.

Salehinejad

Baarbe

Sankar

, et al. Recent advances in recurrent neural networks, 2017, https://arxiv.org/abs/1801.01078

65.

Baly

El-Khoury

Moukalled

, et al. Comparative evaluation of sentiment analysis methods across Arabic dialects. Proced Comp Sci 2017;117: 266–273.

66.

Hochreiter

The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Uncertain Fuzz 1998;6(02): 107–116.

67.

Keras. Deep learning for humans, 2015, https://github.com/keras-team/keras

68.

Oswal

BV.

CNN-text-classification-keras, GitHub repository, 2016, https://github.com/bhaveshoswal/CNN-text-classification-keras

69.

Nagy

Sentiment-analysis-NLTK-ML and LSTM, GitHub repository, 2017, https://github.com/nagypeterjob/Sentiment-Analysis-NLTK-ML-LSTM

70.

Abdulla

Al-Ayyoub

Al-Kabi

MN.

An extended analytical study of Arabic sentiments. Int J Big Data Intel 2014;1(1–2): 103–113.

71.

Dahab

Ibrahim

Al-Mutawa

A comparative study on Arabic stemmers. Int J Comp Appl 2015; 125(8), pp. 975–8887.

72.

Al-Shammari

Lin

Towards an error-free Arabic stemming. In: Proceedings of the 2nd ACM workshop on improving non English web searching, Napa Valley, CA, 30 October 2008, pp. 9–16. New York: ACM.

73.

Sembok

TMT

Ata

BMA

Bakar

. A rule-based Arabic stemming algorithm. In: Proceedings of the 5th European Conference on European Computing Conference. Wisconsin. Stevens Point, WI: World Scientific and Engineering Academy and Society (WSEAS), 2011, pp. 392–397.