The ascent of network traffic classification in the dark net: A survey

Network traffic can be identified and sorted into distinct categories using a port-number-based classification system [6]. This method is typically used in firewalls, routers, and other network devices to control and manage the flow of traffic on a network. The process of port-based traffic classification begins with the identification of the port number associated with a particular packet of traffic. This can be done by examining the header of the packet, which contains information about the IP addresses as well as the port numbers. Once the port number has been identified, the traffic is then classified based on a predefined set of rules. These rules can be based on a variety of factors, such as the IP addresses and the port number itself. For example, traffic that is sent to or received from port 80, which is typically associated with web traffic, may be classified as "web traffic" and given a higher priority than other types of traffic. Similarly, traffic that is sent to or received from port 22, which is typically associated with SSH traffic, may be classified as "secure traffic" and given a higher priority than other types of traffic. Once the traffic has been classified, it can then be managed and controlled using a variety of techniques, such as filtering, blocking, or prioritising. This allows network administrators to control the flow of traffic on their network and ensure that critical applications and services are given the resources they need to function properly [7]. Overall, port-based traffic classification is a powerful tool for managing and controlling network traffic and is widely used in a variety of network devices and applications. So, it is very important to understand and use it properly in order to have better network performance. The algorithm 1 iterates through each packet in the network and extracts the destination port number from the packet header. It then checks if the destination port number is in a predefined list of web ports, SSH ports, or email ports. If it matches one of these lists, the packet is classified as web traffic, secure traffic, or email traffic, respectively. If it does not match any of these lists, it is classified as general traffic.

2.2 Payload-based traffic classification

The technique of payload-based traffic classification involves the identification and classification of network traffic by analysing the packet or frame payload’s content [9], as shown in Algorithm 2. This method is typically used in network devices such as firewalls, intrusion detection systems, and deep packet inspection (DPI) devices to control and manage the flow of traffic on a network [10]. The process of payload-based traffic classification begins by examining the content of the packet or frame payload. This can be done by using a variety of techniques, such as regular expressions, string matching, or machine learning algorithms. These techniques are used to extract features from the payload and identify patterns or characteristics that can be used to classify the traffic. For example, if the payload contains certain keywords or phrases that are commonly associated with a specific type of traffic, such as "HTTP" or "FTP," the traffic can be classified as web or file transfer traffic [11]. Similarly, if the payload contains certain patterns or characteristics that are commonly associated with a specific type of traffic, such as encryption or compression, the traffic can be classified as secure or compressed traffic. Once the traffic has been classified, it can then be managed and controlled using a variety of techniques, such as filtering, blocking, or prioritising [12]. Network administrators can regulate the traffic flow on their networks and guarantee that essential applications and services receive the necessary resources to operate smoothly. Payload-based traffic classification is an effective technique for monitoring and managing network traffic, especially in identifying and preventing harmful traffic like phishing, malware, and spam. However, it requires more advanced and sophisticated tools and resources and is more difficult to implement and maintain than port-based traffic classification.

2.3 User-behavior based traffic classification

User-behaviour-based traffic classification is a technique that uses information about user behaviour to classify network traffic, as shown in Algorithm. This approach focuses on analysing the behaviour of individual users and their interactions with the network rather than just looking at the technical characteristics of the traffic itself. The application of machine learning techniques [13] is one of the most important components of user-behaviour-based traffic classification. These models are trained on massive amounts of information acquired from the network, including information about user behaviour, network usage patterns, and other relevant metrics. The methods are developed to identify trends and patterns in the data that can be utilised to classify traffic into various groups. One of the advantages of user-behaviour-based traffic classification is that it is able to identify and classify new types of traffic that may not have been seen before. This is important in today’s rapidly changing digital environment, where new types of traffic and applications are constantly emerging. Additionally, this approach is less dependent on the technical characteristics of the traffic, which can change over time or be modified by attackers. Another advantage of user-behaviour-based traffic classification is that it can be used to detect and prevent malicious activity on the network [14]. For example, by analysing the behaviour of individual users, it is possible to identify unusual or suspicious activity that may indicate a cyberattack. This can include things like repeated login attempts, unusual traffic patterns, or the use of known malicious IP addresses. In conclusion, user-behaviour-based traffic classification is a powerful technique that can be utilised to analyse and classify different kinds of network traffic based on user behaviour. By using machine learning algorithms and analysing large amounts of data, this approach is able to identify new types of traffic and detect malicious activity on the network [15]. This can help organisations better understand and manage their network traffic and improve the security and performance of their networks.

2.4 Statistical flow feature-based approach

The statistical flow feature-based method is a classification method for network traffic that analyses the statistical properties of network traffic flow to categorise it into various groups [16] as shown in Algorithm 4. This approach typically involves calculating various statistical features of the traffic flow, such as the mean, variance, skewness, kurtosis, and entropy, and using these features to differentiate between different types of traffic. One equation commonly used in the statistical flow feature-based approach is the Shannon entropy equation, which calculates the entropy of a traffic flow based on the probability of each packet in the flow: H = - ∑ p ( i ) * log 2 ( p ( i ) )(1)

Where H is the Shannon entropy, p (i) is the probability of packet i, and ∑ is the summation over all packets in the flow. Using this formula, we can determine how random or unpredictable the traffic flow is, which helps us classify it into various categories. For example, a traffic flow with high entropy may be classified as random or unstructured, while a traffic flow with low entropy may be classified as structured or predictable.When it comes to accurately classifying various forms of network traffic, the statistical flow feature-based approach is a useful method because it considers the statistical properties of the traffic flow and can differentiate between different forms of traffic based on these characteristics. [17, 18].

Unsupervised machine learning is a technique for programming a computer to recognise trends and classify data without labelled training material, as shown in Algorithm 3. Unsupervised machine learning is applied in network traffic classification to recognise patterns in network traffic and categorise it into various types, including legitimate traffic, spam traffic, and harmful traffic [31] and [34]. One common unsupervised machine learning approach used in network traffic classification is clustering. Clustering is an approach to categorising similar data points by taking into account their features. In network traffic classification, clustering algorithms are used to group similar flows of network traffic together based on their features, such as IP address, port number, and packet size [35]. The equation for the ascent of network traffic classification in the Dark Net using an unsupervised machine learning approach can be represented as: C = f ( D , ML )(2) where C = network traffic classification; D = darknet data; and ML = unsupervised machine learning approach. The function f represents the steps required to classify darknet traffic using unsupervised machine learning techniques. Data from the Dark Web is exposed to unsupervised machine learning methods like correlation-based learning and techniques for reducing dimensionality in order to reveal hidden patterns and correlations. The objective of unsupervised machine learning in network traffic classification is to recognise patterns and associations in the data and then group them accordingly. The reliability of the Dark Web data utilised for categorising network traffic, as well as the efficiency of the unsupervised machine learning algorithm utilised, are critical factors in achieving satisfactory outcomes. Therefore, it is important to carefully select the unsupervised machine learning approach and to pre-process the Dark Net data to ensure the best possible results.

Supervised machine learning approaches can be used to address this challenge by utilising a labelled dataset to train the algorithm to recognise different types of network traffic [32] and [36], as shown in Algorithm 4.The equation for this supervised machine learning approach can be represented as follows: y = f ( X )(3) In this case, X is the features or input data that characterise network traffic, and Y is the target or predicted class label of the traffic. The function f can be represented using a machine learning technique such as a D-Tree, ANN, or SVM. With the help of a series of mathematical operations, the algorithm is able to make predictions about the desired class label y based on the input data X. A more precise method of identifying the origin of darknet traffic is possible through the application of supervised machine learning techniques to the classification of network traffic. One equation commonly used in such approaches is the logistic regression equation, represented as: h ( x ) = 1 / ( 1 + exp ( - wx ) )(4) Where h(x) is the predicted probability of the input x being a certain class, w is the weight vector, and x is the feature vector. The features are considered based on network traffic patterns, and the weights are learned from a labelled dataset through training the logistic regression model. Supervised machine learning algorithms such as D-Tree, SVM, and neural networks can also be used for network traffic classification in darknets. The choice of algorithm would depend on the complexity and data size as well as the desired accuracy and computational efficiency of the classification system. The use of supervised machine learning techniques on the dark web can be a useful tool for traffic classification, which in turn can enhance network security and analysis.

Semi-supervised machine learning is a useful approach for network traffic classification, especially when labelled data is limited. In this approach, the classifier is trained on both labelled and unlabeled data to improve the accuracy of the classification, as shown in Algorithm 5.

The following equation describes the semi-supervised machine learning technique utilised in the development of network traffic identification in the Darknet. LossFunction ( L ) = ( 1 - λ ) * Cross - Entropy Loss ( CEL ) + λ * UnsupervisedLoss ( UL )(5) Where: L is the function of loss that was maximized during training of the network.λ is a weighting factor that determines the importance of the unsupervised loss component in the final result CEL is the cross-entropy loss component, which measures the discrepancy between expected and observed classifications. The autoencoder architecture has an unsupervised loss component called UL that measures the deviation between the original input and the reconstructed output. To better categorise traffic on the Darknet, a semi-supervised machine learning approach combines the best features of supervised and unsupervised learning methods. The supervised component uses labelled data to train the network to recognise patterns in the network traffic and classify it into different categories, while the unsupervised component uses the autoencoder architecture to learn the underlying patterns in the data without the need for labelled data [33, 37]. The final loss function takes into account both the supervised and unsupervised components, with the weighting factor determining the importance of each component in the final result. By utilising both labelled and unlabeled data, this method can greatly enhance the precision of network traffic categorization within the Dark Net. Let X be the dataset, Y be the labels, and h be the semi-supervised learning algorithm. The objective is to identify the optimal value of h that minimises the loss function L, represented by: L ( X , Y , h ) = 1 / n * ∑ i = 1 n Y i ≠ h ( X i )(6) Where n is the total number of samples in the dataset, Y _i is the true label for sample X _i , and h (X _i ) is the predicted label for sample X _i . The equation calculates the number of misclassifications made by the algorithm, and the goal is to minimize this value.

The use of deep learning approaches in network traffic classification has gained popularity in recent years, including for classifying traffic in the dark net [39]. This is due to the ability of deep learning algorithms to automatically learn complex and abstract features from large amounts of data, leading to improved accuracy in classification compared to traditional methods, as shown in Algorithm 6. One common equation used in deep learning is the feedforward neural network (FNN) equation: y = f ( W * x + b )(9) In the context of deep learning, the output (y) is generated by applying an activation function (e.g., sigmoid, ReLU) to the input (x), multiplying it by a weight matrix (W), and adding it to a bias (b). During training, the weights and biases are modified to minimise the loss function that quantifies the discrepancy between the predicted and actual outputs. To classify network traffic, several deep learning models have been utilised, such as CNNs, RNNs, and autoencoders. These models can be trained on labelled network traffic data to learn the patterns and features that differentiate different types of traffic. The final output of the model is a predicted class label for a given network flow or packet.

The collection of data sets is a crucial stage in the classification of network traffic. The quality and diversity of the data used to train and test the classifier will directly impact its accuracy and effectiveness. There are several ways to collect data for network traffic classification, including:

Capturing live traffic This involves collecting real-time traffic data as it flows through the network. This can be done using tools such as network taps, network probes, or packet capture software.Using public datasets: There are several public datasets available that contain network traffic data, such as the ISCX dataset, the CAIDA dataset, and the UNB Anomaly Detection dataset. These datasets can be used for research and experimentation.

Using synthetic data In some cases, it may be necessary to generate synthetic data that simulates different types of raffic. This can be useful for testing and evaluating the performance of the classifier.

Labelling the data Once the data has been collected, it must be labelled with the correct traffic type. This can be done manually or through the use of automated labelling tools.It is important to ensure that the data used for network traffic classification is diverse and representative of the types of traffic that will be encountered in the real world. This will help to ensure that the classifier is able to accurately classify a wide range of traffic types. There are several sources where you can collect datasets for network traffic classification, as shown in Table 2.

The comparative evaluation of various machine and deep learning techniques for Darknet traffic classification revealed several limitations based on the performance analysis shown in Figure 2.

OPEN IN VIEWER

Fig. 2

Performance metrics utilized in the analysis of network traffic.

In conclusion, the comparative evaluation of Darknet traffic classification techniques revealed several statistical limitations related to the accuracy, simplicity of execution, real-time implementation, unknown traffic identification, and time and space complexity. These limitations must be carefully considered when selecting an appropriate technique for real-world applications.

Network traffic classification in the dark net is a challenging task due to the anonymous nature of the communication and the use of encryption technologies. The following are some recent open research challenges for the ascent of network traffic classification in the dark net:

Encrypted traffic classification Encrypted traffic poses a significant challenge for network traffic classification in the dark net. Developing accurate classification methods that can identify the type of encryption used and the purpose of the communication is a critical research challenge.

Multi-protocol classification The use of multiple protocols in the dark net makes it difficult to classify network traffic. Developing techniques that can handle multiple protocols simultaneously is a significant research challenge.

Dynamic traffic classification The traffic patterns in the dark net are continuously changing, making it challenging to develop a static classification method. Developing techniques that can adapt to the changing traffic patterns is an open research challenge.

Anomaly detection Dark net traffic often involves malicious activity, making it important to develop methods that can detect anomalous traffic patterns. Developing effective anomaly detection techniques is a critical research challenge.

Large-scale traffic classificationThe amount of traffic in the dark net is enormous, making it challenging to handle and classify. Developing techniques that can handle large-scale traffic classification is an open research challenge.

Adversarial attacks Attackers can try to evade traffic classification methods by manipulating the traffic patterns. Developing methods that can detect and mitigate adversarial attacks is an open research challenge.