An Empirical Study of Big Data Classification Methods and Challenges

1. Introduction

In recent years, for the development of digital technology, enormous data has been required from various fields such as healthcare, Internet of Things platforms, social media platforms, and so on. These widespread usages of digital technology have demanded the growth of big data. Big data refers to the integration of vast and complex datasets that existing database management tools struggle to handle effectively. A classical database is a traditional relational database that is designed to store and manage structured data with a fixed scheme. Big data involves large, expanding datasets that are complex and sourced from multiple autonomous origins. The key difference lies in architecture and capability: classical databases rely on vertical scaling, fixed schemas, and batch processing, making them rigid and slow for modern, diverse data. In contrast, big data leverages distributed, horizontally scaled architectures to handle massive amounts of unstructured, semistructured, and structured data, enabling near real-time processing and dynamic analysis. Essentially, classical systems are for organized, static data, while big data platforms are for fluid, vast datasets generated at high speed from sources such as social media and sensors. The primary difference between big data and classical data lies in the methods, procedures, objectives, and strategies utilized to extract valuable insights from the information. Previous technologies were unable to manage the storage and processing of such enormous data, leading to the development of the big data concept. Big data describes large datasets that cannot be effectively managed or handled by classical database systems. (Banchhor & Srinivasu, 2021; Benabderrahmane et al., 2017) and quite challenging due to the expeditious gathering of data, storage, and other related techniques (Raghav et al., 2017; Thanekar et al., 2016). Big data has five key characteristics: volume, velocity, variety, value, and veracity. The big data generally utilizes two traditional approaches, namely classification (Manogaran et al., 2018) and clustering (Yildirim, 2019), which are relevant to data mining techniques. Here, the classification approach belongs to the supervised learning approach, whereas clustering is an unsupervised approach that requires no pretraining strategy for cluster performance. The classification is defined as grouping the data according to the respective similarities and differences from the gathered data, and providing data tagging. Clustering is a technique that groups data points based on their similarities, forming clusters without any prior labels or predefined knowledge of the groupings. This makes it possible for better identification and segregation of data, and also makes it easy to find and locate the related data. Big data classification is strongly admired in real-time applications such as healthcare management, epidemic outbreak prediction, and so on (Khanna et al., 2021).

Specifically, big data analytics, particularly in sectors such as healthcare and the stock market, were quite complex due to the large volumes of data and the high computational costs required. The challenges faced in the field of big data can be solved efficiently using a classification approach. The classification approach proceeded with analysis strategy, processing techniques, and storage (Suthaharan, 2012). There were multiple possible techniques used for classification, which initially worked by extracting huge amounts of information from a large set of data (Koturwar et al., 2015). However, two major tools, such as Apache Spark and MapReduce techniques, are used for the classification. Apache Spark refers to a kind of tool used for security analysis in big data (Brahmane & Krishna, 2021; Lighari & Hussain, 2017). It is also known for speedy cluster computing that is efficiently managed for high accuracy. On the other hand, MapReduce is also a technique, embedded with a mapper and reducer to break down the large or complex data into a simple form for classification (Bhukya & Gyani, 2015). This technique utilizes a Hadoop file system that reads and writes to the disk, and also has shortcomings due to inefficiency and redundancy (Elsebakhi et al., 2015). These two efficient methods are the key tools to support the traditional particle swarm optimization technique and genetic algorithm (Ditzler et al., 2017).

Machine learning (ML) techniques have been used in big data classification (Wang & Alexander, 2016). There were several classification techniques based on ML to classify the big data, namely, K-nearest neighbors (KNNs; Saadatfar et al., 2020), Naïve Bayes (Shah et al., 2020) algorithm, decision tree (DT; Shaik et al., 2023), Hadoop MapReduce (Nagesh & Prabhu, 2017), and random forest (RF; Manikandan, 2018). It also required supervised and unsupervised techniques such as a DT and a support vector machine (SVM; Huang et al., 2014). In addition, several optimization-based approaches were developed to solve big data classification. The DT is a hierarchical model adopted with nodes and leaves, which highly divides the input space into class regions. However, KNN is another technique that was adopted for finding the nearest samples and selecting them with the respective traditional methods. Moreover, the hybrid model of SVM–KNN is utilized to compute the query distances, which are comparatively slower than other traditional approaches. In big data, the computation process was much more complex, so fuzzy classification was developed. The high usage of the fuzzy technique results from weak identification and increases computational expenses, which is the major drawback of this method (Manikandan, 2018). The development of neural networks has led to the rapid growth of deep learning (DL) techniques such as convolutional neural networks (CNNs), Deep CNN (DCNN), long short-term memory (LSTM), and so on. DL offers the capability to tackle data analysis and learning challenges posed by a large amount of input data. Notably, it excels at automatically extracting complex data representations from extensive amounts of unsupervised data. As data volumes increase and advancements in graphics processors and processing power continue, DL is becoming more crucial for delivering big data predictive analytics solutions, necessitating further investigation.

The main objective of this study is not to provide a comprehensive overview of all the relevant work in ML and DL, but rather to highlight current research endeavors, the difficulties associated with big data, and key issues about learning from vast amounts of data, as well as future trends. Specifically, this review analyzes 55 research articles relevant to big data classification utilizing advanced strategies and algorithms, exploring their advantages and disadvantages to formulate advanced proposals for effective big data classification in the future. This systematic review contributes to exploring the potentials and difficulties of big data classification to formulate an efficient contribution. Furthermore, the review analyzes the advanced methods of big data classification, limitations, datasets used, and performance metrics contemplated for big data classification. This review is initiated by offering the background of multiple big data classification models and provides an overview of the technical details of several mechanisms involved in the models, which paves the way for improving an effective model for future research. Moreover, the review underscores that the DL models provide exceptional performance compared to other ML classifiers.

The research article is categorized in the following structure: The classification of multiple methods of big data is categorized in Section 2, analysis and discussion of various big data classification techniques are presented in Section 3, Section 4 exposes research gaps and future works, and lastly, the conclusion is in Section 5.

1.4 Overview of Big Data Classification

Categorizing the big data problems based on the type makes it easier to see the characteristics of each kind of data. These characteristics provide insight into the data's acquisition process, processing the data into the appropriate format, and the frequency with which new data becomes available. Figure 1 depicts the architecture of big data classification.

OPEN IN VIEWER

Figure 1.

Architecture of big data classification.

Analysis Type: In the context of big data, analysis type indicates whether the data is batch-processed for subsequent analysis or examined in real time. Selecting the analysis type should be meticulously carried out because it influences a number of other choices regarding technology, tools, products, data sources, and expected data frequency.

Processing Methodology: The kind of data processing approach used for further applications, such as predictive, analytical, ad-hoc query, and reporting. Proper selection of processing methodology aids in determining whether the tools and approaches are suitable for use in the big data solution.

Data Frequency and Size: Understanding the size and frequency of data aids in selecting the storage mechanism, storage format, and required preprocessing tools when frequency and size are known. Data frequency and size depend on the source of the data.

Data Type: Types of data that need to be processed, including master data, transactional data, and historical data. Identifying the data in storage is made easier by understanding the data type.

Content Format: This involves the format of the incoming data, such as semistructured, unstructured, such as audio, video, and pictures, or structured, such as Relational Database Management System (RDMBS). Format dictates the necessary processing of incoming data and is essential for selecting the tools and methods, as well as for establishing a business-oriented solution.

Data Source: Provides the information of data sources collected from social media and the web, machine-generated, and human-generated.

Data Consumers: A list of every potential user of the data that has been processed:

Hardware: Indicates the type of hardware on which the big data solution gets executed, and can be commodity hardware or state-of-the-art. Knowing the hardware's constraints aids in selecting a big data solution.

This section will detail the methodology, emphasizing the systematic approach used in article selection. It will also include an extensive literature review that provides an overview of big data classification. Figure 2 outlines the stages followed in the PRISMA-based review.

OPEN IN VIEWER

Figure 2.

Flow diagram of a systematic approach.

The proposed work conducts a systematic literature review adhering to PRISMA guidelines to provide a comprehensive review. Initially, a detailed search filters articles from data sources. To include the most recent works in the field of big data classification, only articles from 2020 to 2024 were considered. The literature search was conducted across several electronic databases, including IEEE, MDPI, Springer, Elsevier, Research Gate, Wiley, and others. The research strategy was refined iteratively to ensure broad coverage of relevant studies while excluding irrelevant or duplicate records. The search includes keywords such as “Big data,” “Classification of big data,” “big data medical,” and “big data analytics” for finding the relevant research articles. After selecting the articles, each one was evaluated for relevance to the research topic; papers with irrelevant titles, abstracts, contributions, papers of poor quality, and insufficient analysis were excluded based on eligibility criteria. Finally, 55 papers were reviewed for an in-depth analysis. The PRISMA flow diagram visually represents the number of articles selected, identified, and excluded, along with the reasons for exclusion. This systematic approach enhances the validity and reliability of the proposed review.

2.1 Preprocessing Techniques

Figure 3 illustrates the preprocessing techniques for big data classification, which is also an initial step for all DL and ML techniques, where the unwanted background from the image datasets is removed to enhance the quality of an effective classification process. Several preprocessing techniques are handled by various approaches, such as feed-forward neural network (Ashraf et al., 2020), K-means algorithm (Manikandan, 2018), MapReduce (Banchhor & Srinivasu, 2021; Game et al., 2022; Lakshmanaprabu et al., 2019; Narayana et al., 2022; Sujitha & Seenivasagam, 2021), linguistic hedges neuro-fuzzy classifier with selected features (LHNFCSF; Azar & Hassanien, 2015), maximum Likelihood (Boulila et al., 2021), bottom-hat filter (Naronglerdrit & Mporas, 2021), manifold analysis and nearest neighbor propagation (Jiang & Li, 2019), and locality sensitive hashing–synthetic minority oversampling technique (Hassib et al., 2020). Most reviewed papers utilized MapReduce techniques to reduce the backlogs from the datasets. Furthermore, the above-mentioned techniques are highly reliable and more viable, which reduces the time duration for computation and also earns a high classification performance of the model.

OPEN IN VIEWER

Figure 3.

Preprocessing techniques for big data classification.

2.2 Feature Extraction Techniques

Feature extraction means extracting the key features from the given data and works by reducing the high-dimensional parameters. Due to this feature extraction, the model can be highly trained for better classification and improve the computational speed effectively. In other words, raw data is transformed into numerical formats that are highly compatible with both ML and DL methods. This approach is an important and mostly used method for effective classification, some of the methods are collected from recent articles such as the GoogleNet method (Ashraf et al., 2020), Spark framework (Tchito Tchapga et al., 2021), gray level difference matrix (GLDM; Almutairi et al., 2024), SVM (Ashraf et al., 2020; Zhu & Chen, 2020), Spark-Standalone mode (Xing & Li, 2019), principal component analysis (PCA; Wu, 2022), and DT (Rawal & Agarwal, 2019) are shown in Figure 4. These techniques are deployed for accurate classification and provide superior performance.

OPEN IN VIEWER

Figure 4.

Feature extraction and selection techniques for big data classification.

2.2.1 GoogleNet Method

The GoogleNet method, a robust DL architecture specifically a CNN, is employed for feature extraction in big data classification. It excels in efficiently extracting complex and highly discriminative features from large datasets.

2.2.2 Spark Framework

The Spark framework prepares data for iteration, allows for repeated querying, and loads it into memory. The main program manages multiple workers and collects their results. Workers read data partitions from a distributed file system, perform computations, and save the results to disk.

2.2.3 GLDM

A GLDM is employed in feature extraction for big data classification, especially for image data. It captures textural information within an image by analyzing the frequency of different gray level combinations between neighboring pixels, offering valuable features for distinguishing between various data classes.

2.2.4 SVM

An SVM aids in feature extraction for big data classification by pinpointing the most relevant features in high-dimensional datasets. This reduces the feature space while ensuring high classification accuracy, particularly in handling complex, nonlinear relationships between features and class labels.

2.2.5 Spark-Standalone Mode

The Spark-Standalone mode allows for cluster launching either manually or via a launch script. In its application master/worker management mode, the master primarily handles resource management and scheduling. However, in Spark-Standalone, this master/worker mode is vulnerable to single-point failure issues.

2.2.6 PCA

The PCA produces new variables that are linear combinations of the original variables. It can extract the most informative features from large datasets while retaining the most relevant information from the original dataset.

2.3 Feature Selection Techniques

This crucial technique for ML and DL approaches involves selecting extracted data by reducing irrelevant parameters, thereby enhancing the model's learning capability while minimizing computational expenses. The analysis of new articles utilized a library for SVMs (Lin et al., 2016), Whale optimization algorithm (WOA; Hassib et al., 2020), Pearson correlation-based Black hole entropy (BHE) fuzzy clustering (Narayana et al., 2022), and improved Dragonfly algorithm (IDA; Lakshmanaprabu et al., 2019), which are shown in Figure 5. The feature selection methods reduce the complexity of the model for effective classification.

OPEN IN VIEWER

Figure 5.

Optimization techniques for big data classification.

2.3.1 WOA

WOA is employed for feature selection by treating each feature subset as a whale's position. Each subset contains a random number of features, up to the total number of features in the original dataset. The whale representing the subset with the fewest features and highest classification accuracy is deemed the best solution.

2.3.2 Pearson Correlation-Based BHE Fuzzy Clustering

The Pearson correlation-based BHE fuzzy clustering is used as the feature selection technique in the existing method. BHE fuzzy clustering generates samples using the fuzzy membership function and performs feature selection by integrating three key aspects: the Bayesian inference model, fuzzy clustering, and BHE-based information. This method is particularly effective for large datasets due to its integration of fuzzy clustering with BHE. Additionally, the Pearson correlation distance is used to characterize clustering behavior.

2.3.3 IDA

In the conventional method, the IDA is used for the feature selection process. The IDA depends on both static and dynamic swarming capabilities, such as partitioning, formation, cohesion, attraction to food sources, and evasion of predators. These swarming behaviors closely mirror the two main phases of optimization using metaheuristics.

2.4 Review of Big Data Classification Models

Big data classification techniques are one of the efficient strategies for analyzing, processing, storing, and classifying the data accordingly for easy locating and aggregation without complexity. The reviewed papers have huge techniques that effectively solve the problem. Moreover, the ML and the DL are the prominent techniques used in many classification tasks as well as other complex problems to make simple and unique, and these techniques are considered for high accuracy performance. In this paper, the techniques, methods for processing, and other strategies for classifying big data are emphasized to forge multiple models in the future. The taxonomy of different classification techniques is shown in Figure 6. Table 1 presents the abbreviations of the model included in Figure 6.

OPEN IN VIEWER

Figure 6.

Technique used in big data classification.

OPEN IN VIEWER

Table 1.

Abbreviations.

DCM	Deep computation model
CNN	Convolutional neural network
CNN–LSTM	Convolutional neural network with long short-term memory
D- CNN	Distributed convolutional neural network
SILDDNE	Structure-informed locally distributed deep nonlinear embedding
MR-RBM-FI	Restricted Boltzmann machine with Hadoop MapReduce and fuzzy integral
RNN	Recurrent neural network
BRNN	Bidirectional recurrent neural network
Deep RNN	Deep recurrent neural network
SVM	Support vector machine
DT	Decision tree
RF	Random forest
KNN	K-nearest neighbor
C4.5 DT	C4.5 decision tree
DTCAA	Decision tree credit assessment approach
DHBoost	Distributed heterogeneous boosting-inspired ensemble classifier
Xception-LSTM	Extreme inception and long short-term memory
Light GBM	Light gradient boosting machine
LMNN	Linear-morphological neural network
CNB	Correlative Naïve Bayes
CGCNB-MRM	Cuckoo–Grey wolf-based Correlative Naive Bayes classifier with MapReduce model
MLNN	Morphological-linear neural network
KSC	K-means spectral clustering

2.4.1 ML Techniques for Big Data Classification

In big data classification, ML techniques are the most prominently used methods inspected in recent papers, which are highly required for classification, and the survey is provided in the context below.

2.4.1.1 DT Method

Game et al. (2022) introduced the DT model for big data classification, which was ideally done using a divergence-based grey wolf optimization (DGWO) algorithm. The DGWO algorithm selects the major attributes for classification, minimizes the class mixture, and makes the determination of class easier, hence providing better performance. However, this model does not provide high accuracy and also requires maximum computational expenses for the complete procedure. Chern et al. (2021) developed a hybrid method to solve the big data problems in credit assessment and achieve better results compared to the artificial neural network (ANN) model, also lowering the computational complexity of classifying the big data. However, the model requires more time and drives complications during the classification process, which includes the model's intended purpose to minimize the complexity of the attributes. Moreover, the attributes are not enough for the data mining model, in addition suffer from low sensitivity. Rawal and Agarwal (2019) deployed C4.5 DT-based classification for big data, which involves calculating the weight parameters between several nodes. This mechanism was also represented as a classifier of statistics that brought efficient classification with high accuracy. Moreover, the minority range of this model was specifically designed for measuring some kind of impurities.

2.4.1.2 Gradient Boosting Method

Kadkhodaei et al. (2021) utilized an ensemble method, a distributed heterogeneous boosting-inspired ensemble classifier (DHBoost), where the boosting approach enhanced the automatic selection for proper learning procedures and provided increased classification accuracy. Also, this results in maximum computational expenses and is also sensitive to unwanted noises. Ait Hammou et al. (2019) also initialized with the boosting model, extreme gradient boosting (XGBoost), to classify the early and late stages of cancers. This method has scalable benefits to handle large datasets and provides efficient computation. Despite the method being slow for the training process and vulnerable to overfitting problems. Hussein and Abdulkader (2022) developed a Light Gradient Boosting Machine (Light GBM; Abbasniya et al., 2022; Asif et al., 2023) classification model deployed with VGG 19 DCNN. The Light GBM model has several benefits of scalability and reliability, and provides significant results compared to DT. Despite that, the model has overfitting issues and provides computational complexity. In addition, the selection of leaves during tree generation led to the majority loss of important data.

2.4.1.3 SVM Method

Tchito Tchapga et al. (2021) utilized the ML model, SVM, for finding the optimal hyperplane parameter, which aids in error minimization during classification, and also utilized the CNN approach for classification that highly extracts the high-level representations belonging to the big data and provides approximate results. Despite that, the SVM model only provides better performance using medium or small datasets, but not large datasets. Sujitha and Seenivasagam (2021) also developed an SVM model with different algorithms, binary classification, and multiclass classification with threshold technique (T-BMSVM) for classification. The combination of SVM and T-BMSVM outperforms, with promising performance, high scalability, and provides maximum convergence for classification. Despite this, the model was not implemented using a large number of datasets, which consumes more time and cost.

2.4.1.4 Complement Naïve Bayes Method

Banchhor and Srinivasu (2021) utilized a correlative Naïve Bayes (CNB) classifier with cuckoo search and grey wolf optimization technique for big data classification, which has achieved comparatively high results with other CNB-based classifiers. The model provides quite high accuracy as well as reliability for better outcomes. However, this approach was highly expensive during computation. Banchhor and Srinivasu (2020) deployed the Cuckoo–Grey wolf-based Correlative Naive Bayes classifier with MapReduce model (CGCNB-MRM), which highly selects an important parameter with the help of an optimizer and affords classification with high speed and fewer computation costs. Despite the performance rate being improved, the output was always affected based on the quality of the datasets. This limitation can provide low accuracy and decreased efficiency of the model.

2.4.1.5 Other ML Techniques

Xing and Bei (2019) introduced a class-based weighted KNN model to develop a classification model; this method significantly reduced the computational time and was also proficiently performed using a large number of datasets. The improved KNN model only focused on a single-class classification, which decreases the sample sparsity of the category description. Manikandan (2018) utilized an RF (Lakshmanaprabu et al., 2019) classifier that generates test error in the forest building process, as well as an internal unbiased estimation, which was highly capable of dealing with the classification process. Using this RF model, high computation costs and lower convergence speed for classification were required. Liang et al. (2018) utilized the extreme inception and LSTM (Xception-LSTM) model, which tends to extract spatial–temporal features that are intended for segmentation and classification. However, the Xception-LSTM model required high computational expenses, and the problem can be solved using computer hardware. Azar and Hassanien (2015) developed an LHNFCSF for preprocessing, which deliberately reduced the dimensionality, discarded redundant noise, and achieved high-speed classification. Somehow, the results of the LHNFCSF model are not completely interpretable and are not implemented in complex problems. Hernández et al. (2020) developed linear-morphological neural network (LMNN) and morphological-linear neural network (MLNN). Especially, the morphological neurons in the model possibly generate great responses that provide great initial capacity for big data classification. Moreover, the deployed model was not able to solve the overfitting problems due to the two high-dimensional datasets. Xing and Li (2019) developed a hybrid model of k-means spectral clustering (KSC) for big data classification that achieved clustering by merging the merits of KSC, which decreased the computational complexity as well as minimized the hyperspectral dimensionality of the data. Sometimes, the k-means became sensitive to the initial condition, and however hard to determine the data for classification. Jiang and Li (2019) deployed a radial basis function (RBF) neural network that performs global as well as local approximation performances and also increases the accuracy for classification. However, the RBF neural network consumes more time and is limited to complex data.

2.4.2 DL Techniques for Big Data Classification

In this section, the DL methods are described for big data classification with the upper hand techniques and their limitations from recent articles.

2.4.2.1 CNN Methods

Boulila et al. (2021) distributed CNNs (D-CNNs) as a type of CNN model, which utilizes an effective maximum likelihood supervised technique to efficiently classify big data and provide high accuracy. The huge amount of data led to the development of the curse of dimensionality problem, resulting in complex classification. Naronglerdrit and Mporas (2021) employed the most common DL method, a CNN (Almutairi et al., 2024; Wu, 2022), in which the model is highly preprocessed using a bottom-hat approach that enhances the image dataset to ensure effective classification. Also, the segmented process was capable of segmentation and also achieving high accuracy for classification performance. Sometimes, the CNN model was not reliable due to large memory footprints.

2.4.2.2 DCNN Methods

Ashraf et al. (2020) used the DCNN technique that participates in the big data classification problem. In this model, image-based datasets are taken; in addition, two phases, namely the feature extraction phase and the classification phase, were occupied in this process, which results in accurate performance with the help of reliable tools as well as techniques, such as benefits for high classification. Even though the model provides accurate classification, it was not explored in a large-scale image-based dataset, which was the only limitation. Jaya Sudha and Sneha (2022) also introduced a DCNN-based model; here, the neurons are considered as the basic building block to perform subsequent classification. In this approach, the classification task was effectively done using interclass and intraclass, which automatically allocates magnetic resonance imaging, ultrasound, X-rays, and other major components for effective classification. However, the privacy exploitation activity was quite improper and not a reliable model.

2.4.2.3 CNN With LSTM

Salehin et al. (2023) established a hybrid model, combining CNN with LSTM (CNN–LSTM; Rai & Chatterjee, 2022; Shahzadi et al., 2018), using real-time datasets. This technique is efficient in handling multiple sequential data and, more importantly, works in time series data and classifies effectively. However, the CNN–LSTM model was not efficient enough to handle highly complex problems, and hence, effective classification was not determined. Rai et al. (2020) also developed a CNN–LSTM model for automatic prediction problems and cardiac arrhythmias using big datasets. This model was quite efficient in predicting the disease as well as producing better accuracy performance by solving gradient issues. However, the hybrid CNN–LSTM model was inefficient in solving imbalance issues and required high computational expense during prediction.

2.4.2.4 Recurrent Neural Network (RNN) Methods

Hassib et al. (2020) developed a model for big data classification, bidirectional RNNs (BRNNs) that showed many processes for classification and utilized an optimization algorithm, WOA, for accurate selection of features that aid better results. Despite that, the WOA–BRNN model requires more time for computation while using large datasets. Narayana et al. (2022) effectively handle the classification process using a deep RNN (Deep RNN) model. The Ant Cat Swarm Optimization (ACSO) algorithm was implemented in the utilized model, which highly identifies the weight parameters and effectively tunes the model for enhanced classification, also achieving better results. In addition, this model was quite reliable and scalable because of adopting the advantages of the ACSO algorithm, which includes tolerance of faults during classification and deals well with complex patterns. However, the model provides less computational speed at a maximum cost.

2.4.2.5 Other DL Techniques for Big Data Classification

Li et al. (2017) deployed a deep computation model (DCM) that highly classified the big data for uncomplicated processing and storing the data. The utilized DCM model achieved increased performance with the help of a tensor representation, which makes it easy for classification. Moreover, the DCM took more time than the traditional CNN model due to more parameters in the tensor space. Zhu and Chen (2020) introduced the structure-informed locally distributed deep nonlinear embedding (SILDDNE) method, in which the structural and attribute characteristics of each node were taken for improved classification. The SILDDNE method directly utilizes a deep neural network to reduce several parameters and improve computational efficiency. The nodes in the network were connected only with a minimum of edges, which made it quite difficult to learn a vector representation. Brahmane and Krishna (2021) utilized a deep-stacked auto-encoder for classification, in which the model was trained using the Rider Chaotic Biography streamlining (RCBO) algorithm that effectively trained the model and increased the computational speed for classification. Although the model does not deal with large datasets, the computational expense was very high. Zhai et al. (2019) developed a restricted Boltzmann machine with a Hadoop MapReduce and fuzzy integral (MR-RBM-FI) model that involved an eligible MapReduce procedure for effective classification and achieved improved accuracy. However, the RBM model was quite challenging for the energy gradient function and increased computation time.

2.4.3 Federated Learning and Transfer Learning Methods

Liu et al. (2023) proposed a transfer learning-based classifier for big data classification, which deals with both static and real-time data effectively, thus increasing the overall classification. However, the model is computationally expensive while handling larger datasets. Kaleem et al. (2023) introduced a federated averaging strategy for the classification of big data, which effectively addresses challenges in real-world environment settings and data integration, thus demonstrating an improved performance in classification. However, the model requires massive computational resources and time to train the model. Dhiman et al. (2022) presented a federated learning approach to supervise the privacy preservation of large medical and health data. Finally, due to the general privacy protection problems that occur throughout the medical big data life cycle throughout the industry, appropriate options were developed at the management level.

2.4.4 Ensemble Methods

Demidova et al. (2016) proposed a two-level SVM classifier for big data classification, which incorporates particle swarm optimization, which reduces the processing time of the SVM classifier, which plays a major role in addressing the challenges in handling larger data. However, these models can be computationally intensive and complex to tune, and potentially suffer from less interpretability and sensitivity to initial conditions. Ponmalar and Dhanakoti (2022) proposed an ensemble SVM model for intrusion detection in a big data platform. It integrates the Chaos Game optimization algorithm and the ensemble SVM, thus identifying different types and achieving higher accuracy. However, it high computational cost and memory requirements that impact scalability for large-scale real-time data processing. Ramachandran and Manikandan (2021) proposed an ensemble classification algorithm for medical big data processing. It includes SVM and RNN classifiers for the medical data classification, precisely processes the big data, and results in a significant performance than a single classifier. However, the model may face issues such as overfitting and generalizability issues when handling new datasets. Figure 7 displays the performance ranking of the different classification models in terms of metric accuracy for big data classification.

OPEN IN VIEWER

Figure 7.

Performance ranking chart.

2.4.5 Optimization Techniques

In general, optimizers are employed to tune parameters for model training and enhance the computational efficiency for specific tasks. Recent articles indicate that various optimization algorithms are utilized for multiple processes, such as the selection and extraction processes, and fine-tuning the model for improved classification. The utilized optimizers from the review of recent papers such as ACSO (Narayana et al., 2022), Cuckoo–Grey wolf-based Optimization (CGWO; Banchhor & Srinivasu, 2020, 2021), WOA (Hassib et al., 2020), RCBO optimization, improved cat swarm optimization (Lin et al., 2016), genetic algorithm and gradient approximation (Almutairi et al., 2024), DGWO (Game et al., 2022), vulture optimization algorithm (VOA) and Haris Hawks optimization (HHO) are shown in Figure 5.

2.4.5.1 ACSO

The ACSO is the integration of Ant Lion Optimizer (ALO) and Cat Swarm Optimization (CSO). The ALO algorithm simulates the foraging behavior of ant lion larvae, mimicking the interactions between ant lions and ants. The CSO is a swarm intelligence-based optimization method inspired by the behavior of cats, and it is employed to address various optimization problems.

2.4.5.2 CGWO

The CGWO algorithm is an enhancement of the Grey wolf-based optimization (GWO), which incorporates a population-based algorithm, CS. The GWO harnesses the hunting behavior of grey wolves, including chasing, encircling, and attacking prey characteristics. In the CGWO algorithm, the position update mechanism of the GWO is modified by integrating the update equation of the CS, resulting in faster convergence of the CGWO algorithm.

2.4.5.3 WOA

Whales are regarded as some of the smartest animals due to the presence of spindle cells in their brains. These cells facilitate judgment, emotions, and social behaviors similar to those found in humans.

2.4.5.4 DGWO

The GWO algorithm is instrumental in solving various engineering problems. However, it has limitations, such as its performance in finding the global optimal solution, which affects its convergence rate. To overcome the efficiency, exploration, and exploitation limitations of the conventional GWO algorithm, a novel modified approach known as DGWO has been developed.

In this section, Table 2 provides the comparison of ML techniques for Big Data classification. Table 3 presents the comparison of DL techniques for Big Data classification. Table 4 depicts the comparison of federated learning and transfer learning techniques for Big Data classification. Table 5 displays the comparison of ensemble learning techniques for Big Data classification.

OPEN IN VIEWER

Table 2.

Comparison of Machine Learning Techniques for Big Data Classification.

Sl.No	References	Methods	Advantage	Disadvantage	Achievement
1.	Game et al. (2022)	DT	The classification is optimally done with the optimized DT model, therefore providing better performance.	This model does not provide high accuracy and requires more computational expenses.	Achieved accuracy of 90.2%, precision of 94.5%, specificity of 94.8%, sensitivity of 50.4%, and F1-score of 94.4%.
2.	Chern et al. (2021)	DTCAA	The model comparatively performs better than the ANN model and reduces the complexity of big data.	The developed model required more time for classification.	—
3.	Rawal and Agarwal (2019)	C4.5 DT	The Bhattacharya distance is used for reducing the input space and boosting the accuracy.	The time required to construct the DT model is high.	—
4.	Kadkhodaei et al. (2021)	DHBoost	Categorizes the test instances in a short time.	However, the model increased the amount of main memory.	Accuracy of 96.73%.
5.	Ait Hammou et al. (2019)	XGBoost	The model helps in reducing the dimensionality of the data.	The model suffers from sparsity problems.	—
6.	Hussein and Abdulkader (2022)	VGG19 and Light GBM	The expanded dataset size enabled our model to extract additional features.	The model suffers from overfitting issues.	Achieved an accuracy of 97.33%.
7.	Abbasniya et al. (2022)	IRv2-CXL	Using the pretrained model for feature extraction helps in minimizing the training time.	The model is having issues due to the imbalanced data.	Accuracy of 96.46% and F1-score of 97.44%.
8.	Asif et al. (2023)	Res-Inc-LGBM	Highly efficient for the identification and subtype classification of Intracranial Hemorrhage (ICH).	The model is less effective for large data.	Accuracy of 97.7%, Sensitivity of 96.5%, and F1-score of 97.4%.
9.	Tchito Tchapga et al. (2021)	SVM and CNN	The DL model that is CNN provides better performance in using large-scale datasets and is used to perform effective simplification.	Here, the SVM model only provides better performance using the medium dataset, but not the large-scale dataset.	Achieved very good accuracy.
10.	Sujitha and Seenivasagam (2021)	SVM	The T-BMSVM model is incorporated with multiclass classifiers, which provide high classification with high accuracy.	This model is not implemented using a large number of datasets, which consumes more time and cost.	Achieved an accuracy of 86%.
11.	Banchhor and Srinivasu (2021)	CNB classifier	The model attains a better convergence rate and reduced computational complexity due to the utilization of Bayesian optimization.	The model obtains the effective classification, but it does not apply to noisy data.	Accuracy of 82.2%, sensitivity of 84.1%, and specificity of 75.1%.
12.	Banchhor and Srinivasu (2020)	CG-CNB classifier	The developed CGCNB-MRM highly selects an important parameter for classification with high speed and less computation costs.	However, the performance rate was improved, and the output was affected based on the datasets.	Achieved accuracy of 80.7%, sensitivity of 84.5%, and specificity of 79.9%.
13.	Xing and Bei (2019)	Class-based weighted KNN	The improved KNN algorithm reduces computational time and can be performed using large datasets.	The improved KNN model only focuses on single-class classification, which decreases the sample sparsity of the category description.	—
14.	Manikandan, 2018)	RF	The RF classifier generates test errors in the forest building process, an internal unbiased estimation that was highly capable of dealing with the classification process.	This model required a high computation cost and less convergence speed for classification.	Achieved an accuracy of 99.45% and a time of 7.6 min.
15.	Lakshmanaprabu et al. (2019)	RF	The model outperformed better than the Gaussian mixture model and the logistic regression problems, also providing high classification.	The developed model required a high computational cost, and the computational process was slow.	Achieved precision of 94.2%, recall of 89.99%, F1-score of 89.22%, sensitivity of 79.2%, and specificity of 92.14%.
16.	Liang et al. (2018)	Xception-LSTM model	This model extracts the spatial–temporal features, which are highly segmented and classified with high accuracy.	The model required computational expenses, and the problem can be solved by computer hardware.	Accuracy of 90.79%.
17.	Azar and Hassanien (2015)	Neural fuzzy	The LHNFCSF model reduces the dimensionality, discards the redundant noise, and achieves high-speed classification.	The model results are not completely interpretable and are not implemented in complex problems.	Achieved 97.67% accuracy and 7.45 RMSE.
18.	Hernández et al. (2020)	MLNN and LMNN	The morphological neurons in the model possibly generate great responses, which provide a high capacity for classification.	The model cannot solve the overfitting problems due to the two high-dimensional datasets.	Achieved 99.05% of accuracy.
19.	Xing and Li (2019)	KSC	The Minimum Noise Fraction (MNF) method is employed to decrease the dimensionality of hyperspectral data.	Despite its significant advantages in many clustering techniques, the spectral clustering method is not suitable for hyperspectral image clustering.	Achieved 94.68% of accuracy and 0.91 of the kappa coefficient.
20.	Jiang and Li (2019)	RBF neural network	The base width is updated adaptively with the optimization of the cluster center.	The model consumed more time and was limited to complex data.	Achieved more than 85% of accuracy.
21.	Ait Hammou et al. (2019)	XGBoost	The model helps in reducing the dimensionality of the data.	The model suffers from sparsity problems.	—
22.	Hussein and Abdulkader (2022)	VGG19 and Light GBM	The expanded dataset size enabled our model to extract additional features.	The model suffers from overfitting issues.	Achieved an accuracy of 97.33%.
23.	Abbasniya et al. (2022)	IRv2-CXL	Using the pretrained model for feature extraction helps in minimizing the training time.	The model is having issues due to the imbalanced data.	Accuracy of 96.46% and F1-score of 97.44%.
24.	Tchito Tchapga et al. (2021)	SVM and CNN	The DL model that is CNN provides better performance in using large-scale datasets and is used to perform effective simplification.	Here, the SVM model only provides better performance using the medium dataset, but not the large-scale dataset.	Achieved very good accuracy.
25.	Sujitha and Seenivasagam (2021)	SVM	The T-BMSVM model is incorporated with multiclass classifiers, which provide high classification with high accuracy.	This model is not implemented using a large number of datasets, which consumes more time and cost.	Achieved an accuracy of 86%.
26.	Banchhor and Srinivasu (2020)	CG-CNB classifier	The developed CGCNB-MRM highly selects an important parameter for classification with high speed and less computation costs.	However, the performance rate was improved, and the output was affected based on the datasets.	Achieved accuracy of 80.7%, sensitivity of 84.5%, and specificity of 79.9%.
27.	Xing and Bei (2019)	Class-based weighted KNN	The improved KNN algorithm reduces computational time and can be performed using large datasets.	The improved KNN model only focuses on single-class classification, which decreases the sample sparsity of the category description.	—
28.	Manikandan, 2018)	RF	The RF classifier generates test errors in the forest building process, an internal unbiased estimation that was highly capable of dealing with the classification process.	This model required a high computation cost and less convergence speed for classification.	Achieved an accuracy of 99.45% and a time of 7.6 min.
29.	Lakshmanaprabu et al. (2019)	RF	The model outperformed better than the Gaussian mixture model and the logistic regression problems, also providing high classification.	The developed model required a high computational cost, and the computational process was slow.	Achieved precision of 94.2%, recall of 89.99%, F1-score of 89.22%, sensitivity of 79.2%, and specificity of 92.14%.
30.	Liang et al. (2018)	Xception-LSTM model	This model extracts the spatial–temporal features, which are highly segmented and classified with high accuracy.	The model required computational expenses, and the problem can be solved by computer hardware.	Accuracy of 90.79%.
31.	Azar and Hassanien (2015)	Neural fuzzy	The LHNFCSF model reduces the dimensionality, discards the redundant noise, and achieves high-speed classification.	The model results are not completely interpretable and are not implemented in complex problems.	Achieved 97.67% accuracy and 7.45 RMSE
32.	Hernández et al. (2020)	MLNN and LMNN	The morphological neurons in the model possibly generate great responses, which provide a high capacity for classification.	The model cannot solve the overfitting problems due to the two high-dimensional datasets.	Achieved 99.05% of accuracy.
33.	Jiang and Li (2019)	RBF neural network	The base width is updated adaptively with the optimization of the cluster center.	The model consumed more time and was limited to complex data	Achieved more than 85% of accuracy.

Note. DT = decision tree; DTCAA = DT credit assessment approach; ANN = artificial neural network; DHBoost = distributed heterogeneous boosting-inspired ensemble classifier; XGBoost = extreme gradient boosting; Light GBM = light gradient boosting machine; SVM = support vector machine; CNN = convolutional neural network; DL = deep learning; CNB = correlative Naïve Bayes; CG-CNB = Cuckoo–Grey wolf-based CNB; CGCNB-MRM = Cuckoo–Grey wolf-based Correlative Naive Bayes classifier with MapReduce model; KNN = K-nearest neighbor; RF = random forest; Xception-LSTM = extreme inception and long short-term memory; LHNFCSF = linguistic hedges neuro-fuzzy classifier with selected features; RMSE = root mean square error; MLNN = morphological-linear neural network; LMNN = linear-morphological neural network; KSC = K-means spectral clustering; RBF = radial basis function.

OPEN IN VIEWER

Table 3.

Comparison of Deep Learning Techniques for Big Data Classification.

Sl.No	References	Methods	Advantage	Disadvantage	Achievement
1.	Boulila et al. (2021)	D-CNN	The D-CNN model provides high accuracy with the help of an effective maximum likelihood supervised technique and efficiently classifies the big data.	Due to the large amount of data, a curse of dimensionality problem was developed, which results in complex classification.	Achieved 92.06% of accuracy.
2.	Naronglerdrit and Mporas (2021)	CNN	The model highly preprocessed and segmented the data for accurate classification, therefore achieving better performance.	The false results will be attained due to the missing required data during the feature extraction process.	Achieved an accuracy of 93.89%.
3.	Almutairi et al. (2024)	CNN	The developed BDA-BMIC method utilized two optimization techniques, which highly enhance the images for better classification.	During the feature extraction process, the required data were deleted, or the data were missing. This results in false results.	Achieved accuracy of 94.6% and sensitivity of 97.3%.
4.	Wu (2022)	CNN	The model highly preprocessed and segmented the data for accurate classification.	This model is used with limited attributes to classify the remote sensing images.	Achieved an accuracy of 86.3%.
5.	Ashraf et al. (2020)	D-CNN	Using the D-CNN model, the image was classified accurately and uses reliable tools as well as techniques that benefit high classification.	This work does not explore large-scale image-based data.	Achieved an accuracy of 97.73%.
6.	Jaya Sudha and Sneha (2022)	D-CNN	Using the D-CNN model, the image was classified accurately.	For large-scale image-based data, this model is not applicable.	—
7.	Salehin et al. (2023)	CNN–LSTM	The model can be classified using normal medical images.	The model takes more time to adapt to the new dataset.	Precision of 97%, accuracy of 91%, and recall of 94%.
8.	Shahzadi et al. (2018)	CNN–LSTM	The model performs effectively in high-dimensional 3D data.	The model requires more data to train and is inaccurate in predicting pixel labels.	Accuracy of 84%.
9.	Rai and Chatterjee (2022)	Deep learning models, including CNN and hybrid CNN–LSTM, as well as ensemble techniques.	The imbalance data issue is addressed using the SMOTE + Tomek link data resampling technique.	The model is not tested for hardware compatibility.	Accuracy of 99.82%.
10.	Rai et al. (2020)	Hybrid CNN–LSTM model	The complete ECG signal with time interval and ECG time gradient was used as a feature instead of feature extraction.	The model suffers from data imbalance problems.	Achieved 99% of accuracy.
11.	Hassib et al. (2020)	BRNN	This approach showed many processes for classification, and the developed optimization provides accurate feature selection that aids in better results.	The WOA–BRNN model required more time for computation on large datasets.	Achieved an accuracy of 94%.
12.	Narayana et al. (2022)	Deep RNN	The ACSO algorithm finds weights for the Deep RNN model and effectively tunes for better classification, also achieving better results.	This model provides less computational speed at maximum cost.	Achieved the highest accuracy of 0.893, specificity of 0.884, sensitivity of 0.900, and maximum threat score of 0.827.
13.	Li et al. (2017)	DCM	The DCM model achieved high performance for big data classification due to the tensor representation of the model.	The DCM took more time than the traditional CNN model due to more parameters in the tensor space.	Accuracy of 93.2%.
14.	Zhu and Chen (2020)	CNN	The model highly preprocessed and segmented the data for accurate classification.	This work does not explore large-scale image-based data.	Achieved an accuracy of 86.3%.
15.	Zhai et al. (2019)	MR-RBM-FI	The model is feasible and very effective due to the utilization of the MapReduce mechanism to train the base classifiers.	The developed model required more time for classification.	—
16.	Lin et al. (2016)	SVM	The model provides high classification with high accuracy.	This model is not implemented using a large number of datasets, which consumes more time and cost.	Achieved an accuracy of 91.85%.

Note. D-CNN = distributed convolutional neural network; CNN = convolutional neural network; BDA-BMIC = big data architecture-based biomedical image classification; ECG = electrocardiogram; BRNN = bidirectional recurrent neural network; WOA = Whale optimization algorithm; Deep RNN = Deep recurrent neural network; ACSO = Ant Cat Swarm Optimization; DCM = deep computation model; MR-RBM-FI = restricted Boltzmann machine with Hadoop MapReduce and fuzzy integral; SVM = support vector machine; SMOTE = synthetic minority oversampling technique.

OPEN IN VIEWER

Table 4.

Comparison of Federated Learning and Transfer Learning Techniques for Big Data Classification.

Sl. No	References	Methods	Advantage	Disadvantage	Achievement
1	Liu et al. (2023)	TLC	The model deals with static and real-time data more effectively with high classification performance.	Computationally expensive while handling larger datasets.	Achieved an accuracy of 85.96%.
2	Ramachandran and Manikandan (2021)	Optimized FedAvg strategy	Enhanced performance in terms of scalability and decision-making.	Requires massive computational resources and time to train the model on a larger dataset	Achieved an accuracy of 93.27%.
3	Dhiman et al. (2022)	Federated learning approach	Provided privacy completely over the life cycle of medical big data.	Medical data gets lost or altered if a cloud platform that houses it is compromised or deliberately targeted.	Attained the delay of 20 s for cPDS.

Note. TLC=thin layer chromatography; FedAvg=federated averaging; cPDS = cluster primal-dual splitting.

OPEN IN VIEWER

Table 5.

Comparison of Ensemble Learning Techniques for Big Data Classification.

Sl.No	References	Methods	Advantage	Disadvantage	Achievement
1	Demidova et al. (2016)	Two-level SVM classifier	Improved classification performance within time expenditures	Computationally intensive and complex to tune	Achieved an accuracy of 97.26%.
2	Ponmalar and Dhanakoti (2022)	Ensemble SVM	Improved performance and detected different types of attacks.	Requires high memory resources for the detection.	Achieved an accuracy of 96.29%.
3	Ramachandran and Manikandan (2021)	Ensemble classification algorithm	The model shows significant performance improvement over a single classifier.	The model faces overfitting and generalization challenges to unseen data.	—

Note. SVM = support vector machine.

This section elaborates on the analysis and discussion of big data classification techniques utilizing different research works based on classification models, datasets, and performance metrics.

3.1 Analysis Based on Metrics

This section presents a quantitative analysis of the selected research articles using various metrics to provide insights into big data classification techniques, as illustrated in Table 6. The graphical illustration is depicted in Figure 8.

OPEN IN VIEWER

Figure 8.

Analysis based on metrics.

OPEN IN VIEWER

Table 6.

Analysis Based on Metrics.

Metrics	Papers
Accuracy	[1, 2, 10, 11, 13, 16, 17, 18, 20, 21, 24, 40, 42] [43, 44, 32, 23, 45, 47, 27, 48, 35, 30, 28, 29, 33] [36, 22, 50, 51, 51, 53, 31, 25, 54, 34, 38]
Precision	[17, 18, 24, 26, 50]
Sensitivity	[2, 5, 11, 13, 24, 44, 45, 26, 33, 25]
Specificity	[2, 5, 11, 13, 24, 45, 26]
F1-score	[13, 17, 18, 24, 43, 44, 26, 50]
Recall	[17, 18, 50]
Time	[19, 20, 58]

3.1.1 Performance Metrics

The analysis presented in this section demonstrates the performance metrics of each established method with accuracy, specificity, and sensitivity, which are described in the context below.

(A) Accuracy: The accurate classification of big data can be evaluated using the following equation.

Accuracy = T pt + T ng T pt + T ng + F ng + F pt .

(1)

(B) Sensitivity: This metric can be measured usually by identifying the crucial features from the datasets, and highly predicts using the following equation:

Sensitivity = T pt T pt F ng .

(2)

(C) Specificity: The specificity of the established model is calculated using the following equation:

Specificity = T pt T ng + F pt ,

(3)

whereas the above-mentioned equation with the variable T pt , T ng , F ng , and F pt is represented as true positive, true negative, false negative, and false positive, in each metric, respectively.

3.4 Analysis of Different Classification Models

This section discusses the classification models used in different big data classification works. Several established methods, such as DT, CNN–LSTM, DCNN, Light GBM with CNN (LGBCNN), VOA–LGBCNN, and HHO–LGBCNN, are analyzed with various training percentages (TPs).

3.4.1 Performance Analysis of DT

The performance analysis of the DT method using TP of 40, 50, 60, 70, 80, and 90 for big data classification in terms of metric accuracy, sensitivity, specificity, and F1-score is revealed in Tables 8–11, respectively. Here, the accuracy of DT with maximum TP-90, the log loss criterion is 77.65%, entropy is 78.78%, and 87.54%. The accuracy performances at each criterion differ and provide low to high values. However, the sensitivity of log loss, entropy, and gini criteria is 77.87%, 79.25%, and 93.47%, whereas the specificity is 77.43%, 78.31%, and 81.62%, respectively. The F1-score at epochs 300, 400, and 500 is 75.27%, 75.64%, and 87.82%, respectively. This analysis shows the achieved results of DT for big data classification.

OPEN IN VIEWER

Table 8.

Accuracy Analysis of DT.

	Accuracy %
	TP
Criteria	40	50	60	70	80	90
DT at criterion = log_loss	75.10	75.68	75.81	75.98	76.44	77.65
DT at criterion = entropy	75.54	76.10	77.41	77.83	78.35	78.78
DT at criterion = gini	88.10	80.95	83.10	88.17	86.79	87.54

Note. DT = decision tree; TP = training percentage.

OPEN IN VIEWER

Table 9.

Sensitivity Analysis of DT.

	Sensitivity %
	TP
Criteria	40	50	60	70	80	90
DT at criterion = log_loss	75.56	76.17	76.26	76.61	77.34	77.87
DT at criterion = entropy	76.11	76.77	77.74	78.12	78.86	79.25
DT at criterion = gini	93.11	82.54	80.34	90.12	87.10	93.47

Note. DT = decision tree; TP = training percentage.

OPEN IN VIEWER

Table 10.

Specificity Analysis of DT.

	Specificity %
	TP
TP Versus Criteria	40	50	60	70	80	90
DT at criterion = log_loss	74.65	75.20	75.35	75.36	75.54	77.43
DT at criterion = entropy	74.97	75.44	77.08	77.54	77.84	78.31
DT at criterion = gini	83.10	79.36	85.86	86.21	86.47	81.62

Note. DT = decision tree; TP = training percentage.

OPEN IN VIEWER

Table 11.

F1-Score Analysis of DT.

	F1-Score%
	TP
Criteria	40	50	60	70	80	90
CNN–LSTM at epoch = 300	72.16	72.69	73.31	73.69	74.47	75.27
CNN–LSTM at epoch = 400	73.10	73.83	74.28	74.76	74.96	75.64
CNN–LSTM at epoch = 500	77.46	83.55	83.95	81.67	84.04	87.82

Note. DT = decision tree; TP = training percentage; CNN–LSTM = convolutional neural network with long short-term memory.

3.4.2 Performance Analysis of CNN–LSTM

The performance analysis for the CNN–LSTM method with TP of 40, 50, 60, 70, 80, and 90, along with varying epochs 300, 400, and 500 in terms of metric accuracy, sensitivity, specificity, and F1-score, is revealed in Tables 12–15, respectively. At maximum TP-90 with epoch 300, the performance of the CNN–LSTM model is 78.06%, 77.63%, and 78.49%, whereas the accuracy, sensitivity, and specificity at epoch 400 are 78.97%, 79.20%, and 78.73%. At epoch 500, the accuracy, sensitivity, and specificity are reported as 83.19%, 82.20%, and 84.17%, respectively. The F1-score of the CNN–LSTM model concerning epochs 300,400, and 500 is 74.21%, 74.63%, and 76.19%, respectively. This analysis shows the achieved results of CNN–LSTM for big data classification.

OPEN IN VIEWER

Table 12.

Accuracy Analysis of CNN–LSTM.

	Accuracy %
	TP
Criteria	40	50	60	70	80	90
CNN–LSTM at epoch = 300	75.52	75.80	76.63	77.03	77.29	78.06
CNN–LSTM at epoch = 400	75.81	76.24	76.69	77.44	77.55	78.97
CNN–LSTM at epoch = 500	80.95	80.28	86.10	88.10	91.59	83.19

Note. CNN–LSTM = convolutional neural network with long short-term memory; TP = training percentage.

OPEN IN VIEWER

Table 13.

Sensitivity Analysis of CNN–LSTM.

	Sensitivity %
	TP
Criteria	40	50	60	70	80	90
CNN–LSTM at epoch = 300	75.33	75.66	76.55	76.91	77.00	77.63
CNN–LSTM at epoch = 400	75.65	76.06	76.62	77.17	77.36	79.20
CNN–LSTM at epoch = 500	80.90	80.28	87.80	90.58	88.03	82.20

Note. CNN–LSTM = convolutional neural network with long short-term memory; TP = training percentage.

OPEN IN VIEWER

Table 14.

Specificity Analysis of CNN–LSTM.

	Specificity %
	TP
Criteria	40	50	60	70	80	90
CNN–LSTM at epoch = 300	75.71	75.94	76.71	77.15	77.57	78.49
CNN–LSTM at epoch = 400	75.98	76.43	76.77	77.70	77.73	78.73
CNN–LSTM at epoch = 500	81.00	80.27	84.39	85.62	95.16	84.17

Note. CNN–LSTM = convolutional neural network with long short-term memory; TP = training percentage.

OPEN IN VIEWER

Table 15.

F1-Score Analysis of CNN–LSTM.

	F1-Score%
	TP
Criteria	40	50	60	70	80	90
CNN–LSTM at epoch = 300	71.24	72.20	72.83	73.05	73.68	74.21
CNN–LSTM at epoch = 400	72.43	72.94	73.20	74.49	74.54	74.63
CNN–LSTM at epoch = 500	86.35	82.05	86.04	85.69	84.44	76.19

Note. CNN–LSTM = convolutional neural network with long short-term memory; TP = training percentage.

3.4.3 Performance Analysis of DCNN

The performance analysis for the DCNN method with TP of 40, 50, 60, 70, 80, and 90, along with varying epochs of 300, 400, and 500 in terms of metric accuracy, sensitivity, specificity, and F1-score is revealed in Tables 16–19. At maximum TP-90 with epoch 300, the accuracy, sensitivity, and specificity of the established model are 79.67%, 80.48%, and 78.85%, whereas the performance at epoch 400 is 79.85%, 80.69%, and 79.01%, and at epoch 500, the performance percentage is 90.02%, 87.17%, and 92.87%, respectively. At epochs 300, 400, and 500, the DCNN model achieved an F1-score of 78.63%, 79.03%, and 83.21%, respectively. This analysis shows the achieved results of DCNN for big data classification, which is effectively higher than the previous model.

OPEN IN VIEWER

Table 16.

Accuracy Analysis of Deep CNN.

	Accuracy %
	TP
Criteria	40	50	60	70	80	90
Deep CNN at epoch = 300	76.64	76.88	77.20	77.95	78.62	79.67
Deep CNN at epoch = 400	77.00	77.95	78.49	78.77	79.64	79.85
Deep CNN at epoch = 500	88.43	89.24	81.15	91.40	84.01	90.02

Note. CNN = convolutional neural network; TP = training percentage.

OPEN IN VIEWER

Table 17.

Sensitivity Analysis of Deep CNN.

	Sensitivity %
	TP
Criteria	40	50	60	70	80	90
Deep CNN at epoch = 300	77.08	77.16	77.28	78.09	78.87	80.48
Deep CNN at epoch = 400	77.34	78.75	79.44	79.71	80.59	80.69
Deep CNN at epoch = 500	89.14	84.03	82.08	93.32	83.31	87.17

Note. CNN = convolutional neural network; TP = training percentage.

OPEN IN VIEWER

Table 18.

Specificity Analysis of Deep CNN.

	Specificity %
	TP
Criteria	40	50	60	70	80	90
Deep CNN at epoch = 300	76.20	76.60	77.12	77.81	78.37	78.85
Deep CNN at epoch = 400	76.66	77.15	77.54	77.84	78.69	79.01
Deep CNN at epoch = 500	87.73	94.45	80.23	89.49	84.71	92.87

Note. CNN = convolutional neural network; TP = training percentage.

OPEN IN VIEWER

Table 19.

F1-Score of Deep CNN.

	F1-Score%
	TP
40	50	60	70	80	90
CNN–LSTM at epoch = 300	75.76	75.86	76.40	77.16	78.13	78.63
CNN–LSTM at epoch = 400	76.12	76.66	77.56	78.54	78.88	79.03
CNN–LSTM at epoch = 500	80.61	85.02	88.80	86.80	86.63	83.21

Note. CNN = convolutional neural network; TP = training percentage; CNN–LSTM = CNN with long short-term memory.

3.4.4 Performance Analysis of LGBCNN

The performance analysis of the LGBCNN method with TP of 40, 50, 60, 70, 80, and 90, along with varying epochs of 300, 400, and 500 for big data classification in terms of metric accuracy, sensitivity, specificity, and F1-score is revealed in Tables 20–23, respectively. At maximum TP-90 with epoch 300, the accuracy, sensitivity, and specificity of the established model are 82.96%, 80.10%, and 85.82%, whereas the performance at epoch 400 is 83.41%, 80.62%, and 86.20%, and at epoch 500, the performance percentage is 89.85%, 83.38%, and 96.31%, respectively. The F1-score of the LGBCNN method at epoch 300 is 75.63%, at epoch 400 is 76.10%, and at epoch 500 is 87.67%. This analysis shows the achieved results of LGBCNN for big data classification.

OPEN IN VIEWER

Table 20.

Accuracy Analysis of LGBCNN.

	Accuracy %
	TP
Criteria	40	50	60	70	80	90
LGBCNN at epoch = 300	79.96	80.75	81.07	81.50	82.22	82.96
LGBCNN at epoch = 400	80.32	81.21	81.65	81.95	82.81	83.41
LGBCNN at epoch = 500	89.64	87.23	87.05	89.14	87.64	89.85

Note. LGBCNN=light gradient boosting machine with convolutional neural network; TP=training network.

OPEN IN VIEWER

Table 21.

Sensitivity Analysis of LGBCNN.

	Sensitivity %
	TP
Criteria	40	50	60	70	80	90
LGBCNN at epoch = 300	77.13	77.69	78.12	78.17	79.47	80.10
LGBCNN at epoch = 400	77.44	78.53	78.66	78.77	80.30	80.62
LGBCNN at epoch = 500	90.58	82.43	81.81	90.75	86.10	83.38

Note. LGBCNN=light gradient boosting machine with convolutional neural network; TP=training network.

OPEN IN VIEWER

Table 22.

Specificity Analysis of LGBCNN.

	Specificity %
	TP
TP Versus Criteria	40	50	60	70	80	90
LGBCNN at epoch = 300	82.79	83.81	84.03	84.82	84.98	85.82
LGBCNN at epoch = 400	83.19	83.88	84.65	85.13	85.32	86.20
LGBCNN at epoch = 500	88.71	92.03	92.30	87.54	89.18	96.31

Note. LGBCNN = light gradient boosting machine with convolutional neural network; TP = training network.

OPEN IN VIEWER

Table 23.

F1-Score of LGBCNN.

	F1-Score%
	TP
Criteria	40	50	60	70	80	90
CNN–LSTM at epoch = 300	72.85	72.95	73.17	74.38	74.79	75.63
CNN–LSTM at epoch = 400	72.97	74.01	74.14	74.88	75.48	76.10
CNN–LSTM at epoch = 500	85.58	83.03	83.91	77.22	83.38	87.67

Note. LGBCNN = light gradient boosting machine with convolutional neural network; TP = training network; CNN–LSTM = convolutional neural network with long short-term memory.

3.4.5 Performance Analysis of VOA–LGBCNN

The performance analysis for the VOA–LGBCNN with TP of 40, 50, 60, 70, 80, and 90, along with varying epochs 300, 400, and 500 for big data classification, in terms of metric accuracy, sensitivity, specificity, and F1-score, is revealed in Tables 24–27, respectively. At maximum TP-90 with epoch 300, the accuracy, sensitivity, and specificity of the established model are 78.82%, 79.58%, and 88.38%, whereas the performance at epoch 400 is 79.37%, 79.72%, and 79.02%, and at epoch 500, the performance percentage is 88.38%, 94.46%, and 82.30%, respectively. This analysis represents the achieved results of VOA–LGBCNN for big data classification.

OPEN IN VIEWER

Table 24.

Accuracy Analysis of VOA–LGBCNN.

	Accuracy %
	TP
Criteria	40	50	60	70	80	90
VOA–LGBCNN at epoch = 300	75.92	76.59	76.65	76.79	77.36	78.82
VOA–LGBCNN at epoch = 400	76.33	77.13	78.03	78.25	78.69	79.37
VOA–LGBCNN at epoch = 500	85.40	93.70	89.00	80.68	93.91	88.38

Note. VOA–LGBCNN = vulture optimization algorithm–light gradient boosting machine with convolutional neural network; TP = training network.

OPEN IN VIEWER

Table 25.

Sensitivity Analysis of VOA–LGBCNN.

	Sensitivity %
	TP
Criteria	40	50	60	70	80	90
VOA–LGBCNN at epoch = 300	76.28	77.09	77.17	77.34	78.41	79.58
VOA–LGBCNN at epoch = 400	76.94	77.72	78.60	78.89	78.91	79.72
VOA–LGBCNN at epoch = 500	85.12	93.64	92.29	80.92	92.69	94.46

Note. VOA–LGBCNN = vulture optimization algorithm–light gradient boosting machine with convolutional neural network; TP = training network.

OPEN IN VIEWER

Table 26.

Specificity Analysis of VOA–LGBCNN.

	Specificity %
	TP
Criteria	40	50	60	70	80	90
VOA–LGBCNN at epoch = 300	75.56	76.09	76.14	76.25	76.32	78.06
VOA–LGBCNN at epoch = 400	75.72	76.55	77.46	77.61	78.47	79.02
VOA–LGBCNN at epoch = 500	85.68	93.77	85.70	80.45	95.12	82.30

Note. VOA–LGBCNN = vulture optimization algorithm–light gradient boosting machine with convolutional neural network; TP = training network.

OPEN IN VIEWER

Table 27.

F1-Score of VOA–LGBCNN.

	F1-Score%
	TP
Criteria	40	50	60	70	80	90
VOA–LGBCNN at epoch = 300	72.25	72.88	72.97	73.36	73.94	75.40
VOA–LGBCNN at epoch = 400	72.62	73.47	73.76	74.39	75.23	75.43
VOA–LGBCNN at epoch = 500	89.36	81.66	85.40	81.82	81.66	83.14

Note. VOA–LGBCNN = vulture optimization algorithm–light gradient boosting machine with convolutional neural network; TP = training network.

3.4.6 Performance Analysis of HHO–LGBCNN

The performance analysis for the HHO–LGBCNN method with TP of 40, 50, 60, 70, 80, and 90, along with varying epochs 300, 400, and 500 for big data classification in terms of metric accuracy, sensitivity, specificity, and F1-score is revealed in Tables 28–31, respectively. At maximum TP-90 with epoch 300, the accuracy, sensitivity, and specificity of the established model are 77.71%, 77.69%, and 77.72%, whereas the performance at epoch 400 is 78.17%, 78.38%, and 77.96%, and at epoch 500, the performance percentage is 86.38%, 81.43%, and 91.33%, respectively. The HHO–LGBCNN method achieved an F1-score of 74.25%, 75.10%, and 85.31% at epochs 300,400, and 500. This analysis represents the achieved results of HHO–LGBCNN for big data classification.

OPEN IN VIEWER

Table 28.

Accuracy Analysis of HHO–LGBCNN.

	Accuracy %
	TP
Criteria	40	50	60	70	80	90
HHO–LGBCNN at epoch = 300	74.80	75.27	76.18	76.41	77.46	77.71
HHO–LGBCNN at epoch = 400	75.30	75.89	76.66	76.82	77.69	78.17
HHO–LGBCNN at epoch = 500	81.55	79.50	88.11	83.44	95.48	86.38

Note. HHO–LGBCNN = Haris Hawks optimization–light gradient boosting machine with convolutional neural network; TP = training percentage.

OPEN IN VIEWER

Table 29.

Sensitivity Analysis of HHO–LGBCNN.

	Sensitivity %
	TP
Criteria	40	50	60	70	80	90
HHO–LGBCNN at epoch = 300	75.31	75.81	76.08	76.19	77.41	77.69
HHO–LGBCNN at epoch = 400	75.69	76.04	76.48	76.73	77.70	78.38
HHO–LGBCNN at epoch = 500	82.94	79.91	87.71	87.31	95.05	81.43

Note. HHO–LGBCNN = Haris Hawks optimization–light gradient boosting machine with convolutional neural network; TP = training percentage.

OPEN IN VIEWER

Table 30.

Specificity Analysis of HHO–LGBCNN.

	Specificity %
	TP
Criteria	40	50	60	70	80	90
HHO–LGBCNN at epoch = 300	74.30	74.72	76.27	76.62	77.51	77.72
HHO–LGBCNN at epoch = 400	74.91	75.74	76.84	76.91	77.67	77.96
HHO–LGBCNN at epoch = 500	80.16	79.10	88.51	79.58	95.92	91.33

Note. HHO–LGBCNN = Haris Hawks optimization–light gradient boosting machine with convolutional neural network; TP = training percentage.

OPEN IN VIEWER

Table 31.

F1-Score of HHO–LGBCNN.

	F1-Score%
	TP
Criteria	40	50	60	70	80	90
CNN–LSTM at epoch = 300	71.62	71.77	72.22	72.97	73.32	74.25
CNN–LSTM at epoch = 400	72.64	72.85	74.07	74.47	74.87	75.10
CNN–LSTM at epoch = 500	78.27	83.42	79.62	79.15	83.10	85.31

Note. HHO–LGBCNN = Haris Hawks optimization–light gradient boosting machine with convolutional neural network; TP = training percentage; CNN–LSTM = convolutional neural network with long short-term memory.

3.5 Time Complexity Analysis

The complexity analysis of several ML and DL models is represented in Figure 9. At epoch 50, the classification models such as DT, CNN–LSTM, DCNN, LGBCNN, VOA–LGBCNN, and HHO–LGBCNN use time of 48.9, 42.9, 43.6, 44.4, 44.6, and 42.8 s, respectively. Similarly, at epoch 70, the CNN–LSTM, HHO–LGBCNN, DCNN, and LGBCNN models use a shorter time of 56.4, 56.4, and 57.6 s, respectively. Finally, at epoch 99, the HHO–LGBCNN model consistently uses a lower time of 75.9 s, compared to all other classification models. The inclusion of HHO with the LGBCNN model effectively reduces the computational time, thus increasing the model performance.

OPEN IN VIEWER

Figure 9.

Time complexity analysis.

3.6 Memory Analysis

The classification models’ memory usage analysis of several ML and DL models, including DT, CNN–LSTM, DCNN, LGBCNN, VOA–LGBCNN, and HHO–LGBCNN models, is depicted in Figure 10. At epoch 30, the DT, CNN–LSTM, DCNN, and LGBCNN models utilize memories of 188.5 kb, 169.8 kb, and 175.90 kb, while the HHO–LGBCNN model uses less memory of 168.48 kb. Similarly, at epoch 70, LGBCNN, VOA–LGBCNN, and HHO–LGBCNN models utilize memory of 321.0 kb, 323.8 kb, and 292.6 kb, respectively. Finally, at epoch 99, the HHO–LGBCNN model utilizes less memory of 410.3 kb, highlighting its scalability in classifying big data in resource-constrained environments.

OPEN IN VIEWER

Figure 10.

Memory analysis.

In this review, several papers with multiple DL and ML techniques for big data classification are examined and utilized. The achievements, merits, and demerits of several existing methods are determined and helpful for future implementation works. Moreover, the significant analysis of the reviewed papers shows that DL methods are very effective in the classification of big data. Specifically, the hybrid DL methods, such as CNN–LSTM and LGBCNN, are more efficient than the other methods. The DL techniques are efficient in handling multiple sequential data and, more importantly, work in time series data and classify effectively. However, these DL methods have an imbalance dataset issue and also high computational complexity during the prediction. Moreover, the existing ML method faces issues in high computational time due to the utilization of large datasets and also faces scalability issues, which affect the training and lead to inefficient classification and overfitting. Model parallelism requires frequent communication between devices to exchange intermediate outputs and pass gradients for backpropagation. This can introduce significant latency and create performance bottlenecks, especially in models with many interdependent layers, as model parallelism's scalability is constrained by the number of layers in the model. Simply adding more graphics processing units does not improve performance beyond a certain point, limiting its effectiveness for horizontal scaling. In cloud-based big data processing techniques, data from diverse, heterogeneous sources can be difficult to integrate and unify for classification. Compiling this variety of data types and formats is a complex task. Furthermore, in cloud-based preprocessing, data is stored and processed by a third-party provider, increasing its vulnerability to cyber threats, unauthorized access, and data breaches. Storing and processing large volumes of sensitive big data, such as personal or health information, is a major concern for regulatory compliance. In DL for Big Data classification, determining the optimal number of model parameters and enhancing their computational practicality present significant challenges. The large volume of big data often reflects and amplifies past societal biases. As a result, these models can learn and reinforce unfair patterns, leading to biased and discriminatory results. Additionally, the black box nature of deep neural networks makes it hard to understand how they make classification decisions. This lack of clarity complicates accountability and debugging. Without transparency, trust in critical applications, such as medical diagnoses or loan decisions, breaks down, making it very hard to justify or audit a model's possibly biased outcomes. The hybrid DL model has specific challenges in big data classification, which include streaming data, high-dimensional data, and scalability issues.

5. Research Gaps and Future Works

This section explains the limitations and future work of existing methods in big data classification using the recently reviewed papers, which is thus helpful for the development of innovative models in future classification.

6. Applications

7. Conclusion

The research survey aims to develop a highly efficient model for big data classification to classify important data in numerous fields, such as healthcare, marketing, and other fields, with improved security. In this research, several reviewed papers from recent years with multiple DL and ML techniques, as well as the established methods, are examined. In this paper, the achievements, merits, and demerits of several existing methods are determined, which are helpful for future implementation works. These analyses will make future research quite easy, simple, and straightforward approaches to effectively handle classification with more adapted strategies, such as preprocessing, feature extraction, and feature selection. Moreover, the significant analysis of the reviewed papers shows that DL methods are very effective in the classification of big data. Specifically, hybrid DL methods, such as CNN–LSTM and LGBCNN, are more efficient than the other methods. The significant analyses of the established models are measured using several metrics such as accuracy, sensitivity, and specificity. Big data has a transformative effect on the real world. It improves everything from personalized customer experiences and targeted marketing to operational efficiency, predictive maintenance in manufacturing, and groundbreaking changes in healthcare and public services. By examining large datasets, organizations can make better, data-driven decisions. They can also anticipate market trends and innovate more quickly, giving them a strong competitive advantage. However, these advantages come with serious tradeoffs, mainly related to privacy and security. Storing and processing huge amounts of personal information makes companies more susceptible to data breaches and misuse. Setting up and keeping big data systems also requires significant investments in infrastructure, software, and skilled workers. This creates cost challenges for many organizations. Additionally, the problems with data quality, truthfulness, and bias in algorithms can lead to misleading or unfair outcomes.

8. Future Work

Future work will focus on developing dynamic and heterogeneous ensemble techniques that adapt in real-time to evolving data streams, combining diverse base learners and incorporating advanced optimization strategies, such as hybrid cloud-edge training and parallel processing, to enhance the scalability and performance of the model big data classification and to address the problems associated with Big Data classification, including high dimensionality, streaming data analysis, and scalability of DL models. The feature research will be considered, integrating self-supervised learning and automated feature engineering to enable models to automatically discover optimal features and learn from massive unlabeled datasets, which is particularly vital for overcoming labeling bottlenecks and handling complex, high-dimensional data efficiently within distributed computing frameworks. This integrated approach, combining intelligent automation, adaptive learning, and robust optimization, will pave the way for a new generation of big data classification models that are more autonomous, resilient, and effective.

Supplemental Material