An integrated Neutrosophic-BWM-COPRAS based MCDM framework for intrusion detection system software selection

Abstract

Intrusion Detection Systems (IDS) play a significant role in cybersecurity as they detect potential threats and protect network infrastructure. With the existing cybersecurity environment, the selection of the most suitable IDS software tool is very crucial to provide strong cybersecurity practices in organizations. The availability of various IDS software tools with varying features and capabilities makes the selection of the optimal one a complex decision-making problem. Conventional multi-criteria Decision making (MCDM) approaches cannot deal with the uncertainty and ambiguity associated with these criteria, which can result in making wrong decisions. The current study introduces a novel framework for the selection of an optimal IDS software tool that integrates Neutrosophic sets and COCOSO (Combined Compromise Solution) to effectively deal with the uncertainty involved in IDS selection. The Best-Worst Method considers the best and worst criteria for consistent and accurate weight assignment to improve decision accuracy. The Neutrosophic approach deals with ambiguity by considering truth, falsity, and indeterminacy together, and improves the flexibility of the COCOSO approach to assess these IDS software tools. A case study illustrates the usability and efficiency of the introduced framework to select an IDS software tool in secure dynamic environments over conventional approaches.

Keywords

COCOSO Neutrosophic set MCDM IDS software cyber security BWM

1 Introduction

Intrusion Detection Systems (IDS) have become a crucial part of any cybersecurity infrastructure to serve as a proactive tool for detecting, preventing, and monitoring security risks in networks. With the rise in cyber-attacks, organizations all over the world are investing in intrusion detection systems as a vital part of their digital infrastructure. IDS software acts as a line of Défense against malicious activity, unauthorized access, and data breaches by monitoring, detecting, and responding to security breaches within networks. However, the selection of an IDS software tool is a challenging process as it considers multiple criteria simultaneously, such as detection accuracy, system compatibility, resource consumption, cost, ease of integration, and scalability. Each of these criteria has a varying level of significance as per organizational security requirements.^1–3 Therefore, choosing IDS software requires a robust decision-making approach that can take into consideration multiple frequently conflicting criteria.

Traditional MCDM approaches, such as TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) and AHP (Analytic Hierarchy Process), have widely used for IDS software tool selection and assessment.^4–6 However, these techniques often lack in the consideration of allied ambiguities and uncertainties in the decision-making process. Thus, in such real-time decision-making environments where exact information is unavailable or where decisions include subjective, confusing information, or criteria may overlap, these traditional approaches might not yield reliable results.⁷

To address these gaps, recent studies have widely incorporated fuzzy and Neutrosophic sets, which are more flexible - allowing decision makers to manage indeterminate, uncertain, and inconsistent information specially for IDS tool selection.⁸ Neutrosophic logic considers truth, falsity, and indeterminacy to represent data, thus providing a better and flexible structure to manage indeterminate, uncertain, and inconsistent information.⁹ This paper introduces an integrated Neutrosophic-based COCOSO MCDM approach which combines Best Worst Method (BWM) to determine the weights for selecting the IDS software tool. BWM, a new and growingly popular MCDM weighing technique, improves consistency by limiting decision-makers to comparing only the best and worst criteria, making the weighting process dependable and efficient.¹⁰ The COCOSO approach is intended to synthesize compromise solutions and ideally suited to manage the intricate trade-offs in multi-criteria evaluation.^11,12

1.1 Novelty and contributions

Traditional MCDM approaches such as COPRAS, VIKOR, and TOPSIS etc. works either with crisp or fuzzy data, which may not effectively reflect the indeterminateness and vagueness intrinsic in such cyber-security related decision-making scenarios. By utilizing Neutrosophic sets, our proposed framework clearly models indeterminateness along with truth and falsity which allows a more realistic way of handling uncertainty in IDS evaluation.

These traditional approaches often used either subjective or inconsistent pairwise comparison for weight allotment which reduces reliability.

The proposed framework uses the BWM approach to generate more reliable, and less redundant weights which helps to reduce judgment errors and to ensure reliability and robustness in muti-criteria weighting for IDS evaluation. BWM delivers dependable weights, which strengthen the COCOSO ranking mechanism.

These traditional approaches often used a single compromise measure like VIKOR uses regret minimization and TOPSIS uses distance to ideal. However, our framework NB-COCOSO integrates the COCOSO compromise strategy. It combines weighted sum, exponential, and relative significance-based aggregation to provide robust ranking. This hybrid compromise results in consistent and stable ranking mechanism in IDS evaluation.

The state-of-the-art works for IDS tool selection are confined and commonly unreliable in handling uncertainty and expert inconsistency. The proposed NB-COCOSO framework is one of the first which formally integrating the Neutrosophic theory, BWM, and COCOSO in IDS tool selection, thereby bridging the methodological gap.

To illustrate the utility of this approach, the current work applies the Neutrosophic-BWM-COCOSO (NB-COCOSO) framework to a case study of IDS software tools selection. The results demonstrate its effectiveness in providing clear, uncertainty-resilient decision outcomes and highlight its superiority over traditional approaches. This research contributes a structured, adaptable framework to cybersecurity decision-making, enabling organizations to make informed and reliable IDS tool choices in complex, dynamic environments. The main objectives of this study are as follows:

Develop an integrated framework for decision-making that combines the BWM method with Neutrosophic-based COCOSO to improve the objectivity and robustness of choosing the optimal IDS software tool.

Address the intrinsic vagueness, uncertainty and indeterminacy allied with the decision-making method by using Neutrosophic logic which allows a more realistic and adaptable assessment of IDS software alternatives.

Assign weights using BWM approach which ensures reliable and efficient weighting to evaluation criteria by concentrating on best and worst criteria while maintaining consistency.

Determine and prioritize essential criteria for assessing IDS software tools.

Validate the proposed NB-COCOSO framework by comparing the outcomes with other conventional MCDM methods (COPRAS, DBA, VIKOR, and TOPSIS) to ensure consistency, reliability, and effectiveness.

Employ the introduced NB-COCOSO framework in a real-life scenario, demonstrating the model's efficacy, usefulness, and adaptability to dynamic cybersecurity requirements.

The rest of the paper is organized as mentioned: Section 2 discusses the state-of-the-art available in open literature. Section 3 introduces the proposed NB-COCOSO framework which describes the integrated framework using approaches Neutrosophic, BWM and COCOSO method. Section 4 presents the step-by-step discussion on the simulation of proposed approach. Section 5 discusses the detailed results and findings followed by conclusion in section 6.

2 State-of-the-art

In the evolving environment of cyber security, the choice of an efficient IDS becomes a crucial task for organizations. The significance of IDS software assessment has extensively acknowledged, as it permits organizations to make comparisons and deploy the best IDS system in their organization environment, which can further reduce the risk of security breaches.¹³ Schrötter et al.¹⁴ compared well-known IDS tools such as Snort, Zeek, and Suricata. The authors used the IPv6 plugin Suite to detect stateful attacks for improving Snort. Hu et al.¹⁵ assessed Snort and Suricata, for the identification of efficiency on networks. They considered parameters like system resource utilization, identification accuracy, packet drop frequency, and packet processing power for performance evaluation. Moreover, they also described the complications while using open-source IDSs in large networks and suggested solutions for network managers that can help in establishing new IDSs.

Waleed et al.¹⁶ considered three Open-Source IDS, namely Snort, Zeek, and Suricata, to analyse the performance over parameters like packet size, ruleset size, detection engine algorithms, and packet capturing modules. Wahyu et al.¹⁷ attempted to analyse the performance of these IDS tools, Snort, Suricata, Zeek, along with two more tools, OSSEC and honeypot Cowrie. The authors used parameters like throughput, delay, packet loss, and jitter for evaluation. They also evaluated the tools for handling DDoS attacks. Snort performed better over other IDS tools. Boukebous et al.¹⁸ compared two network-based IDS tools, Snort-3 and Suricata, based on multithreading, cross-platform support, and expanded bindings. The authors analysed the performance in terms of packet processing rate, accuracy, number of packets, memory, and processor usage. Snort-3 performed better over Suricata; however, Suricata performed comparably with Snort-3.

In the next year, Ghazi et al.¹⁹ analysed both the tools based on their detection abilities, frameworks, and performance metrics such as processing speeds, traffic volumes, and threat types. This study also explored the possibilities of integration of IDS tools with AI and machine learning. Chinnasamy et al.²⁰ discussed the rule-based detection engine, security information, signature-based threat detection, and event management of the Suricata IDS tool. This study also pointed out the efficiency of Suricata as compared to Snort and Zeek in terms of scalability and performance. Over the course of that year, Yang²¹ used four evaluation criteria, scalability, detection accuracy, false positive rate, and resource efficiency to evaluate the performance of four IDS tools. Being an open source and with strong community support, the traditional tools: Snort and Zeek still considered as valuable. However, Darktrace and QRadar performed better with respect to scalability and accuracy over Snort and Zeek. In that year itself, Almuseelem²² analysed the four IDS tools Snort, Zeek, Suricata, and OSSEC but in cloud environments and for analysing an Amazon VPC's traffic. Performance of OSSEC proved to be better over Snort and Suricata in a period of 3 h.

Few studies^23,24 examined the performance of machine learning algorithms to select the best IDS tool. AbdulRaheem et al.²⁵ introduced an ML-based Snort and Zeek to make the classification between the benign traffic and DDoS attack traffic. The ML-based Snort and Zeek consumed less processing time as compared to the existing methods. Limited research studies explored MCDM approaches for the evaluation of IDS tools based on multiple evaluation criteria. Seelammal & Vimala²⁶ determined the optimal feature selection using an MCDM approach in dynamic environments. The authors also discussed how the large datasets can manage in an optimized way. Belal and Sundaram²⁷ introduced a hybrid framework, MEREC-VIKOR, for evaluating attack detection based on multiple conflicting criteria. They used the CICIDS dataset for the simulation of the proposed framework.

In state-of-the-art, few studies stated that the integration of fuzzy with the conventional MCDM approaches may improve the results. Almotiri²⁸ used the fuzzy AHP approach for the selection of the most efficient traffic detection approach. In addition, they implemented the TOPSIS approach to evaluate and classify alternatives based on their overall performance. During that year, AlHarbi et al.⁵ used the same AHP-TOPSIS approach for the assessment of ML-based IDS software tools in hesitant fuzzy conditions. In the subsequent year, Alyami et al.²⁹ applied the same approach to evaluate the efficiency and effectiveness of five well-known IDS tools, such as Suricata, Zeek, Security Onion, Snort, and OSSEC, in a security environment. The study shown the more benefits of Suricata over Snort. Abushark et al.³⁰ applied the same approach to assess the ML-based IDS software tools based on optimization. The authors identified the cybersecurity attributes that can help developers to make an effective and efficient IDS tool. Another study applied the same hybrid approach for optimal decision making for detecting malevolent attacks but in Vehicular Ad-hoc Network (VANET).

Within that year, Abdel-Basset et al.³¹ applied the q-rung orthopair fuzzy sets for dealing with uncertainty and vagueness. They used the q-rung orthopair fuzzy weighted geometric for combining the experts and specialists’ opinions. They used the COCOSO method to assess the 6 IDS tools (Suricata, Zeek, Security Onion, Snort, Wazuh, and OSSEC) for their reliability and effectiveness. The authors declared that Suricata is the best among six IDS based on multithreading performance. AbdelMouty & Abdel-Monem³² used the Neutrosophic EDAS method to rank and assess the security risks in the power system. Alhakami⁶ used the Fuzzy-AHP TOPSIS approach for the assessment of effective Défense against Gen V Multi-Vector attacks. Due to limited research available on the evaluation of IDS tools, the state-of-the-art included the research studies which can used while implementing IDS tools. It is tough to implement these tools as they built for specific systems and conditions. Therefore, there is a substantial requirement for assessing the effectiveness of the IDS tools based on different criteria. Table 1 summarizes only those research studies which presented the evaluation of IDS tools.

Table 1.

Summary of state-of-the-art for evaluating IDS software tools.

Authors/Year	Schrotter et al., 2019¹⁴	Hu et al., 2022¹⁵	Alyami et al., 2022²⁹	Abdel-Basset et al., 2022³¹	Waleed et al., 2022¹⁶	Boukebous et al., 2023¹⁸	Wahyu et al., 2023¹⁷	Jarabaa, 2024³³	Ghazi et al., 2024¹⁹	AbdulRaheem et al., 2024²⁵	Chinnasamy et al., 2025²⁰	Almuseelem, 2025²²	Yang, 2025²¹
Snort	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Zeek	✓	✗	✓	✓	✓	✗	✓	✗	✗	✓	✓	✓	✓
Suricata	✓	✓	✓	✓	✓	✓	✓	✓	✓	✗	✓	✓	✗
OSSEC	✗	✗	✓	✓	✗	✗	✓	✗	✗	✗	✗	✓	✗
Security Onion	✗	✗	✓	✓	✗	✗	✗	✗	✗	✗	✗	✗	✗
Wazuh	✗	✗	✗	✓	✗	✗	✗	✗	✗	✗	✗	✗	✗
Darktrace	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✓
QRadar	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✓
Honeypot Cowrie	✗	✗	✗	✗	✗	✗	✓	✗	✗	✗	✗	✗	✗
Methods Used for Weights to Evaluation Criteria
AHP	✗	✗	✓	✗	✗	✗	✗	✓	✗	✗	✗		✗
Entropy	✗	✗	✗	✓	✗	✗	✗	✗	✗	✗	✗		✗
Average	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗		✓
Fuzzy Environment Used
Fuzzy Set	✗	✗	✓	✓	✗	✗	✗	✗	✗	✗	✗		✗
Fuzzy offLogic	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗		✓
Neutrosophic Set	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗		✗
Methods Used for Ranking to IDS Software Tools
TOPSIS	✗	✗	✓	✗	✗	✗	✗	✗	✗	✗	✗		✗
IPV6 Plugin Suite	✓	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗		✗
COCOSO	✗	✗	✗	✓	✗	✗	✗	✗	✗	✗	✗		✗
ML models	✗	✗	✗	✗	✗	✗	✗	✓	✗	✓	✗		✗
VIKOR	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗	✗		✓

Table 2 discusses the limitations of state-of-the-art models as compared to COCOSO approach. However, as per state-of-the-art, no single model is ideal for all conditions. As per our knowledge and after analysing Tables 1 and 2, it can be concluded that this research work is the first one that examines twelve available IDS software tools using an integrated Neutrosophic MCDM-based technique. These are Zeek, Splunk, OSSEC, Sagan, Snort, Suricata, Tripwire, Security Onion, Manage Engine, Gatewatcher, CrowdSec, and AIDE. The rapid usage of COCOSO on different applications is visible in state-of-the-art due to its understandable and less complex nature.^34–36 So, this study uses the COCOSO MCDM approach for assessing the effectiveness.

Table 2.

Limitations of state-of-the-art models.

Features	TOPSIS	VIKOR	COCOSO
Dependency on ideal solutions	significantly relies on ideal and anti-ideal values; this may result in ranking instability.	depends on proximity to ideal or anti-ideal solutions; it may induce sensitivity to data normalization.	combines multiple compromise approaches, minimizing excessive dependency on a single ideal value.
Adaptability for Aggregation	restricted to satisfaction and dissatisfaction measures.	restricted to closeness based on Euclidean distance.	supports an adaptable compromise solution by integrating three approaches: cumulative weighting, exponential correlation, and compromise.
Stability of Rankings	Rankings can differ substantially with minor changes in data or normalization approach.	can produce unstable rankings due to sensitivity to scaling & normalization	more stable rankings because of the combined weighting and aggregation approaches.
Dealing with uncertainty	not intrinsically formulated for dealing with uncertain data.	restricted capacity in dealing with uncertainty & fuzziness.	better suited for integration with fuzzy or Neutrosophic-based environments.

3 Proposed NB-COCOSO Framework

A novel NB-COCOSO MCDM-based framework is introduced to select the optimum IDS tool based on predefined evaluation criteria as illustrated in Figure 1. This integration enhances decision-making since it combines consistency (BWM), addressing uncertainty (Neutrosophic), and reliable aggregation (COCOSO), providing an improved stable, flexible, and reliable evaluation framework compared to previous MCDM models.

Figure 1.

Proposed NB-COCOSO framework.

The first phase of the framework identifies the IDS tools and the evaluation criteria based on experts’ opinion and the state-of-the art. The framework takes these identified IDS tools as input for the evaluation. Second phase uses the BWM approach to assign the weights to these evaluation criteria. In the third phase, the data collection is done by the model. The experts use the Neutrosophic Fuzzy Set (NFS) to give judgments for each IDS tool with respect to the performance evaluation criterion. These inputs considered as the data for the proposed framework. In the final phase of the framework, the COCOSO approach is used to prioritize the IDS tools. The COCOSO approach needs crisp values to work on. So, during phase 3, the Neutrosophic data is converted into crisp values before being used by the COCOSO. The proposed framework integrates the three most viable approaches, NFS, BWM, and COCOSO. This section discusses all these approaches in detail, and the next section will discuss the simulation of the proposed framework.

3.1 Neutrosophic Fuzzy Set

The Neutrosophic Fuzzy Set is a computational model that synthesizes the traditional fuzzy and intuitionistic fuzzy theory by introducing a three-dimensional view of uncertainty. Florentin Smarandache³⁷ introduced this concept, which is effective when imprecision, vagueness, and indeterminacy coexist in real-world applications. It defines 3 independent membership components, namely true membership $(T_{m})$ , false membership $(F_{m})$ and indeterminacy membership $(I_{m})$ which allows NFS to model more complex situations. However, these components must satisfy the following condition.

0 \leq T_{m} (x), I_{m} (x), F_{m} (x) \leq 1

Here $T_{m} (x)$ denotes the degree to which an element ‘x’ belongs to a set, $F_{m} (x)$ denotes the degree to which an element ‘x’ does not belong to the set and $I_{m} (x)$ denotes the degree of uncertainty, i.e., whether the element ‘x’ belongs to the set or not. Neutrosophic fuzzy sets are more flexible than intuitionistic fuzzy sets as their sum need not necessarily be 1. An assertion can concurrently have high degrees of truth, falsity, and indeterminacy as mentioned in equation (1).

\begin{aligned} 0^{-} \leq T_{m} (x) + I_{m} (x) + F_{m} (x) \leq 3^{+} \end{aligned}

(1)

Consider an element ‘x’ in a set X of "IDS software tools":

$T_{m} (x) = 0.8$ signifies 80% of experts believe ‘x’ belongs to IDS software tools .

$I_{m} (x) = 0.1$ signifies 10% of experts are uncertain about ‘x’ belonging to set X.

$F_{m} (x) = 0.3$ signifies 30% of experts believe ‘x’ does not belong to set X .

The membership value for element (x) can be expressed as triplets < $T_{m} (x), \; I_{m} (x), F_{m} (x) > i . e .$ <0.8,0.1,0.3>. In this scenario ‘x’ can be any evaluation criteria which is rated by an expert in a set of evaluation criteria ‘EC’.

3.1.1 Assessment aggregation and conversion into crisp values

An individual expert assigns a Neutrosophic value to their decision for each evaluation criterion, EC. The expert's perspectives are further aggregated for computing the overall assessment. The aggregation process simply combines these individual assessments given by multiple experts. Let's consider N cyber-experts who are offering their assessment for an element ‘x’. Each cyber-expert is assessing in terms of Neutrosophic value $(l b, m b, u b); T_{m} (x), I_{m} (x), F_{m} (x)$ . Here, ‘lb’, ‘mb’, and ‘ub’ denote lower-bound, mid-bound, and upper-bound, respectively. Expert's assessment aggregation $(E A)$ can be performed using equation (2).

\begin{aligned} E A = \frac{\sum_{i = 1}^{N} < (l b_{i}, m b_{i}, u b_{i}); T_{m_{i}}, I_{m_{i}}, F_{m_{i}} >}{N} \end{aligned}

(2)

These aggregated values will again be a Neutrosophic value. Equation (3) is used to transform these Neutrosophic values into crisp values $(C_{E A}) .$

\begin{aligned} C_{E A} = | (l b \times m b \times u b) \frac{T_{m} + I_{m} + F_{m}}{9} | \end{aligned}

(3)

These crisp values will be further used by the COCOSO model to prioritize the IDS software tools. These crisp values will be used in the formation of dataset.

3.2 Best-Worst Method (BWM)

BWM is an MCDM approach that is utilized to determine the priority weights of evaluation criteria by only making comparisons among the most significant and least significant criteria alongside other criteria.^38,39 However, AHP requires many pairwise comparisons while deriving weights to the evaluation criteria. This reduces redundancy and the mental efforts of experts which further reduces the chances of inconsistency. BWM expresses the problem as a mathematical optimization problem that helps to minimize the extreme deviation between the expert's pairwise comparisons. This ensures that the derived weights are also mathematically consistent. It comprises a consistency ratio which measure the integrity of expert judgments. It provides dependable weights, that strengthen ranking mechanism of COCOSO. Algorithm-1 gives a step-by-step description for computing priority weights using BWM.

3.3 COCOSO MCDM approach

COCOSO is introduced to provide a solution for the complex multi-variable evaluation problems. This MCDM approach combines different viewpoints of compromise programming and the Weighted Sum and Product models.^40,41 The decision-making community has significantly acknowledged this approach. Algorithm-2 discusses the step-by-step procedure for COCOSO, which uses the crisp values ‘ $C_{E A}$ ’ from equation (3) and weights ‘PW’ from Algorithm-1 for each evaluation criterion ‘EC’.

4 The simulation of proposed NB-COCOSO framework

This study simulates the proposed decision support system to assess the IDS tools based on different criteria. The proposed decision support system was executed using multiple libraries of Python, such as Pandas, Statistics, NumPy, etc. Pandas’ library is utilized for data preprocessing, normalizing the decision matrix, and applying different MCDM methods. The NumPy library is utilized for data representation, vector normalization, applying weights to criteria, and other statistical operations. This study uses Spearman's correlation ranking for validation of the proposed work, which is implemented using NumPy arrays and statistics. The proposed NB-COCOSO works in 4 phases as illustrated in Figure 2. This section discusses the detailed simulation of each phase by considering a case study.

Figure 2.

Simulation phases of the proposed NB-COCOSO.

4.1 Phase 1: Data collection

Phase 1 is simulated in four steps as shown in Figure 2.

Step 1: This study identified the evaluation criteria to ensure a comprehensive evaluation of these IDS tools across various functional domains. The four criteria (EC₁, EC₂, EC₃ and EC₄) - Categories (open source/closed source/freeware), Log source locations (host log files, application log files, network packets, sensor alerts), Targets (network, application, host), Monitored system(NIDS, HIDS, hybrid)-are aligned with classifications used in state-of-the-art,^29,42,43 strategies from NIST,^44,45 insights from cyber experts. These criteria consider not only the technical abilities but also the real-time deployment deliberations industries face while selecting an IDS.

A total of N= 40 experts were selected as per their professional backgrounds and domain knowledge, with expertise in security, MCDM techniques, or IDS tool deployment. The process of expert selection considered both academic qualifications and practical knowledge, ensuring that the committee included individuals with at least 15 years of research/industry experience, related publications, or involvement in IDS software development projects. Think Tank meetings were conducted to finalize the evaluation criteria so that the optimal selection of the IDS software tool can be conducted. Figure 3 illustrates the hierarchical structure of the evaluation of IDS tools used in this study. This section discusses the significance and meaning of each criterion and sub criteria in detail.

Figure 3.

Alternatives and evaluation criteria used in simulation.

Categories (EC₁): IDS software tools can be differentiated in terms of categories. They are either open source, closed source, or freeware.

Open Source (EC₁₁)- The tools for which the source code is publicly and freely available. The source code can be modified and distributed.

Closed Source (EC₁₂)-The tools for which the source code is not publicly and freely available. The code can be implemented and distributed by only specific vendors.

Freeware (EC₁₃)-The tools for which the software is freely available and can be distributed without any cost. However, the source code is not publicly available for modifications.

Log Source Location (EC₂): It defines the points in a network from where data is gathered to detect intrusions. It plays a crucial role in monitoring and analysing traffic and other system activities.

Host Log Files (EC₂₁)- These files capture information about the activities and security events happening on a particular host.

Application Log Files (EC₂₂)- These files capture events, user activities, transactions, and errors.

Network Packets (EC₂₃)- These are the raw data transferred across a network, which operate as a primary source for causing network traffic logs.

Sensor Alerts (EC₂₄)- These are the records generated by sensors or monitoring devices installed across a network. These sensors continuously monitor traffic for detecting suspicious and malicious activity and generating alerts.

Targets (EC₃): They signify those entities that IDS monitors for anomalies and malicious activities. It can be categorized into 3 categories:

Network (EC₃₁): It includes the traffic and devices that are on the network.

Application (EC₃₂): It focus on observing applications for their transactions, and data flows.

Host (EC₃₃): It refers to the individual devices or servers that are being monitored for malicious activities.

Monitored Systems (EC₄): It specifies the infrastructure, entity, or environment that IDS monitors to detect malicious activities or threats. These systems can be classified according to the kind of IDS deployed.

NIDS (EC₄₁): It monitors traffic throughout the network or a segment of the network to identify threats that try to exploit vulnerabilities during communication over the network.

HIDS (EC₄₂): It monitors an individual host for malicious activities. These activities can be in system logs, files, applications, etc., which can change the behaviour of the host.

Hybrid (EC₄₃): It combines host and network-based monitoring to link data from several sources. It can improve the detection accuracy.

The 12 IDS software tools (Zeek, Splunk, OSSEC, Snort, Sagan, Suricata, Security Onion, Tripwire, ManageEngine Log360, Gatewatcher, CrowdSec, and AIDE)/ alternatives (Al₁, Al₂…Al₁₂) were selected to signify an extensive scope of IDS technologies that are aligned with both academia and industry. Thereafter, consistency checks are applied to the data obtained from the experts by conducting the pairwise comparison to identify contradictions or biases in the responses of the experts. Statistical measures, such as Spearman's rank correlation, are used to determine the degree of concordance among the experts, wherever required. Table 3 summarizes the key functionalities of each of these software tools suggested by the experts for this study.^29,46

Table 3.

Summary of IDS software tools.

IDS Software Tools	Monitored System	Category	Key Functions
Zeek (Al₁)	NIDS	Open Source + Freeware	• Network Traffic monitoring and Analysis • Intrusion Detection • Log Correlation
Splunk (Al₂)	Hybrid	Closed Source + Freeware (partially)	• Anomaly Detection • Log & Event Management • SIEM • Visualization • Compliance Reportage
OSSEC (Al₃)	HIDS	Open Source + Freeware	• Host Intrusion detection • Log Analysis • File Reliability Monitoring
Snort (Al₄)	NIDS	Open Source + Freeware	• Packet Assessment • Protocol Analysis • Threat Detection • Alerting
Sagan (Al₅)	NIDS	Open Source + Freeware	• Network Amalgamation • Real-time Log Analysis • Threat detection & Correlation • Alerting
Suricata (Al₆)	NIDS	Open Source + Freeware	• Packet Assessment • Anomaly Detection • Detection and prevention ability • Scalability
Security Onion (Al₇)	Hybrid	Open Source + Freeware	• Extensive monitoring • Suspicious incident detection & Analysis • Functionalities of both HIDS & NIDS
Tripwire (Al₈)	HIDS	Open Source + Freeware (Community)/Closed Source	• File reliability Monitoring • Incident Assessment • Configuration Management
ManageEngine (Log 360) (Al₉)	Hybrid	Closed Source	• Log Analysis • Threat detection & Activity Monitoring • SIEM Amalgamation
Gatewatcher (Al₁₀)	NIDS	Closed Source	• Threat Detection • Network Monitoring • Alerting • Malware Detection
CrowdSec (Al₁₁)	Hybrid	Open Source + Freeware	• Behaviour Analysis • Threat Detection
AIDE (Al₁₂)	HIDS	Open Source + Freeware	• File Reliability Monitoring • Simple & Light Weight • System Integrity Authentication

Step 2: Experts are advised to give judgments using linguistic scale such as equally pivotal, slightly pivotal etc., which were represented using the Neutrosophic scale⁴⁷ (refer to section 3.1) as mentioned in Table 4.

Table 4.

Neutrosophic scale.

Score	Linguistic Phrase	NTS
1	Equally Pivotal (EP)	<(11,1); (0.50,0.50,0.50)>
3	Slightly Pivotal (SP)	<(23,4); (0.30,0.75,0.70)>
5	Strongly Pivotal (SrP)	<(45,6); (0.80,0.15,0.20)>
7	Very strongly Pivotal (VSrP)	<(67,8); (0.90,0.10,0.10)>
9	Absolutely Pivotal (AP)	<(99,0); (1.00,0.00,0.00)>
2	Sporadic values between two close scales (SpV)	<(12,3); (0.40,0.60,0.65)>
4		<(34,5); (0.35,0.60,0.40)>
6		<(56,7); (0.70,0.25,0.30)>
8		<(78,9); (0.85, 0.10,0.15)>

The integration of NFS in the framework provides an intrinsic mechanism to capture uncertainty, indeterminacy, and hesitation in expert ratings, which increases overall reliability. Use of NFS itself minimizes the impact of individual subjectivity.

Step 3: Thereafter, the framework performed the Expert's assessment aggregation (EA) using a weighted arithmetic mean as given in equation (2) on the assessments provided by the experts. These aggregated values were corroborated by making comparisons across distinct subsets of experts. Higher correlation indicates robustness; however, deviations were further analysed. Due to the space limitation, EA values for only 3 IDS tools are mentioned in Table 5.

Table 5.

Expert’ assessment aggregation for different evaluation criteria.

EC/Al	Zeek	OSSEC	Snort
EC₁₁	<7.9,8.4,4.4;0.93,0.05,0.07>	<8.2,8.6,3.6;0.94,0.04,0.06>	<8.2,8.6,3.6;0.94,0.04,0.06>
EC₁₂	<1.0,1.4,1.8;0.46,0.54,0.56>	<1,1.4,1.8; 0.46,0.54,0.56>	<1.7,2.7,3.7;0.33,0.71,0.69>
EC₁₃	<6.0,7.0,8.0;0.85,0.13,0.15>	<8.2,8.6,3.6;0.94,0.04,0.06>	<8.0,8.5,4.5;0.93,0.05,0.08>
EC₂₁	<2.3,3.3,4.3;0.33,0.68,0.58>	<6.5,7.5,8.5;0.88,0.1,0.13>	<2.5,3.5,4.5;0.33,0.68,0.55>
EC₂₂	<3.4,4.4,5.4;0.57,0.39,0.33>	<5.6,6.6,7.6;0.82,0.16,0.18>	<1.6,2.6,3.6;0.34,0.69,0.68>
EC₂₃	<8.0,8.5,4.5;0.93,0.05,0.08>	<2.5,3.5,4.5;0.33,0.68,0.55>	<7.8,8.4,3.6;0.91,0.06,0.09>
EC₂₄	<5.7,6.7,7.7;0.82,0.16,0.19>	<4.4,5.4,6.4;0.76,0.19,0.24>	<6.5,7.5,8.5;0.88,0.1,0.13>
EC₃₁	<7.7,8.3,5.3;0.83,0.06,0.08>	<1.5,2.5,3.5;0.35,0.68,0.68>	<8.2,8.6,3.6;0.94,0.04,0.06>
EC₃₂	<5.6,6.6,7.6;0.8,0.18,0.21>	<2.5,3.5,4.5;0.33,0.68,0.55>	<2.6,3.6,4.6;0.33,0.66,0.45>
EC₃₃	<2.5,3.5,4.5;0.33,0.68,0.55>	<7.8,8.4,3.6;0.91,0.06,0.09>	<1.7,2.7,3.7;0.33,0.71,0.69>
EC₄₁	<7.7,8.3,5.3;0.92,.0.06,0.09>	<1.6,2.6,3.6;0.34,0.69,0.68>	<8.2,8.6,3.6;0.94,0.04,0.06>
EC₄₂	<4.4,5.4,6.4;0.71,0.79,0.27>	<8.2,8.6,3.6;0.94,0.04,0.06>	<1.0,1.6,2.2;0.44,0.56,0.59>
EC₄₃	<5.5,6.5,7.5;0.8,0.175,0.2>	<4.6,5.6,6.6;0.74,0.21,0.26>	<2.6,3.6,4.6;0.33,0.66,0.45>

Step 4: These aggregated Neutrosophic values EA were subsequently converted into crisp values (C_EA) using equation (3). These are the final values that are used to make the decisions to assess the IDS software tools and used as final decision matrix for the COCOSO approach. These final experts’ ratings are mentioned in Table 6. Finally, the dataset includes 12 IDS tools evaluated against 13 predefined criteria as mentioned in Figure 3.

Table 6.

Final aggregated expert's ratings assigned to each IDS tool.

Al	EC₁₁	EC₁₂	EC₁₃	EC₂₁	EC₂₂	EC₂₃	EC₂₄	EC₃₁	EC₃₂	EC₃₃	EC₄₁	EC₄₂	EC₄₃
Al₁	34.07	0.44	42.19	5.73	11.57	35.70	37.90	36.32	36.67	6.78	39.89	29.74	35.02
Al₂	29.34	0.44	29.34	50.65	36.20	6.78	20.11	2.48	6.78	27.78	2.85	29.34	22.86
Al₃	29.34	3.25	35.70	6.78	2.85	27.78	50.65	29.34	6.89	3.25	29.34	0.62	6.89
Al₄	29.34	0.44	35.70	6.89	11.40	27.78	52.72	35.70	21.45	6.89	35.70	2.85	12.67
Al₅	39.89	0.61	42.91	44.55	48.62	8.47	46.65	11.40	10.37	28.00	10.37	6.14	28.12
Al₆	34.23	0.67	24.54	41.66	42.75	22.58	34.07	21.19	33.53	42.19	22.58	28.00	22.58
Al₇	5.62	51.00	17.29	38.17	34.07	12.03	46.01	42.19	43.47	43.47	10.37	18.05	50.38
Al₈	43.47	44.05	26.38	46.01	28.00	1.64	18.14	4.92	9.87	35.70	1.64	22.58	10.37
Al₉	0.80	29.34	9.87	29.34	46.01	9.87	39.65	28.00	42.19	47.62	9.27	18.05	43.46
Al₁₀	0.44	22.58	2.76	29.08	16.46	48.38	45.96	29.34	30.82	45.35	29.34	4.40	30.82
Al₁₁	34.07	7.73	44.30	46.01	35.47	10.72	29.08	29.08	35.47	21.45	28.00	10.37	45.35
Al₁₂	0.93	35.70	44.96	37.63	17.48	0.55	3.06	0.36	2.10	29.34	0.36	29.34	10.37

4.2 Phase 2: Weight calculation

For each evaluation criterion, both local and global weights are calculated using the BWM method discussed under Algorithm 1. This study first discusses the steps for computing weights at the main criteria level, i.e., for EC₁ to EC₄. Best and worst criteria were selected as $E C_{B} = E C_{2}$ and $E C_{W} = E C_{1}$ respectively. After that, a pairwise comparison was performed, and the two vectors: best among others $(B_{O})$ and others to the worst $(O_{W})$ were built.

\begin{aligned} B_{O} & = [9, 1, 2, 4] \end{aligned}

(4)

\begin{aligned} O_{W} & = [1, 9, 8, 6] \end{aligned}

(5)

Afterwards, the problem was formulated as an optimization problem to generate weights for each evaluation criterion, and the vector PW was initialized as below:

P W = [0.0418, 0.477, 0.1925, 0.2887]

Consistency conditions were checked for the generated weights. The pair-wise comparison consistency level is acceptable with a value of 0.1667 and an associated threshold value of 0.2681. Likewise, the local weights were computed for all sub-categories as mentioned under phase 2.

\begin{aligned} P W_{1} & = [0.0625, 0.2625, 0.6750] \end{aligned}

\begin{aligned} P W_{2} & = [0.0418, 0.4770, 0.1925] \end{aligned}

\begin{aligned} P W_{3} & = [0.6029, 0.3382, 0.0588] \end{aligned}

\begin{aligned} P W_{4} & = [0.2625, 0.0625, 0.6750] \end{aligned}

Global weights are computed for each evaluation criterion, and the optimal weight vector PW is updated accordingly. The weights of vector PW satisfied the criteria $\sum_{i = 1}^{13} p w_{i} = 1$ .

P W = [0.0026, 0.0110, 0.0282, 0.0200, 0.2275, 0.0918, 0.1377, .01160, 0.0651, 0.0113, 0.0758, 0.0180, 0.1949]

4.3 Phase 3: Rank calculation

In phase 3, ranks were calculated for each IDS tool $(A l_{i}, i \in {1 \dots n})$ using the COCOSO approach using Algorithm 2. Firstly, the decision matrix $[d_{i j}]$ was created with the crisp values (C_EA) produced in phase 1.

[d_{i j}]_{n \times m} = [\begin{array}{ccccccccccccc} 34.07 & 00.44 & 42.19 & 36.67 & 36.32 & 06.78 & 29.74 & 39.89 & 35.02 & 05.73 & 37.90 & 35.70 & 11.57 \\ 29.34 & 00.44 & 29.34 & 06.78 & 02.48 & 27.78 & 29.34 & 02.85 & 22.86 & 50.65 & 20.11 & 06.78 & 36.20 \\ 29.34 & 03.25 & 35.70 & 06.89 & 29.34 & 03.25 & 00.62 & 29.34 & 06.89 & 06.78 & 50.65 & 27.78 & 02.85 \\ 29.34 & 00.44 & 35.70 & 21.45 & 35.70 & 06.89 & 02.85 & 35.70 & 12.67 & 06.89 & 52.72 & 27.78 & 11040 \\ 39.89 & 00.61 & 42.91 & 10.37 & 11.40 & 28.00 & 06.14 & 10.37 & 28.12 & 44.55 & 46.65 & 08.47 & 48.62 \\ 34.23 & 00.67 & 24.54 & 33.53 & 21.19 & 42.19 & 28.00 & 22.58 & 22.58 & 41.66 & 34.07 & 22.58 & 42.75 \\ 05.62 & 51.00 & 17.29 & 43.47 & 42.19 & 43.47 & 18.05 & 10.37 & 50.38 & 38.17 & 46.01 & 12.03 & 34.07 \\ 43.87 & 44.05 & 26.38 & 09.87 & 04.92 & 35.70 & 22.58 & 01.64 & 10.37 & 46.01 & 18.14 & 01.64 & 28.00 \\ 00.80 & 29.34 & 09.87 & 42.19 & 28.00 & 47.62 & 18.05 & 09.27 & 43.46 & 29.34 & 39.65 & 09.87 & 46.01 \\ 00.44 & 22.58 & 02.76 & 30.82 & 29.34 & 45.35 & 04.40 & 29.34 & 30.82 & 29.08 & 45.96 & 48.38 & 16.46 \\ 34.07 & 07.73 & 44.30 & 35.47 & 29.08 & 21.45 & 10.37 & 28.00 & 45.35 & 46.01 & 29.08 & 10.72 & 35.47 \\ 00.93 & 35.70 & 44.96 & 02.10 & 00.36 & 29.34 & 29.34 & 00.36 & 10.37 & 37.63 & 03.06 & 00.55 & 17.48 \end{array}]

Here, n is the number of IDS software tools $(n = 12)$ and m is the number of evaluation criteria (EC) $(m = 13)$ . In step 1, evaluation criteria were segregated in terms of beneficiary and non-beneficiary criteria. The two ECs- EC₂₂ and EC₂₃ are considered as non-beneficiary evaluation criteria, and the rest are considered as beneficiary evaluation criteria. These two non-beneficiary criteria are now the rightmost entries in $[d_{i j}]$ . The matrix $[d_{i j}]_{n \times m}$ is normalized as per the equations mentioned in Algorithm 2 for beneficiary and non-beneficiary criteria.

{d_{i j}}^{n o r m} = [\begin{array}{ccccccccccccc} 00.78 & 00.00 & 00.93 & 00.84 & 00.86 & 00.08 & 01.00 & 01.00 & 00.65 & 00.00 & 00.70 & 00.27 & 00.81 \\ 00.67 & 00.00 & 00.63 & 00.11 & 00.05 & 00.55 & 00.99 & 00.06 & 00.37 & 01.00 & 00.34 & 00.87 & 0027 \\ 00.67 & 00.06 & 00.78 & 00.12 & 00.69 & 00.00 & 00.00 & 00.73 & 00.00 & 00.02 & 00.96 & 00.43 & 01.00 \\ 00.67 & 00.00 & 00.78 & 00.47 & 00.84 & 00.08 & 00.08 & 00.89 & 00.13 & 00.03 & 01.00 & 00.43 & 00.81 \\ 00.92 & 00.00 & 00.95 & 00.20 & 00.26 & 00.56 & 00.19 & 00.25 & 00.49 & 00.86 & 00.88 & 00.83 & 00.00 \\ 00.79 & 00.00 & 00.52 & 00.76 & 00.20 & 00.88 & 00.94 & 00.56 & 00.36 & 00.80 & 00.62 & 00.54 & 00.13 \\ 00.12 & 01.00 & 00.34 & 01.00 & 01.00 & 00.91 & 00.60 & 00.25 & 01.00 & 00.72 & 00.86 & 00.76 & 00.32 \\ 01.00 & 00.86 & 00.56 & 00.19 & 00.11 & 00.73 & 00.75 & 00.03 & 00.08 & 00.90 & 00.30 & 00.98 & 00.45 \\ 00.01 & 00.67 & 00.17 & 00.97 & 00.66 & 01.00 & 00.60 & 00.23 & 00.84 & 00.53 & 00.74 & 00.81 & 00.06 \\ 00.00 & 00.44 & 00.00 & 00.69 & 00.69 & 00.95 & 00.13 & 00.73 & 00.55 & 00.52 & 00.86 & 00.00 & 00.70 \\ 00.78 & 00.14 & 00.98 & 00.81 & 00.69 & 00.41 & 00.33 & 00.70 & 00.88 & 00.90 & 00.52 & 00.79 & 00.29 \\ 00.01 & 00.70 & 01.00 & 00.00 & 00.00 & 00.59 & 00.99 & 00.00 & 00.08 & 00.71 & 00.00 & 01.00 & 00.68 \end{array}]

In the next step, the weighted normalized matrix $w d_{i j}$ was created by multiplying each value of ${d_{i j}}^{n o r m}$ with $p w_{j}$ . Here, $P W = [0.0026, 0.0110, 0.0282, 0.0200, 0.2275, 0.0918, 0.1377, .01160, 0.0651, 0.0113, 0.0758, 0.0180, 0.1949]$ which was taken from phase 2.

w d_{i j} = [\begin{array}{ccccccccccccc} 0.002 & 0.000 & 0.026 & 0.054 & 0.100 & 0.001 & 0.018 & 0.076 & 0.126 & 0.000 & 0.097 & 0.060 & 0.074 \\ 0.002 & 0.000 & 0.018 & 0.007 & 0.006 & 0.006 & 0.018 & 0.005 & 0.072 & 0.020 & 0.047 & 0.198 & 0.025 \\ 0.002 & 0.001 & 0.022 & 0.008 & 0.080 & 0.000 & 0.000 & 0.056 & 0.000 & 0.000 & 0.132 & 0.098 & 0.092 \\ 0.002 & 0.000 & 0.022 & 0.030 & 0.098 & 0.001 & 0.001 & 0.068 & 0.026 & 0.001 & 0.138 & 0.098 & 0.075 \\ 0.002 & 0.000 & 0.027 & 0.013 & 0.031 & 0.006 & 0.003 & 0.19 & 0.095 & 0.017 & 0.121 & 0.190 & 0.000 \\ 0.002 & 0.000 & 0.015 & 0.049 & 0.058 & 0.010 & 0.017 & 0.043 & 0.070 & 0.016 & 0.086 & 0.123 & 0.012 \\ 0.000 & 0.011 & 0.010 & 0.065 & 0.116 & 0.010 & 0.011 & 0.019 & 0.195 & 0.014 & 0.119 & 0.173 & 0.029 \\ 0.003 & 0.009 & 0.016 & 0.012 & 0.013 & 0.008 & 0.014 & 0.002 & 0.016 & 0.018 & 0.042 & 0.222 & 0.041 \\ 0.000 & 0.006 & 0.005 & 0.063 & 0.077 & 0.011 & 0.011 & 0.017 & 0.164 & 0.010 & 0.101 & 0.183 & 0.005 \\ 0.000 & 0.005 & 0.000 & 0.045 & 0.080 & 0.011 & 0.002 & 0.056 & 0.107 & 0.010 & 0.119 & 0.000 & 0.065 \\ 0.002 & 0.002 & 0.028 & 0.053 & 0.080 & 0.005 & 0.006 & 0.053 & 0.172 & 0.018 & 0.072 & 0.179 & 0.026 \\ 0.000 & 0.008 & 0.028 & 0.000 & 0.000 & 0.007 & 0.018 & 0.000 & 0.016 & 0.014 & 0.000 & 0.228 & 0.062 \end{array}]

After that $S_{i}$ and $P_{i}$ were computed by aggregating the sum and product of weighted normalized values. The final aggregated score was computed using $a {s_{i}}^{1}$ , $a {s_{i}}^{2}$ and $a {s_{i}}^{3}$ scores. The equations for the same are mentioned in Algorithm 2. Later, the final aggregated score was computed as given in equation (6). The value for each computed score for ECs is mentioned in Table 7.

\begin{aligned} A g g_{i} = (a {s_{i}}^{1} * a {s_{i}}^{2} * a {s_{i}}^{3})^{1 / 3} + \frac{1}{3} (a {s_{i}}^{1} + a {s_{i}}^{2} + a {s_{i}}^{3}) \end{aligned}

(6)

Table 7.

Aggregated scores.

Al	$a {s_{i}}^{1}$	$a {s_{i}}^{2}$	$a {s_{i}}^{3}$	$A g g_{i}$
Zeek	0.08	2.90	0.45	1.61
OSSEC	0.08	2.39	0.46	1.42
Snort	0.07	2.40	0.40	1.37
Suricata	0.08	2.79	0.48	1.60
Sagan	0.08	2.71	0.48	1.57
Security Onion	0.09	2.75	0.51	1.62
Splunk	0.10	3.52	0.54	1.95
Tripwire	0.09	2.47	0.49	1.49
Manage Engine	0.09	3.17	0.53	1.80
Gate Watcher	0.07	2.45	0.41	1.40
CrowdSec	0.09	3.30	0.53	1.86
AIDE	0.06	2.00	0.36	1.17

The alternative/Al with the highest $A g g_{i}$ value is ranked as first, and the rest Als are ranked accordingly. With highest $A g g_{i} = 1.95$ Splunk is identified as the best IDS software tool.

4.4 Phase 4: Methodology validation

Ensure a more comprehensive validation, the proposed framework has been benchmarked in comparison with multiple widely recognised MCDM techniques that are employed in state-of-the-art for IDS evaluation, including TOPSIS, DBA, VIKOR, and COPRAS. These models were selected because they signify diverse decision-making strategies and have well recognized and validated in existing IDS assessment literature. For quantitative validation, this study conducted a comparative performance analysis among all considered methods using the same dataset and evaluation criteria. The results prove that the proposed NB-COCOSO framework ensures more consistent rankings and lower susceptibility to data normalization when compared with traditional methods. Table 8 presents the ranking details of each of the MCDM approaches.

Table 8.

Rank comparison among state-of-the-art methodologies.

IDS Software (Al)	NB-COCOSO	COPRAS	TOPSIS	DBA	VIKOR
Zeek	5	5	10	2	5
OSSEC	9	12	1	10	8
Snort	11	11	12	9	9
Suricata	6	7	11	7	7
Sagan	7	9	3	8	4
Security Onion	4	10	5	6	6
Splunk	1	1	6	1	1
Tripwire	8	8	8	11	10
Manage engine (Log 360)	3	4	4	5	3
Gatewatcher	10	6	9	3	11
CrowdSec	2	3	2	4	2
AIDE	12	2	7	12	12

To further validate consistency and reliability, this study conducted a Spearman's rank correlation test among the rankings generated by NB-COCOSO and those obtained from other MCDM techniques. This is an important test to validate results and to ensure the strength of the decision-making process. The Spearman's rank correlation coefficient is computed as given in equation (7).

\begin{aligned} ρ_{c} = 1 - \frac{6 \sum {d_{i}}^{2}}{n (n^{2} - 1)} \end{aligned}

(7)

Here, $d_{i}$ refers to the difference between the ranks of two MCDM approaches. The correlation can be interpreted as based on the values $ρ_{c} = 1$ : strong positive correlation, $ρ_{c} = - 1$ : negative correlation, $ρ_{c} = 0$ : no correlation and $0 < ρ_{c} < 1$ : positive correlation. Figure 4 illustrates the results of Spearman's rank correlation coefficient among NB-COCOSO with other considered approaches.

Figure 4.

Spearman's rank correlation coefficient values.

Further, to strengthen the research findings, this study also performed a sensitivity analysis to check the impact of weights and evaluation criteria on the ranking.

4.4.1 Sensitivity analysis for weight parameters

Sensitivity analysis was performed for the proposed NB-COCOSO framework by excluding the weights derived using BWM, instead assigning equal importance to all evaluation criteria, i.e., considering each weight as 1. This analysis facilitates observing the robustness of the framework by identifying whether the final ranking of IDS tools is impacted by the expert-assigned weight or remains consistent even with identical weights across criteria. Such a comparison provides a more comprehensive insight into the stability of the decision-making process and validates that the proposed framework does not excessively depend on weight variations, thus improving its reliability and real-world applicability. The comparison results are shown in Figure 5. To further strengthen the results, Spearman's correlation coefficient test was conducted between the ranking of BWM-based weights and equal weights.

Figure 5.

Sensitivity analysis for parameter ‘weight’.

4.4.2 Sensitivity analysis for evaluation criteria

The sensitivity analysis was augmented to examine the impact of each evaluation criterion (EC₁, EC₂, EC₃, EC₄) on the ranking of IDS software tools by systematically removing them one at a time. Starting with the elimination of EC₁, the rankings were recalculated, followed by the exclusion of EC₂, and so on until all evaluation criteria had assessed independently. The comparison results of ranking for each case are illustrated in Figure 6.

Figure 6.

Sensitivity analysis for evaluation criteria.

The Spearman's rank correlation coefficient test was conducted across all cases with the original ranking, considering all evaluation criteria. Coefficient values are mentioned in Table 9.

Table 9.

Results of Spearman's correlation test.

Removal of Evaluation Criteria	Spearman's Rank correlation Coefficient ( $ρ)$
EC₁	0.92
EC₂	0.88
EC₃	0.89
EC₄	0.80

5 Results and discussions

Hybrid MCDM was used for the evaluation of IDS software tools based on an integration of Neutrosophic BWM and COCOSO to effectively manage uncertainty as well as the subjectivity arising from weighting and ranking. The proposed NB-COCOSO framework efficiently deals with the uncertainty and subjective judgments of experts during the ranking process. The BWM approach was implemented to derive the weights for the evaluation criteria to improve the consistency, accuracy, and reliability. Subsequently, the COCOSO approach aggregated the scores for evaluation criteria and generated the final ranking of IDS software tools.

5.1 Selection of IDS software tools

The present research considered the 12 different well-known IDS Software tools used by organizations. In state-of-the-art research, no research study considered all these well-known IDS Tools collectively. Many of these tools are not even considered in the state-of-the-art (refer to Table 1).

5.2 Analysis of evaluation criteria weights determination using BWM

The initial phase of the analysis included expert opinions to prioritize evaluation criteria. Section 4.2 presents the computed local and optimal weights for each evaluation criterion. The evaluation criterion 'Hybrid' has the highest weight as 0.195, and 'open source' has the lowest weight as 0.0026 among 13 evaluation criteria. The pairwise comparison consistency ratio (0.1667) for all evaluation criteria is acceptable, which is below the threshold value of 0.2681. Consistency ratio for all sub-categorized evaluation criteria is also acceptable (EC₁–0.12, EC₂–0.16, EC₃–0.07, EC₄–0.125), which signifies the reliable weight calculation.

5.3 Analysis of the ranking of IDS software tools using COCOSO

After the determination of evaluation criteria weights, these 12 IDS tools were evaluated and ranked through the COCOSO method. Table 7 depicts the different scores over the evaluation criteria and the final aggregated compromise scores, which provide the final ranking. The ranks illustrated in Table 8 depict that the IDS software tool ‘Splunk’ is at the top rank with the maximum aggregated score as 1.95, and ‘AIDE’ at the last with the minimum score as 1.17. The IDS tool ‘Crowdsec’ got the second position, followed by ‘Manage Engine’ at the third position. The tools ‘Snort’, ‘Suricata’, and ‘Security Onion’ got the 11^th, 6^th, and 4^th rank with the aggregated score values 1.37, 1.60, and 1.62, respectively.

5.4 Analysis of Spearman's correlation test

This study implemented Spearman's correlation test to validate the proposed methodology with the other state-of-the-art methodologies. As discussed in section 4.4, this value must satisfy $- 1 \leq ρ_{c} \leq 1$ . These statistical values from Figure 4 describe the positive relationship among the ranks gained from NB-COCOSO with other methodologies (NB-COCOSO/COPRAS), (NB-COCOCO/TOPSIS), (NB-COCOSO/DBA), and (NB-COCOSO/VIKOR). However, the NB-COCOSO and VIKOR show a strong positive relationship with $ρ_{c} = 0.92,$ as the target of both approaches is to find a compromise that is nearest to the ideal one. Both approaches assess alternatives based on the best and worst solutions. Likewise, NB-COCOSO and DBA also depict a strong positive relationship with $ρ_{c} = 0.7,$ as the COCOSO method uses aggregation for weights and the DBA method relies on the dynamic nature of weights. The correlation values were consistently high and positive, affirming that the proposed framework is consistent with these well-recognised techniques, while still offering enhanced robustness. Additionally, pairwise rank correlation comparisons emphasize that the NB-COCOSO model maintains higher consistency. These results validate that the proposed framework is not only inconsistent with the state-of-the-art MCDM techniques but also offers measurable enhancements in ranking reliability, consistency, and robustness of decision-making for IDS software tool selection.

5.4 Analysis of sensitivity analysis

To comprehensively evaluate the consistency of the proposed NB-COCOSO framework, a two-fold sensitivity analysis was conducted by varying the weight of evaluation criteria and systematically removing evaluation criteria.

5.1.1 Sensitivity for parameter weight

From Figure 5, it can be analysed that the rankings attained with equal weights (initialising PW vector with 1) are majorly consistent with those obtained from the BWM. IDS tools ‘Splunk’ and ‘CrowdSec’ attained the same first and second rank in both scenarios. However, minor deviations are observed in specific positions, such as a swap between the rankings of ‘Manage Engine’ and ‘Zeek.’ This analysis demonstrates that the proposed framework is relatively stable, as the overall ranking of tools remains largely unaffected by changes in the weighting approach. The minor differences depict the influence of expert-assigned weights in fine-tuning the ranking.

The Spearman's rank correlation coefficient between the BWM-based ranking and the equal-weight ranking was observed as ρ = 0.9, which confirms the robustness and reliability of the overall ranking of IDS tools.

5.5.2 Sensitivity for evaluation criteria

Sensitivity analysis is extended to verify the impact of sequential removal of evaluation criteria on the ranking results of NB-COCOSO. The results in Figure 6 demonstrates that the overall ranking is majorly stable with the proposed framework in most cases. However, few positional changes are observed for IDS tools when the evaluation criteria are removed. Like, ‘Tripwire’ was originally at the 8th position by NB-COCOSO. However, after removing the EC1, the tool is at the 10th rank. The tool has 11th, 6th, and 5th rank after removing criteria EC2, EC3, and EC4, respectively. The results depict that the NB-COCOSO framework is majorly stable and not excessively dependent on any single evaluation criterion. However, in some cases, the greater variations are observed in the absence of certain evaluation criteria, indicating their stronger distinguishing influence in fine-tuning the ranking of IDS software tools.

This consistency, further validated by high Spearman's rank correlation values mentioned in Table 9 for all cases, emphasizes that the NB-COCOSO framework is stable and not excessively dependent on any single criterion.

5.5 Discussion

The results emphasize the superiority of the IDS software tool Splunk over other IDS software tools in terms of overall performance. The BWM efficiently acquired weights for evaluation criteria, generating more realistic weight distributions. The Neutrosophic approach captured expert hesitation and uncertainty while rating the IDS software tools concerning evaluation criteria, yielding more genuine ratings among IDS tools. Meanwhile, the COCOSO method assisted a comprehensive aggregation of evaluation criteria, balancing conflicting goals.

Spearman's correlation ranking test depicts the positive relationship between the NB-COCOSO and other state-of-the-art techniques, which proves the strength of the proposed framework.

The sensitivity analysis further emphasized the reliability of the proposed integrated framework NB-COCOSO, by demonstrating the ranking results concerning the dependency on the weight parameter and the dependency on the evaluation criteria. This validates the feasibility of the proposed framework for selecting optimal IDS software tools in uncertain environments.

In conclusion, the integrated NB-COCOSO model proves to be a robust decision-support system for evaluating IDS software tools. It helps organizations in selecting the optimal solution for enhanced cybersecurity.

6 Conclusion and future work

This study aimed to identify the optimal IDS software tool among twelve different IDS tools using an integrated decision-making framework called NB-COCOSO. It combines Neutrosophic, Best Worst Method, and COCOSO MCDM approach. The BWM was utilized to determine the comparative importance of evaluation criteria, while Neutrosophic was used to manage the uncertainty among the experts’ opinions, and the COCOSO method was employed to provide a rank to the IDS software tools as per their performance. Validate the robustness and strength of the proposed integrated methodology, Spearman's correlation ranking test was applied to different state-of-the-art methods. The positive correlation coefficient value confirmed a strong correlation between the proposed methodology and the state-of-the-art methods, which demonstrates the consistency and reliability of the proposed approach. Furthermore, two types of sensitivity analysis were conducted to inspect the impact of evaluation criterion weights and to examine the impact of types of evaluation criteria on the final ranking results, which proved the stability of the proposed decision-making framework.

The results of this study give valuable insights for cybersecurity professionals in selecting the most effective and efficient IDS software tool based on multiple evaluation criteria. Future work can extend this integrated approach by incorporating other evaluation criteria and by exploring other MCDM approaches. It can be extended by validating the framework using benchmark IDS datasets (e.g., NSL-KDD, CICIDS2017, UNSW-NB15) to access the model's effectiveness.

Footnotes

ORCID iD

Supriya Raheja

Ethical approval and informed consent statements

Ethical approval was not required for this study, as it did not involve human participants, animal subjects, or sensitive data.

Consent to participate

not applicable

Consent for publication

not applicable

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

Zhao

Wang

Chen

. A comparative study of IDS systems based on machine learning techniques. J Inf Secu Appl 2022; 65: 103119.

Chen

Wang

. Evaluating intrusion detection systems with advanced machine learning techniques: a multi-criteria analysis. J Cybersecur Res 2022; 12: 185–202.

Kumar

Kaur

. Comparative analysis of machine learning-based intrusion detection systems in cybersecurity. Comput Secur J 2023; 15: 201–219.

Iftikhar

Azween

Abdullah S

. Towards the selection of best neural network system for intrusion detection. Int J Phys Sci 2010; 5: 1830–1839.

Alharbi

Seh

Alosaimi

, et al. Analyzing the impact of cyber security related attributes for intrusion detection systems. Sustainability 2021; 13: 12337.

Alhakami

. Evaluating modern intrusion detection methods in the face of gen V multi-vector attacks with fuzzy AHP-TOPSIS. PloS One 2024; 9: e0302559.

Hossain

Ali

Khan

. Enhancing decision-making in cybersecurity: a fuzzy MCDM approach for IDS selection. J Cyber Secur Mobility 2023; 12: 43–58.

Smarandache

. Neutrosophic theory and its applications in cybersecurity: advances and challenges. Int J Neutrosophic Sci 2023; 9: 78–91.

Salama

Smarandache

. Introduction to neutrosophic logic and set theory: foundations, methods, and applications. Int J Neutrosophic Sci 2022; 7: 89–101.

10.

Rezaei

. Best-worst method: recent advances and applications in multi-criteria decision-making. Int J Oper Res 2022; 23: 435–449.

11.

Zavadskas

Kaklauskas

Sarka

. The COPRAS method for multi-criteria decision-making. Int J Oper Res 1996; 10: 103–115.

12.

Zavadskas

Turskis

Medine

. Recent developments in the cocoso method for multi-criteria decision-making applications. Int J Decis Support Syst 2023; 14: 250–267.

13.

Milenkoski

Vieira

Kounev

, et al. Evaluating computer intrusion detection systems: a survey of common practices. ACM Comput Surv (CSUR) 2015; 48: 1–41.

14.

Schrötter

Scheffler

Schnor

. Evaluation of intrusion detection systems in IPv6 networks In: ICETE 2019, p. 408–416.

15.

Asghar

. Analysing performance issues of open-source intrusion detection systems in high-speed networks. J Inf Secur Appl 2022; 51: 102426.

16.

Waleed

Jamali

Masood

. Which open-source ids? Snort, suricata or zeek. Comput Netw 2022; 213: 109116.

17.

Wahyu

Fauziah

Nahrowi

, et al. Strengthening network security: evaluation of intrusion detection and prevention systems tools in networking systems. Int J Adv Comput Sci Appl 2023; 14: 1–10.

18.

Boukebous

AAE

Fettache

Bendiab

, et al. A comparative analysis of snort 3 and suricata. In: 2023 IEEE IAS Global Conference on Emerging Technologies (GlobConET), 2023, IEEE. pp.1–6.

19.

Ghazi

Hamid

Zaiter

, et al. Snort versus suricata in intrusion detection. Iraqi J Inf Commun Technol 2024; 7: 73–88.

20.

Chinnasamy

SivaKrishnaiah

Anjali

, et al. Managing network security in IT sector using the suricata. In: 2025 3rd International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS), 2025, IEEE, pp.1721–1728.

21.

Yang

. Evaluation of intrusion detection systems in cyber security using fuzzy OffLogic and MCDM approach. Neutrosophic Sets and Systems 2025; 85: 20.

22.

Almuseelem

. Perspective chapter: intrusion detection systems in cloud environment. IntechOpen 2025. DOI: 10.5772/intechopen.1008756.

23.

Tait

Khan

Alqahtani

, et al. Intrusion detection using machine learning techniques: an experimental comparison. In: 2021 International Congress of Advanced Technology and Engineering (ICOTEN), pp.1–10. IEEE.

24.

Albulayhi

Abu Al-Haija

Alsuhibany S

, et al. Iot intrusion detection using machine learning with a novel high performing feature selection method. Appl Sci 2022; 12: 5015.

25.

AbdulRaheem

Oladipo

Imoize

, et al. Machine learning assisted snort and zeek in detecting DDoS attacks in software-defined networking. Int J Inf Technol 2024; 16: 1627–1643.

26.

Seelammal

Devi

. Multi-criteria decision support for feature selection in network anomaly detection system. Int J Data Anal Tech Strat 2018; 10: 334–350.

27.

Belal M

Sundaram D

. An intelligent protection framework for intrusion detection in cloud environment based on covariance matrix self-adaptation evolution strategy and multi-criteria decision-making. J Intell Fuzzy Syst 2023; 44: 8971–9001.

28.

Almotiri

. Integrated fuzzy based computational mechanism for the selection of effective malicious traffic detection approach. IEEE Access 2021; 9: 10751–10764.

29.

Alyami

Ansari

MTJ

Alharbi

, et al. Effectiveness evaluation of different IDSs using integrated fuzzy MCDM model. Electronics 2022; 11: 859.

30.

Abushark Y

Irshad Khan

Alsolami

, et al. Cyber security analysis and evaluation for intrusion detection systems. Comput Mater Continua 2022; 72: 1765–1783.

31.

Abdel-Basset

Gamal

Sallam

, et al. An optimization model for appraising intrusion-detection systems for network security communications: applications, challenges, and solutions. Sensors 2022; 22: 4123.

32.

AbdelMouty

Abdel-Monem

. Neutrosophic MCDM methodology for assessment risks of cyber security in power management. Neutrosophic Syst with Appl 2023; 3: 53–61.

33.

Jarabaa

. Assessing the Widely Used Cyber Security Tools Using Multi-Criteria Decision Making 2024 . Doctoral dissertation, Mutah University.

34.

Yazdani

Zarate

Kazimieras Zavadskas

, et al. A combined compromise solution (CoCoSo) method for multi-criteria decision-making problems. Manage Decis 2019; 57: 2501–2519.

35.

Nabeeh

Sallam

. A combined compromise solution (CoCoSo) of MCDM problems for selection of medical best bearing ring. Neutrosophic Optimization and Intelligent Systems 2024; 1: 1–13.

36.

Nguyen T

Nguyen

Nguyen D

. A new hybrid pythagorean fuzzy AHP and COCOSO MCDM based approach by adopting artificial intelligence technologies. J Exp Theor Artif Intell 2024; 36: 1279–1305.

37.

Smarandache

. Neutrosophic logic as a theory of everything inlogics. Multispace and multistructure. Neutrosophic transdisciplinariry (100 collected papers of sciences), 2010, pp. 525–527.

38.

Cheng

Chen

. Decision making with intuitionistic fuzzy best-worst method. Expert Syst Appl 2024; 237:121215.

39.

Tanaji B

Roychowdhury

. BWM Integrated VIKOR method using neutrosophic fuzzy sets for cybersecurity risk assessment of connected and autonomous vehicles. Appl Soft Comput 2024; 159: 111628.

40.

El Saber

Mohamed

AbdelAziz

. Decarbonization transportation: evaluating role of cyber security in transportation sector based on neutrosophic techniques in a climate of uncertainty. Neutrosophic Sets and Systems 2023; 60: 570–582.

41.

Lin

. Information security evaluation of industrial control systems using probabilistic linguistic MCDM method. Comput, Mater Continua 2023; 77: 199–222.

42.

Thakkar

Lohiya

. A survey on intrusion detection system: feature selection, model, performance measures, application perspective, challenges, and future research directions. Artif Intell Rev 2022; 55: 453–563.

43.

Muneer

Farooq

Athar

, et al. A critical review of artificial intelligence based approaches in intrusion detection: a comprehensive analysis. J Eng 2024; 2024: 3909173.

44.

Scarfone

Mell

. Guide to intrusion detection and prevention systems (IDPS). NIST Spec Publ 2007; 800: 1–127.

45.

Choi

. NIST's software un-standards. Geo L Tech Rev 2025; 9: 65.

46.

Jakse Schönfelder

Gustafsson

. Intrusion Detection on Network Video Surveillance Systems. 2024.

47.

Raheja

Garg

. A ranking framework for the selection of IoT cloud platforms using hybrid multi-attribute decision-making method. Int J Intell Comput Cybern 2024; 17: 824–843.