AdaTrans: An adaptive transformer for IoT Malware detection based on sensitive API call graph and inter-component communication analysis

Abstract

With the development of the Internet of Things (IoT), mobile devices are playing an increasingly important role in our daily lives. There are various malware threats present in these mobile devices, which can steal users’ personal information. Some malware exploits Inter-Component Communication (ICC) to execute malicious activities for unauthorized data access and system control, enabling communication between different components within an app and between different apps. In this paper, we propose an Adaptive Transformer-based malware framework (named AdaTrans) that combines sensitive Application Programming Interface (API)- and ICC-related features. The framework first extracts sensitive function call subgraphs (SFCS) to reflect the caller-callee relationships, and then utilizes ICC interactions to reveal hidden communication patterns in malicious activities. Moreover, we propose a novel adaptive Transformer model to detect malicious behaviors. We evaluate our framework on real-world datasets and demonstrate that AdaTrans consistently outperforms other existing state-of-the-art systems.

Keywords

Internet of things ICC Malware detection transformer

1 Introduction

IoT technology has seamlessly integrated into the modern society, becoming an integral part of our daily lives [1]. Its proliferation and adoption have significantly improved people’s comfort and convenience. The proliferation of mobile IoT devices has resulted in a profusion of IoT services [2] that are conveniently accessible through the Android platform. Consequently, the Android-based operating system has emerged as one of the most popular systems in the IoT. The unreliable third-party marketplace provides many Android-based IoT applications for specific IoT scenarios [3].

Malware attacks on mobile IoT devices have become a concern in the field of IoT security [4 –6], posing a severe threat to the security and confidentiality of users’ data. For example, malware can be secretly implanted in devices without the user’s consent to carry out malicious activities such as stealing personal information, disrupting systems, and abusing the device. Moreover, mobile IoT devices often have inadequate security measures, such as outdated operating system versions, unupdated security patches, and untrusted applications [7, 8]. Notably, the Android system is highly vulnerable to malware attacks due to its openness and adaptability. This highlights the need to develop consistently effective malware detection systems, which are critical to ensuring the security of mobile IoT devices and the privacy of user data [9, 19].

In recent years, there has been growing interest in developing new malware detection techniques that are not based on signatures. One promising approach is based on analyzing the behavior of applications. Malware often exhibits certain patterns of behavior that are different from legitimate applications. Some graph mining-based techniques such as function call graph (FCG) [12, 17] and control flow graph (CFG) [14] are mainly employed. Such technologies mainly focus on extracting patterns of malicious behavior. However, thousands of nodes may need to be processed in a given FCG. In this paper, we propose a feature extraction scheme based on the SFCS features, which extracts SFCS from FCG. Specifically, the SFCS features account for only a small fraction of all API calls in FCG, and therefore our solution can greatly reduce the number of features that need to be analyzed. This paper proposes a feature extraction based on the SFC to reflect the complex caller-callee relationships.

Although existing sensitive API call-based detection systems [18] can reflect malware behavior characteristics, such approaches have some limitations. For example, sensitive API calls [18] can only reflect malicious behavior characteristics within an application and thus are vulnerable to malware. The Inter-component communication (ICC) [26, 27] mechanism in the Android system enables a wide range of interactions and data sharing between applications, providing a powerful means for different application components to communicate with one another, but also representing a significant security threat, as it can be easily exploited by malicious actors. As a result, using ICC features can more fully cover application behavior. Motivated by this observation, we propose a feature fusion scheme that combines ICC features with SFCS features. The ICC feature can capture the information flow across components by monitoring the ICC interaction between different Android applications, which are complementary to the SFCS features.

Based on the above ideas, this paper proposes a new malware detection framework (AdaTrans). Firstly, we employ the static analysis method to construct the function call graph (FCG). Then, we extract SFCS features from the FCG to describe potential malware behavior patterns. However, the SFCS features ignore essential information related to the interaction of different components. To address this limitation, we extract ICC features, which comprehensively cover the interactions between different components within an app and between different apps, such as broadcast receivers and content providers. Because of the complementary feature representation power between SFCS and ICC features, we propose a feature fusion scheme that combines the two feature sets to provide a more comprehensive representation of malicious behavior patterns. Moreover, we propose a novel adaptive Transformer model, referred to as AdaTrans, serving as a multi-head attention-based sequence-to-sequence model. AdaTrans incorporates an adaptive factor into the multi-head attention mechanism to capture important features that reflect the malware behavior. Our evaluation results demonstrate that AdaTrans outperforms existing state-of-the-art malware detection systems on four real-world datasets.

Our major contributions are as follows:

We extract SFCS features to capture the latent behavior patterns of malware while reducing the size of FCG.

We propose an ICC feature-based IoT malware detection system to capture information flow across different components within an app and between different apps by monitoring the ICC interactions.

We propose a new malware detection technique based on SFCS and ICC analysis, which can provide more comprehensive malicious behavior information. Specifically, we built a adaptive Transformer-based detection model to detect malicious behaviors in ICC interactions.

Extensive experiments conducted on real-world datasets demonstrate that AdaTrans consistently outperforms the existing state-of-the-art systems. Experimental results show that AdaTrans can provide superior generalization ability than existing detection systems.

2 Related work

Many recent studies [10, 11] have provided graph analysis methods for mobile application detection, such as function call graphs (FCG) [12, 17], control flow graphs (CFG) [14], Opcode graphs [15], PSI-graph [16], etc. These methods investigate software properties by inspecting manifest files and analyzing app source codes without running the apps. For instance, Fun et al., [12] constructed frequent sub-graphs and used machine learning models to identify malicious behavior patterns. In [13], Xu et al. extracted the n-gram sequences from the control-flow graphs and data-flow graphs on the instruction level. And then they filtered the sequences according to the weights of the classification model, which can capture the malicious behaviors of the malware. In summary, these methods often extract sensitive API calls and assign weights based on their importance to discern malicious apps from benign ones. The detection performance of such a method depends on the quality of the features. In this paper, we extract SFCS features to describe potential malicious behavior patterns of malware.

Besides, dynamic features like a data flow graph (DFG) [25] can be extracted during the execution of a given application. However, such methods are time-consuming and costly due to the dynamic analysis. The complexity, growth rate, and evolution of malware highlight an urgent need to develop advanced malware detection technologies. Anti-malware providers are increasingly offering a variety of anti-malware solutions to perform an in-depth scan of the application. While some graph mining-based techniques (e.g., FCG [12, 17], CFG [14], and sensitive API calls [18]) can effectively detect malware, they ignore ICC-related API calls. In this paper, we develop an ICC-based feature extraction to reveal hidden communication patterns in malicious activities by capturing information flow between different components. Our work focuses on capturing ICC-related features and patterns.

In addition to the above studies, there are some machine learning- and deep learning-based studies [19, 20]. Mourtaji et al. [21] proposed a Windows-based malware detection method that converts malware binaries into a grayscale image, which is more related to our method. GHSOM [22] is a hybrid approach, which combines a variety of different algorithms, including unsupervised and supervised learning algorithms, and a sequence-aware encoding method. In their system, the GHSOM is used to analyze and identify unstructured and complex malware behavior, as well as to cluster and encode similar sequences of malware behavior. By combining multiple approaches, Zhang et al. [23] proposed a cloud-based detection method (SaaS) to extract GLCM and n-gram features. These features are first fed into a random forest-based machine learning model and then enhanced by fuzzy hash to detect whether the given app is a repackaged App. However, these machine learning-based methods have limited generalization capabilities in the modeling process and cannot meet the applicability requirements of potential users for malware detection [24]. This paper proposes a AdaTrans-based detection model to capture the malicious behaviors of malware.

3 System design

3.1 Overview of our system

In this section, we propose a novel malware detection framework, which extracts SFCS and ICC features. This framework conducts four major processes; decompilation, feature generation, feature construction, and malware classification. Figure 1 shows an overview of our framework. In the decompilation stage, we obtain the configuration files and source code of an app by reverse techniques. In the feature generation stage, we design two system modules, namely SFCS module and ICC module, which were used to extract SFCS features and ICC features respectively. In the feature construction stage, they are merged into feature vectors, which are then input into a AdaTrans detection model. Lastly, the malware inference stage employs AdaTrans to detect malware and then uses a classification loss to optimize the whole network.

Fig. 1

The proposed AdaTrans framework.

3.2 Decompilation

Android applications are typically developed using Java programming language and then compiled into Dalvik code (DEX), which is stored in the classes.dex file. The compiled code, along with the necessary resources, is packaged into an APK file. During the decompilation process, a reverse tool is utilized to recreate the source code and class files from each app. By utilizing disassembly tools such as Androguard and apktool [21], we can extract the Dalvik code from the APK, and subsequently extract SFCS features and ICC features.

3.3 Feature generation

3.3.1 SFCS module

Application Programming Interfaces (APIs) as high-level programming language interfaces are designed to encapsulate system calls and reflect the behavior of code segments within a program. Applications leverage various APIs to achieve diverse functionalities. Malware necessitates communication with external programs, and often exploits sensitive APIs to achieve malicious behavior, such as acquiring personal user information, manipulating device functionalities, or engaging in illicit operations. The most common combination of sensitive API calls in malware reveals the presence of malicious activity. For example, some malware employs "getNetworkInfo" and "getWifiState" to obtain the user’s network information, enabling malicious activities such as credential theft and the sale of user data. Consequently, these API calls may insinuate potential malicious intentions while also shedding light on the vulnerabilities of apps.

Given a set of apps, our system collects and preprocesses them on smart devices to identify potentially malicious behavior and patterns. This system utilizes disassembly tools to construct function call graphs (FCGs), which can reflect the caller-callee relationships.

Definition 1 Function Call Graphs (FCGs): A FCG, denoted as G = (V, E), consists of a set of nodes V = {v₁, . . . , v_n} and a set of edges E = {e₁, . . . , e_n}. Each node v_i ∈ V in the FCG represents a function within an app, and each edge e_i,j ∈ E assigns the calling relationship between the caller v_i and the callee v_j.

Analyzing an FCG directly is time-consuming due to its numerous nodes. Our observations revealed the presence of sensitive API calls in the FCG, potentially indicating malware behavior. Malware within the same family often exhibits similar patterns when invoking sensitive APIs, such as consecutively calling multiple sensitive APIs across different methods. Based on these observations, we develop a lightweight static analysis mechanism that extracts both the sensitive API nodes and their neighboring nodes in the FCG. By revealing the patterns of sensitive API calls in malware, detection classifiers can effectively distinguish malware from benign applications and capture specific malware behavior patterns. Consequently, we present the definition of sensitive API calls.

$SenAPI = {{API}_{s_{1}}, . . ., {API}_{s_{n}}} .$ (1)

In the feature extraction phase, the contribution of each sensitive API to the semantic expression of malware is different. Therefore, we use the TF-IDF method to assign distinct weights to individual sensitive APIs, enabling us to differentiate their importance. Utilizing TF-IDF, the maliciousness of sensitive APIs exhibits a positive correlation with the percentage of malware calls and a negative correlation with the percentage of calls from all applications. The maliciousness degree MD can be calculated as follows.

${MD}_{SenAPI} = {md ({API}_{s_{1}}), . . ., md ({API}_{s_{n}})},$ (2)

where md (API_{s
_i}) indicates the maliciousness degree of i-th sensitive API API_{s
_i}. For each md (API_{s
_i}), we have:

$md ({API}_{s_{i}}) = β_{i} * log \frac{| M_{all} | + | B_{total} |}{| M_{{api}_{s_{i}}} | + | B_{{api}_{s_{i}}} |},$ (3) where β_i defines the ratio of malware invoking the sensitive API_{s
_i} to all malware samples. Then, we record the sensitive API_{s
_i} and its neighboring nodes in the FCG to form a sensitive function call subgraph SenSG, which reflects the caller-callee relationships. Then, we use a distance function dis (API_{s
_i}, Nei_{s
_k}) to control the size of SenSG, where Nei_{s
_k} is the set of its k-hop neighbor nodes Nei_{s
_k} = {v_{s
₁}, v_{s
₂}, . . . , v_{s
_k}}, and k is a control parameter defined by users. It should be noted that a sensitive function call subgraph SenSG can include multiple sensitive APIs, as malware often needs to invoke multiple sensitive APIs to carry out malicious activities. Here, we measure the maliciousness degree of a subgraph as follows.

$md ({SenSG}_{s_{i}}) = \sum_{{API}_{s_{i}}}^{{MD}_{SenAPI}} md ({API}_{s_{i}}) .$ (4)

After that, we repeat this process for each sensitive API_{s
_i} in SenAPI. Then, we have: $SFCSFea = {{SenSG}_{s_{1}}, . . ., {SenSG}_{s_{n}}},$ (5) where each sensitive API API_{s
_i} corresponds to a sensitive function call subgraph SenSG_{s
_i}.

3.3.2 ICC module

While sensitive API calls can be used to model malicious patterns and detect malware instances, real-world malware often exploits existing APIs in the Android system to mimic the normal behavior of benign apps in order to evade detection systems based on API calls. This paper presents an ICC-based feature extraction module, which systematically analyzes the ICC patterns exhibited by both benign apps and malwares. Our detection is not limited to sensitive APIs. ICC features are more resistant to evasion compared to SFCS features proposed in this paper due to the integral nature of component interaction within an application, making it less susceptible to easy modification or removal. Such interaction represents one of the most prevalent attack vectors employed by malware, which cannot be directly captured by API call-based detection systems. The ICC-based feature extraction scheme can effectively capture the interaction behavior between malware, providing richer component interaction information. Additionally, this scheme offers enhanced flexibility and universality, as it is not constrained by specific API calls, enabling it to identify and capture a wider range of malicious behavior, thereby enhancing the reliability and robustness of the detection system. The contribution of this study lies in the integration of the sensitive function call subgraph and ICC features, which provides a more effective and comprehensive method for malware detection. We define various categories of ICC-related features as follows.

App Components. The basic building blocks of an Android application include activities, services, content providers, and broadcast receivers. Activities are responsible for presenting the user interface. $ACT = {{Act}_{1}, {Act}_{2}, ..., {Act}_{3}}$ Services are responsible for background processing. $SER = {{Ser}_{1}, {Ser}_{2}, ..., {Ser}_{3}}$ Content providers facilitate the sharing of structured data. $CP = {{Cp}_{1}, {Cp}_{2}, ..., {Cp}_{3}}$ Broadcast receivers provide the ability to receive information. $BC = {{Bc}_{1}, {Bc}_{2}, ..., {Bc}_{3}}$ The development of malware often involves code reuse, resulting in shared malicious behaviors and component reuse among malware within the same family. For example, diverse malwares employ shared malicious components for similar attacks, such as package _ name . BootReceiver, package _ name . Spl - ash, and package _ name . BatteryService. We can obtain the component features as follows. $AC = {ACT, BC, CP, SER}$

Explicit Intents. Intents are used in the ICC mechanism to establish connections between components and enable communication among activities, services, and broadcast receivers. The primary functions of Intents include initiating activities, starting and stopping services, and delivering broadcast messages to Broadcast Receivers. The developer can specify the target components of an Intent to start with by explicitly including the target application package and class name. For Android apps, components within the same application are typically connected using Explicit Intents to facilitate internal communication. However, Explicit intents can be abused by malware as they may be sent to other applications. For instance, a benign app exposes its components to receive system-generated Intents, and then the adversary sends Explicit Intents to these exposed components by the malware attacks. Without sufficient permission protections and rigorous action verification, the adversary can directly initiate and manipulate these benign components.

Implicit Intents. The Intent class defines various system Intent actions and categories. Unlike Explicit Intents, which specify specific application package and class names for components, Implicit intents implicitly declare general actions that can be performed in the components, such as implicitly setting the contents (i.e., actions, categories, and or data fields) of the intents. The Android system then starts the appropriate components by comparing the contents of the intents to the declared Intent Filters, and apps can define their actions using the package name as a prefix. This makes it unusual and suspicious for an app to include Implicit Intent actions defined by other apps.

To understand how the malware uses the ICC mechanism to achieve malicious Intent, Fig. 2 shows an example of malware misusing the implied intent to launch an Intent Spoofing Attack. The benign App exposure component B to receive the Internal Intent 1, and engages in a legitimate action (package _ name . MODIFY). Malicious component A abuses the implied Intent mechanism to induce component B to inadvertently receive the malicious Intent 2 instead of the benign Intents 1.

Fig. 2

An example of Intent Spoofing Attack.

Intent Filters. Intent Filters are utilized to match Implicit Intents, which are typically registered by malware for their activities and services. Malware can initiate component-hijacking attacks by intercepting the Implied Intents and reading the intent data using Intent Filters. The malware injects false information into the returned response while replacing the benign components with malicious ones. In addition, Intent Filters for activities and services is mandated to be recorded in an application’s manifest file, whereas those for broadcast receivers can be registered either statically or dynamically, offering flexibility. Malware often registers Intent Filters for broadcast receivers, such as those that receive phone status (e.g., android . Intent . Action . SMS _ RECEIVED, android . Intent . Action . BOOT _ COMPLETED) or capture changes in signal strength (e.g., android . In - tent . Action . SIG _ STR) for broadcast information. These Intent Filters are rarely found in benign apps. Finally, we extract the ICC-related features for each category as follows.

$ICCFea = {{{AC}_{i}}_{1}^{n}, {{EI}_{i}}_{1}^{n}, {{IU}_{i}}_{1}^{n}, {{IF}_{i}}_{1}^{n}} .$ (6)

The component declaration described in Fig. 3 (lines 1-8) is included in the application manifest. The activity element (line 2) specifies the presence of an activity component (called MapRenderingActivity), which is accompanied by an associated Intent Filter. The action field should be set to “VIEW”, as indicated in line 4, while line 5 specifies that the received data should be in the form of a URI with a geographic scheme. Lastly, line 6 indicates that the category of the Intent can be set as “DEFAULT”. Moreover, Fig. 3 (lines 9-16) shows a representative example of Android ICC mechanism. In this example, the method “map()” renders a map using an Intent when provided with a coordinate. Within this method, an Intent object “I” is initially created as shown in line 11, and subsequently assigns an action with a value “VIEW” (line 12). Specifically, the action can be employed to display various types of data. The Intent’s data is specified in lines 13 and 14 as a URI with a geographic scheme. Lastly, the Android system invokes the “startActivity()” to resolve potential target components.

Fig. 3

A representative example of Android ICC mechanism.

3.4 Feature construction

We extract sensitive function subgraph features SFCSFea and ICC features ICCFea from the SAPI and ICC modules, respectively, which are stored in a text database, and denoted as semantic features F = {SFCSFea, ICCFea}. In the Feature Construction stage, we inject these features into a feature space and construct feature vectors to extract potential semantic patterns for each app. The generated feature vectors are then utilized by a neural network model to classify the malware, enabling the modeling of advanced concepts and facts within the malware. Given a semantic feature set F, we utilize the word embedding method to convert the semantic features into semantic vectors, which reflects the potential behavioral pattern of the malware. In our case, the semantic features of each malware are represented by a fixed two-dimensional matrix F_V = E^m,k, with m denoting the maximum number of semantic features and k representing the size of embedded vectors.

3.5 Adaptive transformer-based Malware detection

In this part, we propose a a novel adaptive Transformer (AdaTrans) model for malware detection, which describes malware behavior and models semantic knowledge. In our study, the AdaTrans, an attention-based sequence-to-sequence model, exhibits exceptional capabilities in identifying potential malware threats by enhancing the representation of key features. This research employs the extracted semantic features (i.e., F = {SFCSFea, ICCFea}) to train the AdaTrans model. Figure 4 illustrates the overall flow of our AdaTrans, providing a comprehensive visualization of the proposed model. Our AdaTrans model incorporates an attention mechanism to highlight key features that play a critical role in semantic representation. In other words, the attention mechanism serves as a critical component in weighting the variances contributed by different semantic features to the overall detection model. By mapping the linear projections Q, K, and V to an output, the attention mechanism enables the model to highlight key information in the input data. To determine attention weights, we calculate the dot product of the query Q and the corresponding key k, and subsequently employ a softmax function to calculate the weights assigned to the values. Through these computations, we derive meaningful attention distributions.

Fig. 4

The proposed AdaTrans model architecture.

$Attention (Q, K, V) = Softmax (\frac{Q K^{T}}{\sqrt{d_{k}}}) V,$ (7) where d_k is the dimension of Q and K. In order o further enhance the capabilities of our detection model, we incorporate the multi-head attention mechanism into the AdaTrans model, which can capture information from diverse representation subspaces at different positions. Specifically, The multi-head attention mechanism allows for parallel attention computations, projecting Q,K and V multiply times using distinct linear projections to dimensions dk, dk and dv, respectively. Each attention head produces an output vector, capturing relevant features within its respective subspace. Then, we have: ${head}^{(i)} = Attention (Q W_{i}^{Q}, K W_{i}^{K}, V W_{i}^{V}),$ (8) Specifically, the multi-head attention mechanism consists of h heads (i.e., the attention functions). Then, we introduce an adaptive factor γ to concatenate the outputs of multiple heads. In this process, each head is adaptively assigned a different weight. By adaptively incorporating multi-head attention, our model effectively captures important features across different subspaces, facilitating a comprehensive understanding of malware behavior. Then, we have: $γ_{i} = \max {1, \frac{| {head}^{(i)} | * h}{\sum_{j = 1}^{h} (| {head}^{(i)} |)}}$ (9) $FCN (x) = max (0, x W_{1} + b_{1}) W_{2} + b_{2} .$ (10)

By combining these components, our method provides a robust solution for malware detection, demonstrating its ability to effectively analyze and classify potential malware threats. $MH = [(γ_{1} \cdot {head}^{(1)}), \dots, (γ_{h} \cdot {head}^{(h)})] W^{O},$ (11) where W and b represent the weight and bias, respectively. Figure 4 provides a detailed illustration of the multi-head attention mechanism used in our AdaTrans model. To facilitate information flow, we incorporate residual connections in the multi-head attention, which enables the direct propagation of local representations from lower layers to higher layers. By preserving the local information at the upper layer, our model can effectively model semantic knowledge and capture malware behavior. Furthermore, we introduce a fully connected network (FCN) as an integral component of our architecture. This FFN layer is used to process the output vectors from the multi-head attention layer. The use of FFN complements the multi-head attention mechanism, enabling our model to capture both local and global information in a synergistic manner.

4 Experiments and evaluation

In IoT networks, it is common for IoT devices to be equipped with multiple IoT apps. However, adversaries often exploit vulnerabilities in IoT apps to install their own malware on targeted Mobile-IoT devices, aiming to spread the malware to more IoT devices. Given this context, it becomes imperative to thoroughly scrutinize the installed apps on IoT devices to safeguard IoT security.

The following sections present a detailed description of our experimental setup, designed to assess the effectiveness of AdaTrans. We conduct extensive experiments to comprehensively evaluate the performance of AdaTrans in malware detection. We implemented the learning component of AdaTrans in Python, using the Tensorflow and Keras platforms, to train and test the malware classifier. The evaluation demonstrates promising results, showing its ability to accurately and efficiently detect malware.

4.1 Datasets

In this section, we conducted evaluations using four publicly available datasets, including DREBIN, MalDroid, VirusShare, and AndroZoo. The DREBIN dataset consists of 5560 malware samples, which cover a diverse range of malware families, such as DroidDream, FakeDoc, Geinimi, and others. The MalDroid dataset was collected from December 2017 to December 2018, in which 10,000 malware samples were randomly selected in order to evaluate the performance of AdaTrans. Moreover, we collected 11,497 malware samples from VirusShare, an online site dedicated to providing a wide range of malware samples from 2012 to the present for research purposes. Another dataset used in our experiments is AMD, which comprises malware samples from 2010 to 2016. These diverse datasets offer valuable insights and enable us to assess the effectiveness of AdaTrans in malware detection. To ensure a fair comparison, we carefully balanced the dataset by setting an equal number of malware and benign apps during model training, thereby preventing the dataset from being heavily skewed toward one class, and minimizing the potential bias in the model’s performance evaluation.

4.2 Comparison with existing malware detection systems

For a fair comparison, we have compared the proposed AdaTrans with two existing state-of-the-art malware detection systems that have exhibited exceptional performance in the field of malware detection. Then, we re-implement these baseline methods under our experimental setting. Figures 6 present a detailed comparison of our AdaTrans with these two state-of-the-art baselines at different feature numbers and dimensions. These baseline systems are introduced below:

SenAPI: In [12], the authors proposed an approach for malware detection system based on sensitive API call features. Their work aimed to model the malicious behavioral patterns exhibited by malware through the analysis of function call graphs.

ICC: Xu et al. [26] proposed a ICC-based feature extraction for malware detection, which reflects the intricate relationships and dependencies between different app components.

Fig. 5

Number of feature contributing in model learning.

Fig. 6

Dimensions of feature contributing in model learning.

A notable observation is that SenAPI shows the worst performance across the four benchmark datasets. This may be because sensitive API calls only reflect the characteristics of malicious behavior, but ignore the interactions between app components and are therefore vulnerable to malware attacks. Therefore, this method is vulnerable to malware. From the results, we can find that ICC outperforms SenAPI in all datasets. The commendable performance of ICC features, closely resembling that of our proposed AdaTrans, underscores the efficacy of ICC features in revealing malicious interactions between app components. These findings also validate the effectiveness and necessity of ICC features.

Effects of Feature Number. As shown in Fig. 5, we conducted an additional study to thoroughly investigate the classification performance of the proposed AdaTrans under varying numbers of features. The experiment involved training the AdaTrans model with different feature sets, ranging from 100 to 500 features, and evaluating its performance. As the number of samples increases, the ICC and SenAPI methods exhibit a better fit to the underlying data distribution. Conversely, when the sample size is too small, the model becomes susceptible to under-fitting. Specifically, we can see that the proposed AdaTrans achieves a satisfactory result without too many features, and our model is insensitive to the number of features. This finding reinforces the practicality of AdaTrans, as it can effectively adapt to various datasets and feature numbers without compromising its performance.

Effects of Feature Dimension. In order to investigate the influence of the feature dimensions on the learning models, we conducted a comprehensive analysis focusing on different dimensions, as shown in Fig. 6. Specifically, we selected the dimensions [30, 50, 100, 150, 200] and evaluated their impact on the model’s performance and ability to capture the underlying data distribution. Interestingly, our analysis reveals that the performance of the model is largely unaffected by the dimensions of feature used for training. Overall, the comprehensive comparative analysis conducted in this paper validate the robustness and effectiveness of the AdaTrans method. The extensive experiments conducted on four benchmark datasets demonstrate that AdaTrans can always achieve high detection rates and low false positive rates.

4.3 Comparison with existing baseline algorithms

Deep learning models have emerged as a promising approach for detecting malware due to their ability to learn complex patterns and behaviors from large-scale datasets. To ensure a fair comparison, we compare our proposed AdaTrans model against seven state-of-the-art deep learning models for malware detection, incluing Convolutional Neural Network (CNN) [28, 29], Deep Neural Network (DNN) [32], Recurrent Neural Network (RNN) [33], Gated Recurrent Unit (GRU) [30], Long Short Term Memory (LSTM) [30, 31], Attention (ATT) [34], and Multi-head Attention (MATT) [34]. Table 1 and Fig. 7 present the evaluation results.

Table 1
AdaTrans versus baselines for malware detection

Dataset CNN [28, 29] RNN [33] DNN [32] GRU [30] LSTM [30, 31] ATT [34] MAATT [34] Our work

DREBIN ACC 91.53 91.04 93.77 93.47 94.51 92.65 94.44 96.30

Precision 91.82 89.05 93.62 92.91 94.03 91.27 94.65 96.26

Recall 91.06 93.68 93.69 93.83 94.95 93.98 94.00 96.19

F-score 91.44 91.31 93.65 93.36 94.49 92.61 94.32 96.22

VirusShare ACC 93.97 93.67 93.87 85.15 93.98 94.11 94.10 96.61

Precision 92.58 91.74 92.03 78.93 92.20 92.55 92.52 95.22

Recall 94.15 94.18 94.42 91.65 94.34 94.16 94.53 97.27

F-score 93.36 92.95 93.21 84.82 93.26 93.35 93.52 96.24

MalDroid ACC 91.33 92.06 90.91 91.11 94.38 91.66 93.06 96.40

Precision 91.51 91.96 89.95 92.41 92.23 91.91 91.26 96.61

Recall 91.95 92.96 93.28 90.27 95.09 92.43 96.30 96.55

F-score 91.73 92.46 91.59 91.33 93.64 92.17 93.71 96.58

AMD ACC 90.08 90.96 89.90 90.18 90.27 89.85 91.75 97.49

Precision 90.27 91.31 91.75 91.50 89.21 87.59 90.83 97.69

Recall 91.51 91.39 88.48 89.56 91.97 92.73 93.72 97.52

F-score 90.89 91.35 90.09 90.52 90.57 90.09 92.26 97.61

Dataset		CNN [28, 29]	RNN [33]	DNN [32]	GRU [30]	LSTM [30, 31]	ATT [34]	MAATT [34]	Our work
DREBIN	ACC	91.53	91.04	93.77	93.47	94.51	92.65	94.44	96.30
	Precision	91.82	89.05	93.62	92.91	94.03	91.27	94.65	96.26
	Recall	91.06	93.68	93.69	93.83	94.95	93.98	94.00	96.19
	F-score	91.44	91.31	93.65	93.36	94.49	92.61	94.32	96.22
VirusShare	ACC	93.97	93.67	93.87	85.15	93.98	94.11	94.10	96.61
	Precision	92.58	91.74	92.03	78.93	92.20	92.55	92.52	95.22
	Recall	94.15	94.18	94.42	91.65	94.34	94.16	94.53	97.27
	F-score	93.36	92.95	93.21	84.82	93.26	93.35	93.52	96.24
MalDroid	ACC	91.33	92.06	90.91	91.11	94.38	91.66	93.06	96.40
	Precision	91.51	91.96	89.95	92.41	92.23	91.91	91.26	96.61
	Recall	91.95	92.96	93.28	90.27	95.09	92.43	96.30	96.55
	F-score	91.73	92.46	91.59	91.33	93.64	92.17	93.71	96.58
AMD	ACC	90.08	90.96	89.90	90.18	90.27	89.85	91.75	97.49
	Precision	90.27	91.31	91.75	91.50	89.21	87.59	90.83	97.69
	Recall	91.51	91.39	88.48	89.56	91.97	92.73	93.72	97.52
	F-score	90.89	91.35	90.09	90.52	90.57	90.09	92.26	97.61

Fig. 7

AdaTrans versus baseline methods for malware detection.

Overall, the performance comparison among five baselines, CNN, DNN, RNN, GRU, and LSTM, exhibited marginal differences, with LSTM demonstrating slightly superior performance compared to the other four baselines under the same experimental settings. The GRU model demonstrated its proficiency in capturing semantic and context-dependency information. While its performance was competitive, it did not surpass the LSTM. This can be attributed to the LSTM’s ability to capture long-term dependencies through its sequential memory mechanism, which sets it apart from the other methods. Moreover, MATT stands out as a close competitor to our proposed AdaTrans model, emphasizing the importance of multi-head attention in capturing malicious behavior patterns. MATT exclusively employs multi-head attention to simultaneously process multiple informative aspects of the input data, thereby modeling the hidden contextual relationships by calculating attention weights. In light of these observations, AdaTrans model consistently outperforms all the neural network models on all metrics, indicating its superiority in detecting malware. Additionally, the utilization of multi-head attention in AdaTrans reinforces the significance of the attention mechanism in capturing meaningful patterns and identifying malicious behaviors, which serves as a pivotal component contributing to its superior performance.

Furthermore, AdaTrans demonstrates a significant improvement in detection performance on four well-established benchmark datasets dataset as well. The experimental results affirm the capability of AdaTrans to achieve superior results when compared to the two state-of-the-art approaches. Our techniques encompass sophisticated feature extraction mechanisms, improved Transformer learning algorithms, and effective fusion strategies that collectively contribute to the exceptional performance exhibited by the AdaTrans method.

Transformer vs. AdaTrans. In this section, we have evaluated our proposed AdaTrans model against the traditional Transformer model. Table 2 presents the evaluation results. It is evident that the proposed AdaTrans model offers several key advantages over the existing Transformer model. By integrating adaptive attention mechanisms and positional encodings, AdaTrans excels in capturing intricate patterns and contextual dependencies within malware samples. Furthermore, Fig. 8 provides valuable insights into the the model’s ability to fit data distributions during model training. As the number of iterations increases, Transformer and AdaTrans are able to capture complex patterns and relationships inherent in the data. However, our AdaTrans can adapt more quickly and fit the underlying data distribution more accurately than the traditional Transformer model. Our approach utilizes adaptive attention mechanisms to carefully select key features, demonstrating enhanced capabilities in malware detection. In summary, our proposed malware classifier based on the adaptive Transformer model demonstrates its effectiveness in capturing malware behavior and modeling semantic knowledge.

Table 2

Test accuracy. Transformer vs. AdaTrans

Dataset	Model	ACC	Precision	Recall	F-score
DREBIN	Transformer	95.85	96.33	95.07	95.70
	AdaTrans	96.30	96.26	96.19	96.22
VirusShare	Transformer	91.21	96.94	83.10	89.49
	AdaTrans	96.61	95.22	97.27	96.24
MalDroid	Transformer	90.36	97.38	83.67	90.01
	AdaTrans	96.40	96.61	96.55	96.58
AMD	Transformer	97.40	97.94	96.85	97.39
	AdaTrans	97.49	97.69	97.52	97.61

Table 3

Results of our work compared to other proposed systems

Other works	Features	Models	Datasets	#Benign	#Malware	ACC	F-score
DREBIN [35]	Static features	SVM	DREBIN	123,453	5,560	93.9%	-
DySign [36]	Fingerprint features	KNN	DREBIN	5,525	3,414	-	85%
PROPEDEUTICA [37]	System call features	DEEPMALWARE	-	1338	9115	97.03%	97.02%
HDNFDroid [38]	Permission and APIs	SHLMD	MUDFLOW	15,098	2,799	94.46%	96.73%
			VirusShare	815	2,013	90.57%	93.47%
YDataC [39]	Permission and APIs	RF	AndroZoo	5,536	5,536	-	88.56%
HyMalD [40]	Opcode and APIs	HyMalD	VirusShare	3889	8231	94.85%	-
Our work	ICC and APIs	AdaTrans	DREBIN	5,560	5,560	96.30%	96.22%
			VirusShare	11,497	11,497	96.61%	96.24%
			MalDroid	10,000	10,000	96.40%	96.58%
			AMD	6,680	6,680	97.49%	97.61%

Fig. 8

Evolution of the training accuracy and loss (Transformer vs. AdaTrans).

4.4 Comparison with state-of-the-art systems

To highlight the importance of our research, we compared the proposed AdaTrans with the state-of-the-art methods proposed by other researchers. Furthermore, various techniques have exhibited high precision in detecting Android malware. Most of these systems are not publicly available, making it difficult to re-implement the same systems with identical parameters. Therefore, all evaluation results shown in Table 6 are the best reported in their original papers, thereby ensuring a fair comparison with other state-of-the-art techniques. DREBIN [35] introduced static analysis for defending against Android malware attacks. DySign [36] emphasized dynamic analysis to generate effective fingerprints for Android malware. Moreover, PROPEDEUTICA [37] is a novel DL architecture that learns multi-scale spatial-temporal system call features with multi-stream inputs, employing ResNext blocks and recurrent neural networks to deal with software heterogeneity. Among these approaches, YDataC [39], HDNFDroid [38], and HyMalD [40] are the most related to us. YDataC [39] used the API calls and permissions as a feature set, and proposed two incremental learning methods for malware detection. HDNFDroid [38] learned rich features using an unsupervised feature learning algorithm MSAE, which models the relevant features at different granularity levels. HyMalD [40] used static features of the opcode sequence and the API call sequence, and then combined the Bi-LSTM and SPP-Net models to build a hybrid framework for detecting malware. However, these methods still cannot compare with the proposed AdaTrans. In comparison with the previous state-of-the-art techniques, AdaTrans shows its superior capabilities in detecting and classifying Android malware.

5 Conclusion

In this paper, we proposed an adaptive Transformer-based malware detection framework (AdaTrans), which provides a comprehensive malicious behavior information of malware though the SFCS and ICC analysis. This framework first extracted SFCS features to reflect the caller-callee relationships. As a complementary, the ICC-related features is then extracted to cover the ICC interactions between components within an app and between different apps. Finally, we proposed a adaptive Transformer to extract the textual semantic information and detect malicious behaviors in ICC interactions. The experimental results demonstrated that our AdaTrans consistently outperforms existing state-of-the-art malware detection systems on four real-world datasets.

In the future, we will enhance feature representation in Android malware detection by introducing more diverse raw features. As the pros and cons of static and dynamic analysis are well known, we will try to develop a new hybrid feature extraction scheme, which involves combining features extracted from both static and dynamic analysis. The exploration of hybrid analysis techniques is expected to improve the accuracy and robustness of AdaTrans in detecting more sophisticated malware threats. In addition, we will explore the use of fast sampling methods to select a representative subset of features, thereby enhancing the efficiency of AdaTrans.

Acknowledgment

This work was supported in part by the Autonomous Region Key R&D Project under Grant 2021B01002, and in part by the Network Information Security Intelligent Defense and Disposal Project under Grant GC-HGX210692.

References

Xing Yang , Lei Shu , Ye Liu , Gerhard Petrus Hancke , Mohamed Amine Ferrag and Kai Huang , Physical Security and Safety of IoT Equipment:A Survey of Recent Advances and Opportunities, IEEE Trans. Ind. Informatics 18(7) (2022), 4319–4330.

Umesh Kumar Lilhore , Agbotiname Lucky Imoize , Chun-Ta Li , Sarita Simaiya , Subhendu Kumar Pani , Nitin Goyal , Arun Kumar Rana and Cheng-Chi Lee , Design and Implementation of an ML and IoT BasedAdaptive Traffic-Management System for Smart Cities, Sensors 22(8) (2022), 2908.

Laphou Lao , Zecheng Li , Songlin Hou , Bin Xiao , Songtao Guo and Yuanyuan Yang , A Survey of IoT Applications in Blockchain Systems:Architecture, Consensus, and Traffic Modeling, ACM Comput.Surv. 53(1) (2021), 18:1–18:32.

Senming Yan , Jing Ren , Wei Wang , Limin Sun , Wei Zhang and Quan Yu , Asurvey of adversarial attack and defense methods for malwareclassification in cyber security, IEEE Commun. Surv. Tutorials 25(1) (2023), 467–496.

Bhardwaj

, Dagar

, Khan

M.O.

, Aggarwal

, Alvarado

, Kumar

, Irfan

and Proshad

, Smart IoT and machine learning-basedframework for water quality assessment and device componentmonitoring, Environmental Science and Pollution Research 29 (2022), 46018–46036.

Kanwalpreet Kour , Deepali Gupta , Kamali Gupta , Divya Anand , Dalia Elkamchouchi

, Cristina Mazas Perez-Oleaga , Muhammad Ibrahim and Nitin Goyal , Monitoring Ambient Parameters in the IoT Precision Agriculture Scenario: An Approach to Sensor Selection and Hydroponic Saffron Cultivation, Sensors 22(22) (2022), 8905.

Yue Liu , Chakkrit Tantithamthavorn , Li Li and Yepang Liu , DeepLearning for Android Malware Defenses: A Systematic LiteratureReview, ACM Comput. Surv. 55(8) (2023), 153:1–153:36.

Khullar

, Singh

H.P.

, Miro

, Anand

, Mohamed

H.G.

, Gupta

, Kumar

and Goyal

, IoT Fog-Enabled Multi-Node Centralized Ecosystem for Real Time Screening and Monitoring of Health Information, Applied Sciences 12(19) (2022), 9845.

Xinjun Pei , Xiaoheng Deng , Shengwei Tian , Lan Zhang and Kaiping Xue , A Knowledge Transfer-Based Semi-Supervised Federated Learning forIoT Malware Detection, IEEE Trans. Dependable Secur. Comput. 20(3) (2023), 2127–2143.

10.

Wuchner

, Cislak

, Ochoa

and Pretschner

, Leveraging compression-based graph mining for behavior-based malware detection, IEEE Trans. Dependable Secur. Comput 16(1) (2019), 99–112.

11.

Abusnaina

, Abuhamad

, Alasmary

, Anwar

, Jang

, Salem

, Nyang

and Mohaisen

D.A.

, A deep learning-based fine-grained hierarchical learning approach for robust malware classification, IEEE Trans. Dependable Secur. Comput (2021), 1–1.

12.

Fan

, Liu

, Luo

, Chen

, Tian

, Zheng

and Liu

, Android malware familial classification and representative sampleselection via frequent subgraph analysis, IEEE Trans. Inf.Forensics Secur. 13(8) (2018), 1890–1905.

13.

, Ren

and Song

, Android malware family classification and characterization using cfg and dfg, in 2019 International Symposium on Theoretical Aspects of Software Engineering (TASE) (2019), 49–56.

14.

Cesare

, Xiang

and Zhou

, Control flow-based malware variant detection, IEEE Transactions on Dependable and Secure Computing 11(4) (2014), 307–317.

15.

Azmoodeh

, Dehghantanha

and Choo

K.R.

, Robust malware detectionfor internet of (battlefield) things devices using deep eigenspacelearning, IEEE Trans. Sustain. Comput. 4(1) (2019), 88–95.

16.

Nguyen

, Ngo

and Le

, A novel graph-based approach for IoT botnet detection, Int. J. Inf. Sec. 19(5) (2020), 567–577.

17.

Cai

, Jiang

, Gao

, Li

and Yuan

, Learning features from enhanced function call graphs for android malware detection, Neurocomputing 423 (2021), 301–307.

18.

Weiping Wang , Jianjian Wei , Shigeng Zhang and Xi Luo , LSCDroid: Malware Detection Based on Local Sensitive API Invocation Sequences, IEEE Trans. Reliab. 69(1) (2020), 174–187.

19.

Kumar Manoj , Srivastava

and Uddin

, Forgery detection using multiple light sources for synthetic images, Australian Journal of Forensic Sciences (2017), 1–8.

20.

Singh

, Chintalacheruvu

S.C.K.

, Garg

, Giri

and Kumar

, Efficient face identification and authentication tool for biometric attendance system. In 2021 8th International Conference on Signal Processing and Integrated Networks (SPIN), IEEE (2021), 379–383.

21.

Mourtaji

, Bouhorma

and Alghazzawi

, Intelligent framework for malware detection with convolutional neural network, in Proceedings of ICNISS (2019), 7.

22.

Hsiao

S.-W.

and Yu

, Malware family characterization with recurrent neural network and ghsom using system calls, in Proceedings of IEEE CloudCom (2018), 226–229.

23.

Zhang

, Ren

, Zhu

and Ren

, Saas: A situational awarenessand analysis system for massive android malware detection, Future Generation Computer Systems 95 (2019), 548–559.

24.

Jitendra Kumar Samriya , Rajeev Tiwari , Xiaochun Cheng , Rahul Kumar Singh , Achyut Shankar and Manoj Kumar , Network intrusion detection using ACO-DNN model with DVFS based energy optimization in cloud framework, Sustain. Comput. Informatics Syst. 35 (2022), 100746.

25.

Tingting Jiang , Lingling Cui and Zedong Lin , Faming Lu: A Survey ofMalware Classification Methods Based on Data Flow Graph, ICPCSEE (1) (2022), 80–93.

26.

Ke Xu , Yingjiu Li and Robert

, Deng: ICCDetector: ICC-Based Malware Detection on Android, IEEE Trans. Inf. Forensics Secur. 11(6) (2016), 1252–1264.

27.

Damien Octeau , Patrick McDaniel

, Somesh Jha , Alexandre Bartel , Eric Bodden , Jacques Klein and Yves Le Traon , Effective Inter-Component Communication Mapping in Android: An Essential Step Towards Holistic Security Analysis, USENIX Security Symposium (2013), 543–558.

28.

McLaughlin

, et al., Deep android malware detection, in Proc. 7th ACM Conf. Data Appl. Secur. Privacy (2017), 301–308.

29.

Wang

, Zhu

, Zeng

, Ye

and Sheng

, Malware traffic classification using convolutional neural network for representation learning, in 2017 International Conference on Information Networking (ICOIN) (2017), 712–717.

30.

Athiwaratkun

and Stokes

J.W.

, Malware classification with lstm and gru language models and a character-level cnn, in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2017), 2482–2486.

31.

Jahromi

A.N.

, Hashemi

, Dehghantanha

, Parizi

R.M.

and Choo

K.R.

, An enhanced stacked LSTM method with no random initializationfor malware threat hunting in safety and time-critical systems, IEEE Trans. Emerg. Topics Comput. Intell. 4(5) (2020), 630–640.

32.

Kim

, Kang

, Rho

, Sezer

and Im

E.G.

, A multimodal deeplearning method for android malware detection using variousfeatures, IEEE Trans. Inf. Forensics Secur. 14(3) (2019), 773–788.

33.

Lakshmanarao

and Shashi

, Android Malware Detection with DeepLearning using RNN from Opcode Sequences, Int. J. Interact.Mob. Technol. 16(1) (2022).

34.

Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , LlionJones , Aidan Gomez

and Lukasz Kaiser , Illia Polosukhin: Attention is All you Need, NIPS (2017), 5998–6008.

35.

Daniel Arp , Michael Spreitzenbarth , Malte Hubner , Hugo Gascon and Konrad Rieck , DREBIN: Effective and Explainable Detection of Android Malware in Your Pocket, NDSS 14 (2014), 23–26.

36.

El Mouatez Billah Karbab , Mourad Debbabi , Saed Alrabaee and Djedjiga Mouheb , DySign: dynamic fingerprinting for the automatic detection of android malware, MALWARE (2016), 139–146.

37.

Ruimin Sun , Xiaoyong Yuan , Pan He , Qile Zhu , Aokun Chen , Andre Gregio , Daniela Oliveira and Xiaolin Li , Learning Fast and Slow: Propedeutica for Real-Time Malware Detection, IEEE Trans.Neural Networks Learn. Syst. 33(6) (2022), 2518–2529.

38.

Huijuan Zhu , Liang-Min Wang , Sheng Zhong , Yang Li and Victor Sheng

, A hybrid deep network framework for android malwaredetection, IEEE Trans. Knowl. Data Eng. 34(12) (2022), 5558–5570.

39.

Woosang Cho , Hojun Lee , Sangchul Han , Young-Sup Hwang and Seong-jeCho , Sustainability of Machine Learning-based Android Malware Detection Using API calls and Permissions, AIKE (2022), 18–25.

40.

Jueun Jeon , Byeonghui Jeong , Seungyeon Baek and Young-Sik Jeong , Hybrid Malware Detection Based on Bi-LSTM and SPP-Net for Smart IoT, IEEE Trans. Ind. Informatics 18(7) (2022), 4830–4837.