DLAN:Modeling user long- and short-term preferences based on double-layer attention network for next point-of-interest recommendation

Abstract

The next Point-of-Interest (POI) recommendation, in recent years, has attracted an extensive amount of attention from the academic community. RNN-based methods cannot establish effective long-term dependencies among the input sequences when capturing the user’s motion patterns, resulting in inadequate exploitation of user preferences. Besides, the majority of prior studies often neglect high-order neighborhood information in users’ check-in trajectory and their social relationships, yielding suboptimal recommendation efficacy. To address these issues, this paper proposes a novel Double-Layer Attention Network model, named DLAN. Firstly, DLAN incorporates a multi-head attention module that can combine first-order and high-order neighborhood information in user check-in trajectories, thereby effectively and parallelly capturing both long- and short-term preferences of users and overcoming the problem that RNN-based methods cannot establish long-term dependencies between sequences. Secondly, this paper designs a user similarity weighting layer to measure the influence of other users on the target users leverage the social relationships among them. Finally, comprehensive experiments are conducted on user check-in data from two cities, New York (NYC) and Tokyo (TKY), and the results demonstrate that DLAN achieves a performance in Accuracy and Mean Reverse Rank enhancement by 8.07% -36.67% compared to the state-of-the-art method. Moreover, to investigate the effect of dimensionality and the number of heads of the multi-head attention mechanism on the performance of the DLAN model, we have done sufficient sensitivity experiments.

Keywords

Point-of-interest recommendation user preferences attention network social information

1 Introduction

Location-based social networks (LSBNs) such as Foursquare, Gowalla, Facebook Place, and Yelp have evolved as a result of the explosive growth of information technology. These social networks collect an extensive amount of user data from the services they offer, including check-in information, comment information, social information, etc. The accumulation of a large amount of user data provides good data support for the industry and academia to learn users’ movement patterns, resulting in a series of more advanced next POI recommendation methods [1-6]. Matrix Factorization (MF) and Markov Chain (MC) are two of the primary next POI recommendation methods that are frequently employed. For instance, the Instance-Region Neighborhood Matrix Factorization(IRenMF) [5] method that Liu et al. presented involves two levels of geographic information in the latent feature learning of users and locations. Cheng et al. [7] proposed a combined MF and MC for the Factorizing Personalized Markov Chain (FPMC) method FPMC-LR. While traditional methods are excellent at recommending the next POI, they typically fall lacking when it comes to utilizing the sequential nature of the user’s check-in record and analyzing the entire movement patterns of the users. Recurrent neural networks(RNN) and their variants have been extensively utilized in applications involving deep learning, facilitating the development of a number of next POI recommendation methods that can more accurately predict users’ movement patterns based on check-in trajectories. A spatio-temporal recurrent neural network (ST-RNN) [8], which takes into account the gaps in space and time between points of interest (POIs), was introduced by Liu et al. To handle local temporal and spatial context models, the method leverages specific spatial transfer matrices customized for varying geographical distances and specific temporal transfer matrices for distinct time periods. In recent work, researchers have tended to model users’ long- and short-term preferences by dividing their check-in sequences into multiple sub-series. For instance, a spatio-temporal gated coupling network (ST-GCN) [9] was proposed by Zhao et al. to capture both short- and long-term user preferences using novel time- and distance-specific gates. With the help of non-local networks for learning long-term user preferences and a geographically dilated RNN for learning short-term user preferences, Sun et al. introduced the long-term and short-term modeling of user trajectories method LSTPM [10].

Despite the superior performance of RNN-based methods in predicting the next POIs, several issues still exist:

•Neglecting Long-term Dependency The inherent structural features of RNN preclude RNN-based next POI recommendation systems from effectively establishing long-term dependencies between input sequences. The user’s check-in sequence in the next POI recommendation, however, is typically lengthy. These methods lack the ability to sufficiently mine the user’s preferences when the input check-in sequence is too long, since they are unable to adequately explore the latent information in the check-in sequence.

•Limiting Recommended Efficiency In the recurrent mechanism of RNN, the state of each moment is not only determined by the input of that moment, but also depends on the output of the previous moment. This structural characteristic causes its inefficiency by not allowing parallel computation and spiking the training time for long sequences.

•Insufficient mining of user’s long-term preference The majority of currently available methods for mining potential information in user check-in sequences concentrate only on the influence of the previous one or several check-in context on recommendations, i.e., they concentrate on mining the short-term preferences of users, ignoring the high-order neighborhood information in user check-in trajectories, that is, the association information between two non-adjacent, non-consecutive but functionally similar check-ins. Consequently, these methods do not sufficiently mine long-term preference in user check-in sequences. Users’ long-term preferences refer to the general preferences of users extracted from their check-in sequences. Both the long-term and short-term preferences of users are critical factors that influence the next POI recommendation [11].

•Neglecting Social Information Most current work tends to employ only the user’s check-in information, time information, and geographic information, while the user’s social information is often neglected, which in turn contributes to the loss of collaborative signals and affects the recommendation effect. According to reference [12], in the next moment, the user’s travel decision is determined by their social friends as well as the context in which they are placed.

To address the concerns raised above, this paper presents DLAN, a Double-Layer Attention Network for the next POI recommendation. DLAN employs a multi-head attention mechanism that integrates first-order and high-order neighborhood information in user’s check-in trajectories. This mechanism not only effectively avoids the problem that RNN cannot establish long-term dependencies between sequences, but also mines users’ long- and short-term preferences to acquire POIs of interest to them. Meanwhile, this study develops a user similarity weighting module to assess the influence of other users on the target user, as well as to identify possible social friends of the target user to establish a virtual social relationship for him/her. This paper’s contributions are mostly in the following areas.

We present a novel extended multi-head attention mechanism module in DLAN. In the multi-head attention mechanism, DLAN assigns different weights to each head, i.e., the independent self-attention space, to extract the long- and short-term preferences of users in parallel, which strongly addresses the problem of RNN not applying to lengthy sequences and enhances suggestion efficiency.

In each self-attention space, DLAN takes into account not only the first-order neighborhood information in the user’s check-in trajectory but also incorporates the non-adjacent, non-consecutive but functionally similar high-order neighborhood check-in information into the interaction, to adequately explore the user’s general interest and then capture the POIs of interest.

In this research, a user similarity weighting module is developed to take into account users’ social information. To measure the influence of other users on the target user, DLAN employs cosine similarity to calculate the similarity between the target user and other users’ check-in trajectories to find his or her potential social friends to build a virtual social relationship for him/her. It also weights the target user and potential social friends to obtain a more accurate representation of the target user. Furthermore, mining social relationships of the target user will assist DLAN to more effectively employ collaborative information in recommendations, reducing the impact of data sparsity on recommendation effectiveness to some extent.

DLAN has been exhaustively experimented on real data sets in two cities, New York City and TKY. The experimental results show that DLAN outperforms seven excellent baseline methods.

2 Related work

In this section, related work for the next POI recommendation is presented in three aspects: Markov-based Methods, RNN-based Methods, and Attention-based Methods.

2.1 Markov-based methods

Earlier researchers adopted a widespread approach used in sequential recommendation tasks: Markov chains [7 , 14]. By adding a local area constraint and only implementing into account check-in records within a specific area centered on the current location, Cheng et al. [17] extended FPMC. It can significantly reduce the computational effort of the algorithm and can eliminate some extraneous noise. At the same time, researchers have explored the possibility of applying commonly utilized matrix factorization or metric embedding techniques to the next POI recommendation [15-17]. For illustration, Feng et al. [15] presented the PRME, which learns user preferences and sequential transitions by modeling POIs and users in the potential space. However, Markov-based methods can only regard a few recent user check-ins, which cannot capture the long-term preferences of users and the sequential characteristics of user check-in sequences.

2.2 RNN-based methods

For the purpose of trying to capture temporal dynamics and sequential correlations in the user’s history check-in information, RNN-based methods and their variants have been employed in the next POI recommendations [8-10 , 19]. By creating specific time gates to describe time intervals, the Time-LSTM [18] applies long and short-term memory networks to the POI recommendation. The spatio-temporal recurrent neural network (ST-RNN) [8], which incorporates spatio-temporal context into the RNN layers, was proposed by Liu et al in 2016. Specifically, a geographic transfer matrix and a temporal transfer matrix, respectively, represent the spatial and temporal contexts. Both Time-LSTM and ST-RNN only consider the influence of recent check-in records on the next POI recommendation, and the role of users’ long-term preferences is not regarded. Therefore, with the goal of learning users’ long-term and short-term preferences, respectively, Zhao et al. constructed a new LSTM unit called Spatio-Temporal Gated Network (ST-GCN) [9], which is produced with two temporal and distance gates. The PLSPL [19] was proposed by Wu et al. and designed a standard LSTM for short-term trajectory mining and an attention layer for capturing users’ long-term preferences.

While the RNN-based methods can effectively model users’ check-in sequences, it is constrained by their susceptibility to the inability to establish long-distance dependencies in sequences. Specifically, a user’s next check-in may be influenced by check-ins that occurred earlier, resulting in its inability to adequately identify and leverage users’ preferences.

2.3 Attention-based methods

The attention mechanism has become the mainstream method for the next POI recommendations [2, 20-24], owing to the emergence of Transformer [25] in recent years. In DeepMove [20], the authors proposed to employ an attention layer and a recurrent network layer to capture users’ long-term preferences and short-term preferences, respectively, in order to guarantee adequate mining of users’ preferences from trajectory sequences with sequential correlation. Luo et al. [21] designed the STAN model to propose for the first time the usage of attention mechanism to aggregate spatio-temporal information between non-adjacent and non-contiguous POIs in the user trajectory and between POIs in the trajectory and candidate POIs. The multi-head self-attention mechanism, as proposed in reference [25], constitutes a significant advancement and development over the self-attention mechanism by combining multiple attention mechanisms. This method has been proven capable of acquiring knowledge from diverse representation subspaces while preserving contextual order information and accurately capturing range dependencies among elements within sequences. As a consequence, it has delivered remarkable outcomes across various areas, such as sentence embedding [26], text classification [27], and sentiment analysis [28].

3 Research objectives

3.1 Symbol description

In this paper, the set of users is represented as U ={ u₁, u₂, ⋯ , u_M }, where u_i ∈ U is a user in the set U, i ∈ [1, M], $M \in ℕ^{*}$ and the set of POIs is represented as L ={ l₁, l₂, ⋯ , l_N }, where l_j ∈ L is a POI in the set L, j ∈ [1, N], $N \in ℕ^{*}$ . The check-in r_k is represented by a triplet, i.e., r_k = (u_i, l_k, t_k), where u_i denotes the user who checked in, l_k denotes the POI of the check-in, and t_k denotes the timestamp for the check-in. For u_i ∈ U, the check-in history of u_i is sorted in ascending order according to the check-in time to obtain his check-in trajectory sequence seq_{u
_i} ={ r₁, r₂, ⋯ , r_m }. Due to the variable check-in length of each user, to facilitate embedding, this paper establishes the user check-in sequence as a fixed length n. If the check-in trajectory length of user u_i is greater than or equal to n, i.e., m ≥ n, DLAN only intercepts the first n check-ins in the trajectory of u_i. And if m < n, DLAN complements 0 on the right side of the check-in trajectory sequence of user u_i, which is subsequently masked during model training.

3.2 Problem definition

Given a target user u_i’s check-in trajectory sequence seq_{u
_i} ={ r₁, r₂, ⋯ , r_m }, where u_i ∈ U, r_m is u_i’s check-in at time t_m, the goal of the next POI recommendation is to recommend the top-k POIs for u_i at time t_m+1 based on his or her current context and historical check-in sequence.

4 Overview of DLAN

The DLAN proposed in this paper is concerned with the next POI recommendation problem. The determination made by the user to access the POI at the next moment is subject to various influences, comprising the present state of the user’s context, their inherent long-term preferences, short-term preferences stemming from the current spatio-temporal context, and the user’s social relationships. In this paper, the emphasis is on the following three factors:

•Long-term Preference The long-term preferences of users have a substantial impact on their selection of travel destinations. In this paper, long-term preference denotes the fundamental preferences of users that remain invariant over time. More specifically, it pertains to the persistent way of life exhibited by a user and manifests as a steady and periodic travel pattern. For example, through analysis of the dataset utilized in this paper, we found that a certain user in the dataset consistently frequented the Coffee Shop near the company on Monday, Wednesday, and Friday afternoons every week in the past year, that is, the user’s visits to the Coffee Shop showed a periodicity and regularity. Therefore, it is not complicated to determine that the user has a long-term preference for drinking coffee near the office on Monday, Wednesday, and Friday afternoons.

•Short-term Preference The impact of users’ short-term preferences on their next check-in cannot be disregarded, and a robust connection is present between users’ next check-in and their current and recent check-ins. In the above instance, we observe that the particular user has a long-term preference for drinking coffee near the company on Monday, Wednesday, and Friday afternoons. Nevertheless, a thorough review of his or her check-in data demonstrates some information that does not correspond to long-term preferences, namely, this user’s glaring absence from the Coffee Shop near their company at the specific time mentioned above. By tracing the user’s check-in data back several times before the user did not visited the Coffee Shop, it is found that the user was geographically located far from his or her company, suggesting that context-dependent changes in short-term preferences can break the user’s consistent travel patterns. Therefore, it is easy to find that users’ long- and short-term preferences jointly influence their decision-making process, and neglecting either their long-term preferences or short-term preferences can hurt recommendation accuracy.

•Social Relationship User social relationships also affect the decision-making process of users regarding their access to POIs. In the research of recommendation methods, the classical user-based collaborative filtering recommendation methods make use of users’ social relationships to accomplish recommendations. The user’s check-in data for the next POI recommendation can commonly be extremely sparse, so the ability to effectively utilize collaborative information of the target user will serve to somewhat mitigate the harm that data sparsity triggers to model performance and enhance the precision of model recommendations.

To sufficiently take into consideration the above three critical factors impacting users’ choice of travel destinations: users’ long-term preferences, users’ short-term preferences, and users’ social relationships, a double-layer Attention Network called DLAN is designed in this paper. First of all, in DLAN, this paper designs a multi-head attention mechanism that fuses the first-order neighborhood and high-order neighborhood information in the user’s check-in trajectory. The structural nature of the multi-head attention mechanism makes it abundant and parallel to exploit the long- and short-term preferences of users. Meanwhile, the introduction of first-order neighborhood and high-order neighborhood information enables DLAN to extract more fully the short-term and long-term preferences of users, respectively. Secondly, to utilize the social information of the target users, this paper also designs a user similarity weighting module, which measures the influence of other users on the target users by utilizing the cosine similarity to search for potential social friends of the target users to build a virtual social relationship for him/her. Furthermore, the user similarity weighting module sufficiently accounts for the collaborative information of users to mitigate the damage caused by data sparsity to DLAN performance to a certain extent. Figure 1 illustrates the model structure of the DLAN. More specifically, DLAN first feeds the user’s historical check-in sequence, the candidate POIs set and the GPS information of the POIs into the model via the Input Layer. For the consideration of the first-order neighborhood information and high-order neighborhood information in the user check-in trajectory and the spatio-temporal correlation between the POIs in the user trajectory, and the candidate set of POIs, the trajectory spatio-temporal interval matrix trajectory and the candidate POIs spatio-temporal interval matrix are calculated as the input of the model in this paper. Secondly, the Embedding Layer obtains an embedding representation of the user check-in trajectory and two spatio-temporal interval matrixes by obtaining the input information. When DLAN feeds various embedded information to Extended Multi-Head Self-Attention Layer, it can achieve the mining of users’ long-term preferences and short-term preferences, and thus capture POIs of more interest to users. Then, DLAN measured the influence of different users in the social network on the target users through User Similarity Weighting Layer and searched for potential social friends of the target users to derive a more accurate representation of the target users. Subsequently, this paper then calculates the probability of each POI in the candidate set by an Attention Matching Layer. Finally, the DLAN outputs the top-k POIs that the user is most likely to visit next through the Output Layer. In the following sections, each of these modules will be described in detail.

Fig. 1

Architecture of DLAN.

5 The proposed method

5.1 Input layer

In addition to the users set, POIs set and users’ check-in trajectories, two spatio-temporal interval matrices are computed as inputs to the model, which will be described in 4.2.1 and 4.2.2.

5.1.1 Trajectory spatio-temporal interval matrix

In this paper, the time interval and distance interval between any two POIs in the check-in sequence seq_{u
_i} ={ r₁, r₂, ⋯ , r_n } of u_i ∈ U are modeled as an explicit temporal interval matrix $V_{u_{i}}^{T} \in ℝ^{n \times n}$ and a spatial interval matrix $V_{u_{i}}^{S} \in ℝ^{n \times n}$ . For each element $V_{j, k}^{S}$ of the spatial interval matrix $V_{u_{i}}^{S}$ , DLAN use the GPS information of POIs and calculate the surface distance between them by the semi-positive vector equation Haversine (GPS_j, GPS_k), as shown in Equation (1). Each element $V_{j, k}^{T}$ in the temporal interval matrix $V_{u_{i}}^{T}$ represents the time interval between the j-th check-in and the k-th check-in, as shown in Equation (4). $V_{j, k}^{S} = Haversine ({GPS}_{j}, {GPS}_{k}), 1 \leq j, k \leq n$ (1) $Haversine (GP S_{j}, GP S_{k}) = 2 * R * arg sin (θ)$ (2) ${\begin{matrix} θ = \sqrt{{sin}^{2} (\frac{γ}{2}) + ψ {sin}^{2} (\frac{λ}{2})} \\ ψ = cos (la t_{k}) cos (la t_{j}) \\ γ = la t_{k} - la t_{j} \\ λ = lo n_{k} - lo n_{j} \end{matrix}$ (3) $V_{j, k}^{T} = | t_{j} - t_{k} |, 1 \leq j, k \leq n$ (4) Where GPS_j = (lat_j, lon_j), GPS_k = (lat_k, lon_k) represents the longitude and latitude information of the POI l_j and l_k, and R is the radius of the Earth.

5.1.2 Candidate POIs spatio-temporal interval matrix

In order to account for the spatio-temporal correlation information between each check-in recorded in a given trajectory and the anticipated next POI present within the candidate set, we further acquired the candidate POIs temporal interval matrix $V_{u_{i}}^{' T} \in ℝ^{N \times n}$ and spatial interval matrix $V_{u_{i}}^{' S} \in ℝ^{N \times n}$ . In this paper, all of the POIs in the dataset set L are used as candidate set. For each element $V_{i, j}^{' S}$ in the spatial interval matrix $V_{u_{i}}^{' S}$ , this paper calculates the surface distance between each POI i ∈ [1, N] in the candidate set and each check-in j ∈ [1, n] in u_i’s trajectory utilizing Haversine (GPS_i, GPS_j), as shown in Equation (5). Since causality is to be considered, i.e., the (m′+1)-th check-in can be predicted only when the first m′ check-in sequences are transmitted in, we design the masks to mask the user’s input sequences during training and select the labels accordingly. We calculate the interval $V_{First}^{' T} \in ℝ^{n}$ between the check-in to be predicted and the current user check-ins’ time {t₁, t₂, ⋯ , t_m′ } employing Equation (6), and subsequently broadcast it by row N, expanding it into a two-dimensional matrix space to obtain $V_{u_{i}}^{' T}$ . $\begin{matrix} V_{i, j}^{' S} & = Haversine (GP S_{i}, GP S_{j}), \\ 1 \leq i \leq N, 1 \leq j \leq n \end{matrix}$ (5) $V_{First}^{' T} = | t_{m^{'} + 1} - t_{j} |, 1 \leq j \leq n$ (6) Where GPS_i = (lat_i, lon_i), GPS_j = (lat_j, lon_j) represents the longitude and latitude information of the POI l_i and l_j.

5.2 Embedding layer

In this layer, we will perform embedding operations on users, POIs, time, and spatio-temporal factors of influence to acquire a dense vector representation of them.

5.2.1 Trajectory embedding layer

The Trajectory Embedding layer encodes the users, POIs, and time as potential vectors, i.e., $e^{u} \in ℝ^{d}$ , $e^{l} \in ℝ^{d}$ , $e^{t} \in ℝ^{d}$ , where d is the embedding dimension of the model. To alleviate the sparsity, the 7 × 24 = 168 hours of a week are mapped into 168 dimensions in hours in this paper. This discretized time representation can accurately describe the movement pattern of users during a week. By Trajectory Embedding Layer, we can obtain the embedding $E (r_{k}) \in ℝ^{d}$ of each check-in r_k of u_i as shown in Equation (7). In Equation (8), the check-in sequence seq_{u
_i} of u_i is encoded as $E ({seq}_{u_{i}}) \in ℝ^{n \times d}$ . $E (r_{k}) = e^{u_{i}} + e^{l_{k}} + e^{t_{k}}, r_{k} \in {seq}_{u_{i}}$ (7) $E ({seq}_{u_{i}}) = {E (r_{1}), E (r_{2}), \dots, E (r_{n})}$ (8)

5.2.2 Spatio-temporal matrix embedding layer

Spatio-Temporal Matrix Embedding Layer is applied to encode temporal and spatial data. Inspired by [29], in this paper, a linear interpolation technique is adopted to learn the discretization of the spatio-temporal matrix, with its setting of an upper bound unit embedding vector and a lower bound unit embedding vector, and the explicit interval is represented as a linear interpolation. The linear interpolation embedding is performed for each element of the spatio-temporal matrix to smoothly represent the spatio-temporal interval of dimension d. The interpolation embedding is calculated as follows: $e_{i, j}^{Δ t} = \frac{e (α^{T}) (α^{T} - Δ t) + e (β^{T}) (Δ t - β^{T})}{α^{T} - β^{T}}$ (9) $e_{i, j}^{Δ s} = \frac{e (α^{S}) (α^{S} - Δ s) + e (β^{S}) (Δ s - β^{S})}{α^{S} - β^{S}}$ (10) Where α^S and α^T denote the upper bound of the spatio-temporal interval, which is the minimum value of the interval. β^S and β^T denote the lower bound of the spatio-temporal interval, that is, the maximum value of the interval.

This layer is mainly divided into two parts: Personalized Spatial-Temporal Interval Encoding and Candidate POIs Spatial-Temporal Interval Encoding, which encode the trajectory spatial-temporal interval matrix and the candidate POIs spatial-temporal interval matrix, respectively. Their embeddings are: $E (V_{u_{i}}^{S}) \in ℝ^{n \times n \times d}$ , $E (V_{u_{i}}^{T}) \in ℝ^{n \times n \times d}$ , $E (V_{u_{i}}^{' S}) \in ℝ^{N \times n \times d}$ , $E (V_{u_{i}}^{' T}) \in ℝ^{N \times n \times d}$ , respectively. Finally, we sum them according to their embedding dimensions and add the corresponding entries of the temporal interval embedding matrix and the spatial interval matrix to obtain the embedding representation of the two spatial-temporal interval matrices $E (V_{u_{i}}^{S, T}) \in ℝ^{n \times n}$ , and $E (V_{u_{i}}^{' S, T}) \in ℝ^{N \times n}$ , as shown in Eqs. (11) and (12). $E (V_{u_{i}}^{S, T}) = sum (E (V_{u_{i}}^{S})) + sum (E (V_{u_{i}}^{T}))$ (11) $E (V_{u_{i}}^{' S, T}) = sum (E (V_{u_{i}}^{' S})) + sum (E (V_{u_{i}}^{' T}))$ (12)

5.3 Extended multi-head self-attention layer

In DLAN, an extended multi-head self-attention module is proposed, which aggregates the first-order and high-order neighborhood information in the user’s trajectory to obtain an updated representation of each check-in. For each Head_j, this paper performs the summation operation in Equation (14) to incorporate the trajectory spatio-temporal matrix first-order and high-order neighborhood information into the interaction. Note that the number of heads p in the multi-head attention mechanism must be divisible by the embedding dimension d, i.e. d = p × z, where z is the dimension of each self-attention space. At the same time, DLAN also assigns different weights to each head, aimed not only to focus on the impact of u_i’s recent check-in on the next moment of decision but also to capture his or her long-term preference. It is worth mentioning that the introduction of first-order neighborhood information and high-order neighborhood information can make DLAN more fully exploit users’ short-term and long-term preferences, respectively. Given the trajectory embedding matrix E (seq_{u
_i}) and the trajectory spatio-temporal embedding matrix $E (V_{u_{i}}^{S, T})$ , the output of the j - th ∈ [1, 2, ⋯ , p] space Head_j is calculated as shown as shown below: $\begin{matrix} Q_{j} & = E ({seq}_{u_{j}}) W_{j}^{Q}, K_{j} = E ({seq}_{u_{i}}) W_{j}^{K}, \\ V_{j} & = E ({seq}_{u_{i}}) W_{j}^{V}, j = 1, 2 \dots, p \end{matrix}$ (13) $\begin{matrix} {Head}_{j} & = Attention (Q_{j}, K_{j}, V_{j}, E {(V_{u_{i}}^{S, T})}_{j}) \\ = softmax (\frac{Q_{j} K_{j}^{T} + E {(V_{u_{i}}^{S, T})}_{j}}{\sqrt{d}}) V_{j} \end{matrix}$ (14) Where $W_{j}^{Q}, W_{j}^{K}, W_{j}^{V} \in ℝ^{d \times z}$ are the projection matrices of the j-th head, $E {(V_{u_{i}}^{S, T})}_{j} \in ℝ^{n \times z}$ and $E {(V_{u_{i}}^{S, T})}_{j} \in E (V_{u_{i}}^{S, T})$ , respectively. After the Concact operation, a new sequence is obtained by projecting the information obtained from p spaces into the same space, and it is defined as an updated representation of the user trajectory, as shown in Equation (15), where $W^{O} \in ℝ^{d \times d}$ is a linear matrix. $R (u_{i}) = Concact ({Head}_{1}, {Head}_{2}, \dots, {Head}_{p}) W^{o}$ (15)

5.4 User similarity weighting layer

The social information of users has been easily neglected in antecedent literature. Therefore, this paper presents a user similarity weighting layer as an approach to effectively leverage the social relationships of users. By measuring the impact of diverse users on a particular target user, and harnessing the collaborative information of the latter, the module identifies potential social friends for the target user. This approach leads to enhanced recommendation precision while mitigating the adverse effects of data sparsity on recommendation performance to some extent. After obtaining updated representation R (u_i) of the trajectory of the target user u_i ∈ U through Equation (15), when the first n - 1 check-ins of u_i are passed into the model, the layer evaluates the similarity between the trajectories of user u_i and each individual, u_v ∈ U, in the social network using cosine similarity as shown in Equation (16). Subsequently, the top q users with high similarity to u_i’s check-in trajectory are searched for as potential social friends of u_i to build a virtual social relationship for him/her, denoted by $U_{q} = {u_{1}^{'}, u_{2}^{'}, \dots, u_{q}^{'}}$ . $w_{i, v} = \frac{\sum_{l} c_{i, l} \sum_{l} c_{v, l}}{\sqrt{\sum_{l} c_{i, l}^{2}} \sqrt{\sum_{l} c_{v, l}^{2}}}$ (16) Where c_i,l ∈ R (u_i), c_v,l ∈ R (u_v), w_i,v ∈ [0, 1]. w_i,v is larger, the higher the similarity of the trajectories between u_i and u_v. Subsequently, in Equation (17) the DLAN weights the target user u_i with each of his or her potential friend u′_k ∈ U_q. Furthermore, to reconcile the importance between the target user and his or her potential social friends, a tuning parameter ∂ is set between them in Equation (18) in this paper, and finally, the final trajectory update representation $R^{'} (u_{i}) \in ℝ^{n \times d}$ of the target user u_i is obtained. $R^{'} (U_{q}) = \sum_{k = 1}^{q} \frac{w_{i, 1}}{\sum_{j = 1}^{q} w_{i, j}} R (u_{k}^{'})$ (17) $R^{'} (u_{i}) = \partial R (u_{i}) + (1 - \partial) R^{'} (U_{q})$ (18)

5.5 Attention matching layer

In this layer, this paper proposes an Attention Matching Layer that uses the output R′ (u_i) of the User Similarity Weighting Layer to calculate the probability of the POIs recommended from the candidate set L. In the Embedding layer, the DLAN obtains the representation $E (L) = {e^{l_{1}}, e^{l_{2}}, \cdot, e^{l_{N}}} \in ℝ^{N \times d}$ of the candidate POIs set L. Given the candidate POIs spatio-temporal interval matrix embedding $E (V_{u_{i}}^{' S, T}) \in ℝ^{N \times n}$ , the probability a (L) ∈ R^N of all candidate POIs is calculated as shown in Equation (19) using the final trajectory update representation R′ (u_i) and the candidate POIs embedding E (L). $\begin{matrix} a (L) = sum \\ (softmax (\frac{E (L) R^{'} {(u_{i})}^{T} + E (V_{u_{i}}^{' S, T})}{\sqrt{d}})) \end{matrix}$ (19) Where the sum function converts the dimension of a (L) to $ℝ^{N}$ . In Equation (19), it is not difficult to notice that the module differs from the general self-attention in that it accounts for the spatio-temporal association information between each check-in in the trajectory and the possible next POI in the candidate set.

5.6 Output layer

In this section of this document, the main task is to select the top-k POIs recommended by the model that are most likely to be visited by users to generate a recommendation list. Before training the model, typically a loss function needs to be set to optimize the model. With the next POI recommendation, however, the number of POIs is very large and users cannot visit all of them, but only a relatively small number of POIs, so the positive and negative class samples are unbalanced and on which we cannot calculate the loss function for each prediction class. For addressing this issue, our proposed model employs the Balanced Sampler method proposed by STAN [21] to negatively sample the candidate set probability a and calculate the loss by randomly selecting G ={ g₁, g₂, ⋯ , g_b } in each training step, where b represents the number of negative samples as a hyperparameter in this model. In addition, after each round of computation, we update the random seeds of the negative sample sampler. Given the label POI, namely the target location l_i ∈ L, the candidate POI probability a (L), and the set of negative samples G, the loss function is calculated as follows: $L = [log (a (l_{i})) + \sum_{j = 1}^{g} log (1 - a (g_{j}))]$ (20)

6 Experiments

6.1 Datasets

The present study evaluates the performance of the DLAN model using authentic user check-in datasets from two cities. Specifically, this paper utilizes user check-in data obtained from the period between April 2012 and February 2013 in New York City (NYC) and Tokyo City (TKY), which have been extensively adopted in the field of the next POI recommendation [30] 1 . They are pre-processed in this paper to exclude undesirable locations with less than 10 check-in records, and to filter users with less than 10 check-in history records to alleviate the sparsity of the dataset. The pre-processed results are shown in yellow Table 1.

6.2 Baseline methods

This paper compares the DLAN model with the following baseline models.

•FPMC-LR [7] coupled Matrix Factorization and Markov Chain together to model both user’s long-term preference and sequential behavior.

•PRME [15] presented a pairwise embedding method for personalized ranking metric embedding.

•ST-RNN [8] utilizes a temporal, distance transition matrix to model the local temporal and spatial context in addition to the RNN used to collect user-generated sequential patterns.

•STGN [9] expands the traditional LSTM with the addition of spatial and temporal gates to capture the user preferences in spatial and temporal dimensions.

•STGCN [9] is an enhanced version of the STGN, employing coupled inputs and forgetting gates.

•PLSPL [19] learns users’ long-term preferences by an attention mechanism and short-term preferences by a standard LSTM.

•STAN [21] captures the influence of spatio-temporal information between non-adjacent check-ins in a user’s check-in trajectory on recommendations using a self-attention layer.

Table 1
Statistics of NYC and TKY datasets

Dataset #Users #POIs #Check-ins #Sparity

NYC 1083 5135 147983 97.34%

TKY 2293 8180 438247 97.66%

Dataset	#Users	#POIs	#Check-ins	#Sparity
NYC	1083	5135	147983	97.34%
TKY	2293	8180	438247	97.66%

6.3 Experiment settings

In this paper, the datasets are divided into three datasets: training, validation, and testing. To ensure the accuracy of the experiments, this study adopts h to represent the experimental actual length. When m < n, DLAN takes m as the actual length of the experiment, i.e., h = m, and when m ≥ n, DLAN still employs n as its actual length, that is, h = n. In accordance with causal methodology, the present paper adheres to the principle of non-utilization of future data for prediction purposes during dataset segregation. The delineation strategy adopted in this study entails the following. The number of check-ins in the training set is h - 3, and the first m′ ∈ [1, h - 3] check-ins are used as training samples. For check-in m′, the (m′ + 1) - th ∈ [2, h - 2] check-in is used as the training label. The validation set takes the first h - 2 check-in sequences as input, and the (h - 1)-th check-in as the label. The test set makes use of the first h - 1 check-ins as input to predict the POI of the user’s h-th check-in.

Subsequently, in this experiment, we set some hyperparameters of the DLAN by specifying the embedding dimension d as 30, the number of heads p as 5, and the tuning parameter ∂ as 7/8. Additionally, the number of potential social friends q is set to 2. The DLAN utilizes the Adam optimizer with default betas, employing a learning rate of 0.004 and a dropout rate of 0.2. Training involves 50 epochs, a batchsize of 16, and the selection of 128 users at each epoch. The maximum user trajectory sequence length is set at n = 100. Finally, in determining the loss calculation process, the optimal number of negative samples b is identified to be 10.

6.4 Evaluation metric

In this study, DLAN is evaluated with Accuracy and Mean Reverse Rank (MRR), which are common evaluation metrics for recommended systems. The accuracy indicates whether the POI actually accessed by the user at the next moment is in the top-k recommendation list, which is denoted as Acc@k in this paper. Because Acc@k treats top-k recommendations as an unordered list and ignores the correct prediction ordering, this paper calculates the index of the correctly recommended POIs in the ordered result list of MRR. Acc@k and MRR are defined as shown in Eqs. (21) and (22), respectively. $Acc @ k = \frac{1}{| U |} \sum_{i \in [1, | U |]} I (l_{u_{i}}, top - k (u_{i}))$ (21) $MRR = \frac{1}{| U |} \sum_{i \in [1, | U |]} \frac{1}{{rank}_{l_{u_{i}}}}$ (22) Where l_{u
_i} is the real POI visited by user u_i and top - k (u_i) is the recommendation list. $I$ is the indicator function, which returns 1 if the condition is true and 0 otherwise. rank_{l
_{u
_i}} indicates the position of the next POI in the recommendation list that the user really visited.

6.5 Experiments results

In this study, DLAN assesses two publicly available datasets, NYC and TKY, and is compared to seven baseline methods, with performance evaluated based on MRR metric of top-20 and the accuracy of all methods on top-5, top-10, top-15, and top-20, as displayed in Table 2. Analyzing the results, it becomes evident that FPMC-LR and PRME perform the worst across all metrics compared with RNN-based methods such as ST-RNN, STGN, STGCN, and PLSPL. The limitations of conventional methods, which only consider users’ recent check-ins, but ignore their long-term preferences. In contrast, RNN-based methods’ ability to capture more nuanced behavioral patterns resulted in more precise recommendations. Notably, the utilization of ST-GCN leads to significant performance improvements relative to ST-RNN on both urban datasets. This is probably due to the difficulties associated with manually adjusting time and distance windows in ST-RNN experiments, which are circumvented by introducing time and distance gates in ST-GCN. This reduces the parameter count while amplifying performance outcomes. Moreover, ST-GCN is surpassed by PLSPL on the same dataset, thanks to the adoption of a sequence segmentation technique that enables the extraction of both short- and long-term user preferences, thereby enhancing the model’s POI-capturing capacity. STAN’s superior performance can be ascribed to its attention mechanism, which overcomes the RNN model’s inherent limitations and accurately captures POIs of interest in users’ check-in sequences. DLAN outperforms STAN on two datasets, a result attributes to its multi-head attention mechanism that effectively partitions the user’s check-in sequence into several independent spaces and assigns distinct weight to each space. DLAN leverages social information and collaborative data to mitigate the negative effects of data sparsity and improve overall performance. Incorporating first-order and high-order neighborhood information from the user’s check-in trajectory in each space allows DLAN to extract the implicit short- and long-term preferences of users in parallel, enhancing recommendation efficiency.

Table 2
Recommendation performance comparison with baselines

NYC TKY

Acc@5 Acc@10 Acc@15 Acc@20 MRR@20 Acc@5 Acc@10 Acc@15 Acc@20 MRR@20

FPMC-LR 0.2126 0.2970 0.3378 0.3585 0.1431 0.1787 0.2483 0.2979 0.3396 0.0874

PRME 0.2236 0.3105 0.3704 0.3930 0.1440 0.1991 0.2664 0.3073 0.3505 0.1161

ST-RNN 0.2923 0.3622 0.4577 0.4858 0.1850 0.2641 0.3237 0.4103 0.4679 0.1438

STGN 0.3381 0.4122 0.5101 0.5413 0.2186 0.2964 0.3464 0.3946 0.4444 0.1575

STGCN 0.3425 0.4279 0.5301 0.5626 0.2346 0.3018 0.3554 0.4122 0.4689 0.1628

PLSPL 0.3678 0.4523 0.5528 0.5794 0.2361 0.3079 0.3756 0.4253 0.4805 0.1653

STAN 0.4582 0.5734 0.6434 0.6828 0.2738 0.3320 0.4040 0.4420 0.5040 0.1855

DLAN 0.5625 0.6796 0.6953 0.7421 0.3742 0.3828 0.4375 0.5156 0.5703 0.2381

Improvement 22.76% 18.52% 8.07% 8.68% 36.67% 15.30% 8.29% 16.65% 13.15% 28.35%

NYC	TKY
FPMC-LR	0.2126	0.2970	0.3378	0.3585	0.1431	0.1787	0.2483	0.2979	0.3396	0.0874
PRME	0.2236	0.3105	0.3704	0.3930	0.1440	0.1991	0.2664	0.3073	0.3505	0.1161
ST-RNN	0.2923	0.3622	0.4577	0.4858	0.1850	0.2641	0.3237	0.4103	0.4679	0.1438
STGN	0.3381	0.4122	0.5101	0.5413	0.2186	0.2964	0.3464	0.3946	0.4444	0.1575
STGCN	0.3425	0.4279	0.5301	0.5626	0.2346	0.3018	0.3554	0.4122	0.4689	0.1628
PLSPL	0.3678	0.4523	0.5528	0.5794	0.2361	0.3079	0.3756	0.4253	0.4805	0.1653
STAN	0.4582	0.5734	0.6434	0.6828	0.2738	0.3320	0.4040	0.4420	0.5040	0.1855
DLAN	0.5625	0.6796	0.6953	0.7421	0.3742	0.3828	0.4375	0.5156	0.5703	0.2381
Improvement	22.76%	18.52%	8.07%	8.68%	36.67%	15.30%	8.29%	16.65%	13.15%	28.35%

6.6 Ablation study

Aiming to validate the impact of different modules of DLAN on model performance, this paper compares DLAN with its four variants: (1) DLAN-VV’: To observe the impact of the first-order neighborhood information and high-order neighborhood information in the user check-in trajectory and the spatio-temporal correlation information between the POIs in the check-in trajectory and the set of candidate POIs on the recommendation performance, DLAN is used in Eqs. (14) and (19) without performing the addition of the trajectory spatio-temporal interval embedding matrix $E {(V_{u_{i}}^{S, T})}_{j}$ and the candidate POIs spatio-temporal interval embedding matrix $E (V_{u_{i}}^{' S, T})$ . (2) DLAN-U: The target user’s potential social friends are also a critical factor affecting his or her decision at the next moment. To test this view, DLAN removes the User Similarity Weighting module to observe the impact of the target user’s social information on his or her decision-making. (3) DLAN-MHA: Our model proposes a multi-head attention module that incorporates first-order neighborhood information and high-order neighborhood information from user check-ins. To evaluate the benefits of this module in extracting user preferences and enabling parallel, in Extend Multi-Head Self-Attention Layer, we set the number of heads p of the multi-head attention mechanism to 1, that is, only a single Self-Attention is used to mine user preferences. (4) DLAN-BASE: In DLAN-BASE, DLAN simultaneously removes the embedding of the two spatio-temporal interval matrices, the user similarity weighting module, and sets the number of heads p to 1 in the Extend Multi-Head Self-Attention Layer to demonstrate the performance advantages of the DLAN model double-layer attention network.

The performance of DLAN with its four variants on the two city datasets through experiments is shown in Table 3. In Table 3, it is not difficult to find that the performance of the complete DLAN is remarkably better than that of its four variants. Specifically, DLAN increases the accuracy by about 2.31% -32.45% and MRR@20 by about 17.85% and 29.89% on the NYC and TKY datasets compared with DLAN-VV’, indicating that the incorporation of both first-order and high-order neighborhood information from user check-in trajectories into the interaction can assist DLAN to adequately explore users’ short-term and long-term preferences, respectively. It also illustrates that aggregating the spatio-temporal correlation information between the POIs in the user check-in trajectory and the candidate POIs set in the attention matching layer can obtain more accurate candidate POI probabilities. DLAN can achieve a significant boost in accuracy, with improvements ranging from 2.14% to 32.45%, as well as an increase in MRR@20 of 11.57% and 13.92% on the two city datasets when compared to DLAN-U. The observed performance gains are arguably attributed to DLAN’s consideration of the effect of potential social friends within the target user’s social network and leveraging the collaborative information of the target user. As can be seen from Table 3, DLAN demonstrates a significant improvement in accuracy and MRR@20, ranging from 2.87% to 19.51% and 17.21% to 49.65%, respectively, over DLAN-MHA on two city datasets. These findings suggest that the DLAN-MHA model presents limitations when exclusively utilizing a self-attention mechanism to extract potential information from user check-in sequences. Specifically, when processing longer sequences, DLAN-MHA fails to effectively and concurrently identify users’ preferences, resulting in lower efficiency. Meanwhile, single-headed attention tends to cause the overfitting phenomenon. The multi-head attention mechanism implemented in DLAN is capable of addressing this issue by employing multiple individual attention spaces to fully explore users’ long- and short-term preferences in parallel, leading to enhanced efficiency and accuracy in recommendations. It is worth noting that when DLAN removes all modules, DLAN-BASE outperforms some previous RNN models, which indicates that a double-layer attention network has a great advantage over RNN-based models in handling long sequences.

Table 3
Ablation Analysis, in which we compare different modules in DLAN

NYC TKY

Acc@5 Acc@10 Acc@15 Acc@20 MRR@20 Acc@5 Acc@10 Acc@15 Acc@20 MRR@20

DLAN-VV’ 0.4609 0.6250 0.6796 0.7031 0.3175 0.2890 0.3671 0.4375 0.5078 0.1833

DLAN-U 0.5078 0.6406 0.6953 0.7265 0.3335 0.2890 0.3828 0.4453 0.5000 0.2090

DLAN-MHA 0.5468 0.6328 0.6953 0.7109 0.3195 0.3203 0.3984 0.4687 0.5234 0.2015

DLAN-ALL 0.4609 0.5859 0.6328 0.6796 0.2792 0.2812 0.3750 0.4296 0.4687 0.1591

DLAN 0.5625 0.6796 0.6953 0.7421 0.3742 0.3828 0.4375 0.5156 0.5703 0.2381

NYC	TKY
DLAN-VV’	0.4609	0.6250	0.6796	0.7031	0.3175	0.2890	0.3671	0.4375	0.5078	0.1833
DLAN-U	0.5078	0.6406	0.6953	0.7265	0.3335	0.2890	0.3828	0.4453	0.5000	0.2090
DLAN-MHA	0.5468	0.6328	0.6953	0.7109	0.3195	0.3203	0.3984	0.4687	0.5234	0.2015
DLAN-ALL	0.4609	0.5859	0.6328	0.6796	0.2792	0.2812	0.3750	0.4296	0.4687	0.1591
DLAN	0.5625	0.6796	0.6953	0.7421	0.3742	0.3828	0.4375	0.5156	0.5703	0.2381

6.7 Sensitivity study

6.7.1 Embedding dimension

In this section, we test the effect of embedding dimension d of the embedding module on the performance of DLAN. In this paper, we adopt d = [10, 2, 30, 40, 50, 60] and perform the test on two city datasets, and the results are shown in Fig. 2. Figure 2 demonstrates the trend of the accuracy of DLAN on the dataset of two cities when the embedding dimension d is changed. It can be observed from Fig. 2 that the model achieves optimal performance at d = [30, 40]. The DLAN performance is poor when d < 30, which is mainly due to the fact that too small an embedding dimension d will lead to less information available to the model during training and inadequate training. Moreover, when d > 40, the DLAN performance decreases due to the overfitting phenomenon.

Fig. 2

Impact of embedding dimension.

6.7.2 Number of heads of the multi-head self-attention

In this section, we examine the effect of different head numbers on the performance of DLAN. As the head number p must be divisible by the embedding dimension d and DLAN can capture the long- and short-term preferences of users, this paper takes p = [2, 3, 5, 6, 10, 15, 30] for the experiment, and the specific results are shown in Fig. 3. Figure 3 shows the trend of the accuracy of DLAN on the two city datasets when p varies. It can be observed that when p = [2, 3, 5], the performance of DLAN on two real datasets shows an increasing trend, and the optimal performance is reached at p = 5, which is attributed to the fact that as the number of heads increases, the multi-head attention mechanism can accurately choose the locations of greater interest to the user. The performance shows a decreasing trend when p = [6, 10] and plateaus when p = [15, 30]. Consequently, the accuracy does not present a trend of increasing or decreasing all the time when the number of heads keeps increasing. This is owing to the reason why if the number of heads is too large, the sequential nature of the user check-in sequence is destroyed and the dependencies between check-ins cannot be captured effectively.

Fig. 3

Impact of number of heads.

7 Conclusion

In this paper, we propose a novel Double-Layer Attention Network, namely DLAN, which not only concentrates on incorporating the first-order neighborhood information and high-order neighborhood information in the user’s trajectory as well as the user’s social relationship information into the interaction, but also adequately extracts the user’s long- and short-term preferences from his check-in trajectory to accomplish a more accurate next POI recommendation task. Specifically, DLAN contains a multi-head attention mechanism module integrating first-order neighborhood information and high-order neighborhood information from the user’s check-in trajectory. The module can selectively focus on the user’s recent preferred locations and the user’s long-term behavior patterns, that is, the user’s long- and short-term preferences, from the user’s trajectory sequence. For the purpose of introducing users’ social information, a user similarity weighting layer is also designed in the DLAN to measure the influence of other friends on the target user on user’s decision. Through a series of adequate experiments on two real datasets, NYC and TKY, this paper demonstrates the advantages of DLAN by not only significantly outperforming the current mainstream models but also validating its different components. It indicates that incorporating users’ first-order neighborhood information, high-order neighborhood information, and social relationship information into the interactions, as well as fully mining users’ long-term and short-term preferences, is necessary to improve the performance of the next POI recommendation.

Acknowledgments

This work is supported by Tianjin “Project + Team" Key Training Project under Grant No.XC202022. This work was carried out during the tenure of an ERCIM ’Alain Bensoussan’ Fellowship Programme.

Footnotes

http://www-public.imtbs-tsp.eu/∼zhang_da/pub/dataset_tsmc2014.zip

References

Xinghe Cheng , Ning Li , Gulsim Rysbayeva , Qing Yang , Jingwei Zhang Influence-aware successive point-ofinterest recommendation, World Wide Web (WWW), 2023.

Song Yang , Jiamou Liu , Kaiqi Zhao Getnext: Trajectory flow map enhanced transformer for next POI Recommendation, In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11–15, 2022, pages 1144–1153. ACM, 2022.

Nicholas Lim , Bryan Hooi , See-Kiong Ng , Yong Liang Goh , Renrong Weng , Rui Tan Hierarchical multi-task graph recurrent network for next POI Recommendation, In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11–15, 2022. ACM, 2022.

Zheng Huang , Jing Ma , Yushun Dong , Natasha Zhang Foutz , JundongLi Empowering next POI recommendation with multi-relational modeling, In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11–15, 2022. ACM, 2022.

Yong Liu , Wei Wei , Aixin Sun , Chunyan Miao Exploiting geographical neighborhood characteristics for location Recommendation, In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, CIKM 2014, Shanghai, China, November 3–7, 2014, pages 739–748. ACM, 2014.

Jingkuan Wang , Bo Yang , Haodong Liu , Dongsheng Li Global spatio-temporal aware graph neural network for next point-of-interest recommendation, Appl. Intell., pages 16762–16775, 2023.

Chen Cheng , Haiqin Yang , Michael Lyu

, Irwin King Where you like to go next: Successive point-ofinterest Recommendation, In IJCAI 2013, Proceedings of the 23rd International Joint Conference on Artificial Intelligence, Beijing, China, August 3–9, 2013, pages 2605–2611. IJCAI/AAAI, 2013.

Qiang Liu , Shu Wu , Liang Wang , Tieniu Tan Predicting the next location: A recurrent model with spatial and temporal contexts, In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12–17, 2016, Phoenix, Arizona, USA, pages 194–200. AAAI Press, 2016.

Pengpeng Zhao , Haifeng Zhu , Yanchi Liu , Jiajie Xu , Zhixu Li , FuzhenZhuang , Victor Sheng

, Xiaofang Zhou Where to go next: A spatio-temporal gated network for next POI Recommendation, In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27–February 1, 2019, pages 5877–5884. AAAI Press, 2019.

10.

Ke Sun , Tieyun Qian , Tong Chen , Yile Liang , Quoc Viet Hung Nguyen , Hongzhi Yin Where to go next: Modeling long- and short-term user preferences for pointof-interest recommendation, In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020, pages 214–221. AAAI Press, 2020.

11.

Dietmar Jannach , Lukas Lerche , Michael Jugovac Adaptation and evaluation of recommendations for short-term shopping Goals, In Proceedings of the 9th ACM Conference on Recommender Systems, RecSys ’15, page 211–218. Association for Computing Machinery, 2015.

12.

Nicholas Lim , Bryan Hooi , See-Kiong Ng , Xueou Wang , Yong Liang Goh , Renrong Weng , Jagannadan Varadarajan Stp-udgat: Spatial-temporal-preference user dimensional graph attention network for next poi recommendation, In Proceedings of the 29th ACM International Conference on Information&Knowledge Management, CIKM ’20, pages 845–854. Association for Computing Machinery, 2020.

13.

Hong Cheng , Jihang Ye , Zhe Zhu What’s your next move: User activity prediction in location-based social networks, In Proceedings of the 13th SIAM International Conference on Data Mining, May 2–4, 2013. Austin, Texas, USA, pages 171–179. SIAM, 2013.

14.

Jia-Dong Zhang , Chi-Yin Chow , Yanhua Li LORE: exploiting sequential influence for location recommendations, In Proceedings of the 22nd ACM SIGSPATIAL International Conference on Advances in Geographic Infor mation Systems, Dallas/Fort Worth, TX, USA, November 4–7, 2014, pages 103–112. ACM, 2014.

15.

Shanshan Feng , Xutao Li , Yifeng Zeng , Gao Cong , Yeow Meng Chee , Quan Yuan Personalized ranking metric embedding for next new POI Recommendation, In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25–31, 2015, pages 2069–2075. AAAI Press, 2015.

16.

Yanchi Liu , Chuanren Liu , Bin Liu , Meng Qu , Hui Xiong Unified point-of-interest recommendation with temporal interval Assessment, In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 1015–1024. Association for Computing Machinery, 2016.

17.

Shenglin Zhao , Tong Zhao , Haiqin Yang , Michael Lyu

, IrwinKing STELLAR: spatial-temporal latent ranking for successive point-of-interest recommendation, In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12–17, 2016, Phoenix, Arizona, USA, pages 315–322. AAAI Press, 2016.

18.

Yu Zhu , Hao Li , Yikang Liao , Beidou Wang , Ziyu Guan , Haifeng Liu , Deng Cai What to do next: Modeling user behaviors by time-lstm, In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19–25, 2017, pages 3602–3608. ijcai.org, 2017.

19.

Yuxia Wu , Ke Li , Guoshuai Zhao , Xueming Qian , Personalized long-and short-term preference learning for next POI Recommendation,Trans. Knowl. Data Eng., IEEE 34(4) (2022), 1944–1957.

20.

Jie Feng , Yong Li , Chao Zhang , Funing Sun , Fanchao Meng , Ang Guo , Depeng Jin, Deepmove: Predicting human mobility with attentional recurrent Networks, In Proceedings of the 2018WorldWideWeb Conference onWorldWide Web, WWW 2018, Lyon, France, April 23–27, 2018, pages 1459–1468. ACM, 2018.

21.

Yingtao Luo , Qiang Liu , Zhaocheng Liu STAN: spatio-temporal attention network for next location Recommendation, InWWW’21: TheWeb Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19–23, 2021, pages 2177–2185. ACM / IW3C2, 2021.

22.

Dongjing Wang , Xingliang Wang, , Zhengzhe Xiang , Dongjin Yu , ShuiguangDeng , Guandong Xu Attentive sequential model based on graph neural network for next poi Recommendation, World Wide Web, pages 2161–2184, 2021.

23.

Xiaodong Yan , Tengwei Song , Yifeng Jiao , Jianshan He Spatio-temporal hypergraph learning for next POI recommendation, In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23–27, 2023, pages 403–412.

24.

Mingxin Gan , Caiping Tan Mining multiple sequential patterns through multi-graph representation for next pointof-interest recommendation, World Wide Web (WWW), pages 1345–1370, 2023.

25.

Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , LlionJones , Aidan Gomez

, Łukasz Kaiser , Illia Polosukhin Attention is all you need, In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010. Curran Associates Inc., 2017.

26.

Zhouhan Lin , Minwei Feng, , Cícero Nogueira dos Santos , Mo Yu , Bing Xiang , Bowen Zhou , Yoshua Bengio A structured self-attentive sentence embedding, In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24–26, 2017, Conference Track Proceedings. OpenReview.net, 2017.

27.

Jacob Devlin , Ming-Wei Chang , Kenton Lee , Kristina Toutanova BERT: pre-training of deep bidirectional transformers for language Understanding, CoRR, abs/1810.04805, 2018.

28.

Yue Feng , Yan Cheng Short text sentiment analysis based on multi-channel cnn with multi-head attention mechanism, IEEE Access, pages 19854–19863, 2021.

29.

Qiang Liu , Zhaocheng Liu , Hao Zhang An empirical study on feature discretization, ArXiv, 2020

30.

Dingqi Yang , Daqing Zhang , Vincent Wenchen Zheng , Zhiyong Yu , Modeling user activity preference by leveraging user spatialtemporal characteristics in lbsns, IEEE Transactions onSystems, Man, and Cybernetics: Systems 45 (2015), 129–142.

NYC					TKY
	Acc@5	Acc@10	Acc@15	Acc@20	MRR@20	Acc@5	Acc@10	Acc@15	Acc@20	MRR@20
FPMC-LR	0.2126	0.2970	0.3378	0.3585	0.1431	0.1787	0.2483	0.2979	0.3396	0.0874
PRME	0.2236	0.3105	0.3704	0.3930	0.1440	0.1991	0.2664	0.3073	0.3505	0.1161
ST-RNN	0.2923	0.3622	0.4577	0.4858	0.1850	0.2641	0.3237	0.4103	0.4679	0.1438
STGN	0.3381	0.4122	0.5101	0.5413	0.2186	0.2964	0.3464	0.3946	0.4444	0.1575
STGCN	0.3425	0.4279	0.5301	0.5626	0.2346	0.3018	0.3554	0.4122	0.4689	0.1628
PLSPL	0.3678	0.4523	0.5528	0.5794	0.2361	0.3079	0.3756	0.4253	0.4805	0.1653
STAN	0.4582	0.5734	0.6434	0.6828	0.2738	0.3320	0.4040	0.4420	0.5040	0.1855
DLAN	0.5625	0.6796	0.6953	0.7421	0.3742	0.3828	0.4375	0.5156	0.5703	0.2381
Improvement	22.76%	18.52%	8.07%	8.68%	36.67%	15.30%	8.29%	16.65%	13.15%	28.35%