A new user similarity model using item genre information for collaborative filtering

Abstract

Collaborative filtering is one of the most widely used approaches in recommendation systems. To enhance the performance of collaborative filtering, it is important to accurately identify similar users or items using a user-item rating matrix. Most similarity measures used in user-based collaborative filtering calculate similarity using the co-rated items between users. However, they suffer from the cold-start problem, because the user-item rating matrix is generally very sparse. To address this issue, this study proposes new similarity measures that exploit item genre information. The proposed similarity measures utilize either the number of rated items per genre or the average rating of genres and they were evaluated by different rating prediction methods including methods to aggregate the predicted ratings for multiple genres to which the item belongs. In addition, this study also proposed an approach to hybridize the proposed similarity measure with conventional similarity measures that do not exploit item genre information. The performance of the proposed similarity measures and the hybrid similarity measures was evaluated using three metrics such as MAE, F1, and NDCG and the empirical results revealed which proposed measures were superior to the others and confirmed the effectiveness and usefulness of the hybrid similarity measures.

Keywords

recommendation systems collaborative filtering similarity measure user similarity genre information

1. Introduction

A recommendation system is a type of information filtering system that aims to provide relevant items to users. In many commercial areas, searching for appropriate or preferred items can be a difficult task for users, especially when the number of items is very large. For this reason, recommendation systems have been commonly used in many commercial systems such as movies, books, e-commerce, and online learning.^1,2 In general, recommendation systems gather a large amount of explicit and implicit data that can reflect users’ preferences or define the characteristics of items and make predictions about users’ tastes. Because such systems often rely on sensitive behavioral data, privacy-preserving learning frameworks have recently attracted increasing attention in related intelligent systems research.³

Collaborative filtering (CF) is one of the most frequently used in recommendation systems and is regarded as one of the best techniques for building recommendation systems.⁴ In CF, there are three approaches; memory-based, model-based and hybrid. Memory-based methods use similarity metrics between users or items and predict new ratings based on average ratings between pairs of similar users or items.^5,6 Meanwhile, model-based methods develop a model that can provide recommendations based on a historical user-item database.^7,8 Memory-based methods are simple, easy to implement, and interpretable, as they directly use the data without requiring additional model training. However, they suffer from scalability issues, particularly with large datasets, as similarity calculations need to be performed for all users or items. On the other hand, model-based methods can handle larger datasets more efficiently and are better at generalizing to unseen users or items. These methods often provide more accurate predictions than memory-based approaches, especially in sparse datasets. However, they require more computational resources due to the need for training complex models and can be more difficult to interpret compared to memory-based methods.

The hybrid approach integrates both memory-based and model-based methods, combining the strengths of each to improve recommendation performance. It addresses the scalability issues of memory-based methods while maintaining interpretability and improves the accuracy of model-based methods by refining predictions. This makes hybrid methods particularly useful in large-scale, real-world recommendation systems, as they balance efficiency and prediction quality. Beyond conventional CF methods, optimization-based intelligent approaches have also been explored in related prediction tasks to improve feature selection and predictive performance.⁹

In general, memory-based methods are popular in commercial recommendation systems¹⁰ due to their simplicity, ease of implementation, and interpretability. One of the primary advantages of memory-based CF is its straightforward approach, where recommendations are generated based on the similarity between users or items. This makes it particularly useful in environments where transparency and ease of understanding are essential, such as personalized product or movie recommendations. However, memory-based CF has notable limitations when applied to large-scale commercial systems. As the number of users and items increases, the method faces scalability challenges because similarity calculations need to be performed for every user or item. Despite these limitations, memory-based CF remains a popular choice in commercial applications with relatively smaller datasets or when real-time computational efficiency is crucial.

Memory-based CF is typically based on processing the user-item matrix that contains the ratings assigned by each user to items and it can be divided into user-based CF (UBCF) and item-based CF (IBCF) depending on whether ratings of items are predicted from similar users or similar items. In UBCF, ratings of items for each user are predicted based on the ratings of similar other users based on the assumption that if user $u$ shows a similar preference to another user $v$ in some items, user $u$ is more likely to have a preference similar to that user $v$ on different items.

The quality of recommendations by UBCF highly depends on a similarity metric to find similar users for each user. Therefore, many researchers have proposed new similarity measures to improve recommendation performance. In some studies, the recommendation performance was enhanced by modifying or combining traditional similarity measures such as Pearson correlation coefficient, cosine, Jaccard and mean squared difference.^10–12 Other studies have proposed new similarity measures to capture different aspects of rating preferences. The similarity measure, named PIP in Ahn¹³ combines three different similarity measures to reflect the proximity, impact and popularity of user ratings.

Various similarity metrics for UBCF have been proposed, but they are still poor in sparse datasets, especially for cold-start users. The cold-start problem is a well-known challenge in collaborative filtering-based recommendation systems, occurring when there is insufficient data for new users or items.¹⁴ This problem can lead to poor performance in recommendation accuracy, as collaborative filtering methods rely heavily on user-item interaction data to compute similarities and make predictions.

One possible approach to alleviate this issue is to utilize genre or category information. Genre information can be easily obtained for movies, music and books and the number of genres is much smaller than the number of items. Hence, a user-genre matrix is considerably less sparse than a user-item matrix. In addition, genre preference is one of an important characteristic that infers users’ behaviors. However, incorporating genre information in similarity calculation is difficult to find in similarity measures for UBCF, while there are some similarity measures to utilize genre information for IBCF.

This study suggests new similarity measures based on the gerne information of items for UBCF to enhance recommendation quality. The main contributions of this study are summarized as follows.

–
This study proposes new genre-based similarity measures to consider genre-specific information such as the relative frequency or mean rating for each genre.
–
This study comprehensively compares the performance of several proposed similarity measures obtained by different combinations of used genre-specific information and the order of differences in genre-specific information between users.
–
This study proposes a scaling method for the proposed similarity measures to combine them with other similarity measures that do not use genre information
–
This study validates the effectiveness of hybridization of the proposed genre-based similarity measure and other similarity measures.

The rest of the paper is organized as follows. First, a brief review of memory-based CF and similarity measures for UBCF is provided in Section 2. The proposed genre-based similarity measures and the method to combine the proposed measures with other similarity measures are explained in Section 3. The experimental procedures and results are presented in Section 4. Finally, implications and conclusions of the study are presented in Section 5.
2. Background and related works

A collaborative filtering recommendation has been widely used in many commercial systems.^15,16 CF filters out items that a user might like on the basis of preferences or reactions by similar users. In this section, we first discuss the definition of the CF problem. Among collaborative filtering approaches, we describe memory-based collaborative filtering, especially user-based collaborative filtering, because the aim of this study is to propose new similarity measures for UBCF. Then, we review related work including some limitations of traditional similarity approaches and advanced similarity measures to enhance the performance of recommendation.

2.1. Collaborative filtering problem

Memory-based collaborative filtering utilizes users’ historical preferences on a set of items to predict the ratings of unrated items. Let $R = [r_{u i}]$ be a given rating matrix with $m$ users and $n$ items to be used for CF. $r_{u i}$ represents the rating value of item $i$ evaluated by user $u$ . The main goal of CF is to find a subset of the unrated items that a specific user might like. To achieve this goal, CF estimates the ratings of unrated items. To predict ratings of unrated items, the first step of CF is to obtain the similarity between users to find the $k$ -nearest neighbors of each user. The second step of UBCF is to predict the ratings of unrated items by considering ratings of similar users as follows.

{\hat{r}}_{u, i} = {\bar{r}}_{u} + \frac{\sum_{v = 1}^{k} s i m (u, v) (r_{v, i} - {\bar{r}}_{v})}{\sum_{v = 1}^{k} | s i m (u, v) |}

(1)

where

{\hat{r}}_{u i}

represents the predicted rating value of item

i

by user

u

{\bar{r}}_{u}

is the mean rating value of user

u

, and

s i m (u, v)

is the similarity value between users

u

and

v

. To enhance the user experience related to recommendations, it is very important to improve the prediction accuracy of recommendation systems. In particular, it is essential to accurately measure the similarity between users for UBCF. Therefore, many researchers have proposed new similarity measures to improve the accuracy.

2.2. Predefined similarity measures

Representative similarity measures such as the Pearson correlation coefficient (PCC), cosine similarity (COS), and the mean squared differences (MSD) have also been widely used in recommendation systems.^17,18 COS measures the angle between two rating vectors, with a smaller angle indicating greater similarity.¹⁵ However, COS does not account for differences in users’ rating scales and always yields a similarity of 1 when only one item is co-rated. Additionally, it primarily considers the direction of rating vectors while ignoring their ignores the lengths.

To consider the preference of the user’s rating, the adjusted cosine similarity measure (ACOS) has been proposed. In COS, raw rating values are directly used, which can lead to misleading similarity scores when users have different rating tendencies. However, ACOS mitigates this issue by centering ratings around each user’s mean rating before computing similarity.¹³ This adjustment removes individual biases in rating scales, making similarity comparisons more reliable. Specifically, by subtracting the user’s average rating from each rating, ACOS ensures that similarity is based on relative rather than absolute ratings, leading to a more accurate measure of users’ preferences.

PCC, one of the popular similarity measures for CF, estimates the similarity between users as the ratio of the cross product of overrating or underrating of means divided by the product of the sum of squares of the mean rating difference.¹⁹ Unlike COS, the range of cosine similarity is -1 to 1 where -1 indicates a negative correlation between users, 0 indicates the neural correlation between users and +1 indicates a positive correlation between users. PCC also has not only limitations similar to COS, but also it cannot be defined on a pair of users with only one common item.

To consider the influence of positive and negative ratings in the similarity calculation, the constrained Pearson correlation coefficient (CPCC) has been proposed.²⁰ CPCC introduces a constraint to mitigate the impact of small sample sizes and biases in rating distributions. Unlike PCC, CPCC replaces the user’s mean rating with a fixed reference value, often the global average rating or a predefined neutral rating (e.g., the middle of the rating scale).

Another enhancement to PCC considers the number of co-rated items between users, as similarity estimates are generally more reliable when users have more commonly rated items. To address this, the weighted Pearson correlation coefficient (WPCC)²¹ and sigmoid function-based Pearson correlation (SPCC)²² have been proposed. WPCC improves PCC by assigning higher weights to user pairs with more co-rated items, enhancing the reliability of similarity estimates. Meanwhile, PCC can produce extreme similarity scores (close to -1 or 1) even when only a few items are co-rated, leading to overconfident predictions. SPCC mitigates this issue by applying a sigmoid function to smooth similarity scores, reducing extreme values and improving stability.

JAC and MSD are other widely used similarity measures in CF. JAC mainly focuses on the number of co-rated items of two users.²³ JAC only considers the ratio of the cardinality of co-rated items to the cardinality of all items rated by both users, so it cannot reflect ratings. MSD is calculated by the ratio of sum square of the difference of ratings on co-rated items to the cardinality of co-rated items.⁶ Unlike JAC, MSD does not consider the number of co-rated items. Moreover, MSD assumes a linear relationship between users or items, which may not always be true in real-world scenarios. It is also highly sensitive to outliers; if a few ratings significantly deviate from the mean, they can disproportionately influence the similarity calculation, leading to inaccurate predictions. This is particularly problematic in sparse datasets, where few ratings may be available to normalize or correct these outliers. To compensate for the limitations of JAC and MSD, a new similarity measure that combines JAC and MSD, called JMSD. JMSD combines the benefits of both, ensuring similarity is not overestimated due to minimal co-rated items while still incorporating actual rating values for better accuracy.

In addition to the studies that found ways to improve or enhance the traditional similarity measures, the studies proposing a new similarity measure for UBCF have been steadily conducted. Liu et al.²⁴ developed the Proximity, Impact, and Popularity (PIP) similarity based on the three terms such as the proximity, impact, and popularity indicators. The proximity factor considers whether the two ratings are in agreement or not from the perspective of sentiments (positive or negative) to give a penalty to ratings in disagreement and the impact factor considers how strongly an item is preferred or disliked by users. The popularity factors give larger weights for ratings that are far from the average rating of a co-rated item. The final PIP score for co-rated items is calculated as the product of these three factors.

Additionally, PIP has several limitations: it is complex, linear, and unnormalized. To address these issues,²⁴ proposed an improved version called Preference Strength Similarity (PSS), which adopts a nonlinear function and considers only the distance between two ratings. Like PIP, PSS consists of three similarity factors: proximity, significance, and singularity. The PSS score for co-rated items is computed as the product of these factors, and the overall similarity between two users is obtained by summing the PSS scores of their co-rated items. Furthermore, Liu et al.²⁴ introduced the new heuristic similarity model (NHSM), which integrates multiple factors influencing user similarity. NHSM combines the strengths of PIP and PSS while mitigating their weaknesses. Unlike traditional measures, it accounts for rating sentiment, preference strength, and co-rated item count, making it a more robust similarity measure for recommendation systems.

Sun et al.¹² developed the triangle similarity (TRI) that considers both the length and the angle of rating vectors between them and proposed the new similarity measure that integrates the triangle and Jaccard similarity (TMJ) to overcome the limitation of TRI that it only considers the co-rating users.

To address the issue in traditional similarity measures, where they sometimes select users who have not rated the target item as $k$ -nearest neighbors, a modified version of JAC, called relevant Jaccard similarity (RJAC), was developed in Bag et al.²⁵ Traditional recommender systems consider only co-rated items to determine similarity and identify nearest neighbors. However, to predict the rating of an unrated item for a specific user, the selected neighbor must have rated that item. When only co-rated items are considered, there is a problem: the target item may have no rating information, making prediction impossible. RJAC resolves this by modifying the traditional JAC to also consider the number of unco-rated items.

Gazdar and Hidri²⁶ proposed a new similarity measure, called OS, by transforming intuitive and qualitative conditions that a similarity measure should satisfy into relevant mathematical equations, such as integral equations, systems of linear differential equations, and nonlinear systems. OS consists of two factors: the percentage of non-common ratings (PNCR), which accounts for the number of co-rated items, and the absolute difference of ratings (ADF), which uses a basic similarity expression for all items rated by both users.

Despite the advancements in traditional and recent similarity measures, they still face issues such as the cold-start problem and data sparsity. These problems can lead to unreliable similarity values, as similarity is typically derived from ratings on common items.

2.3. Genre-based similarity measures

The cold-start problem, an important problem in recommendation systems, refers to the problem that arises when a recommendation system meets new users or items. Even if it is not a cold-start situation, data sparsity is common in recommendation systems, which leads to unreasonable recommendations due to reduced coverage and neighbor transitivity, because users in general rate only a limited number of items. One possible approach to address the cold-start and data sparsity problems is to utilize category or genre information. For many products and items, category or genre information is available and easily obtained and it can be used to abate the cold-start and data sparsity problems.

In Zhang and Song,²⁷ a genre-based CF algorithm was developed using the genres of items. This algorithm selects neighbors based on the user’s genre preference, comparable to those of others in the set of all users and calculates the similarity between items.

Barman et al.²⁸ proposed a new similarity measure based on item genre, named CatSim to improve the performance of IBCF. CatSim first constructs a bipartite graph with nodes representing items and genres. An edge is created between an item node and a genre node when the item belongs to the genre, with the edge weight inversely proportional to the number of genres the item belongs to. Specifically, the bipartite graph structure enables the effective capture of relationships between items and their associated genres, leading to more accurate similarity estimates by considering both item-to-item and item-to-genre connections. This structure helps address limitations of traditional IBCF models by incorporating category-level information, thereby improving recommendation quality, particularly in sparse datasets.

Lin et al.²⁹ proposed a new method of similarity calculation based on both item rating and genre. In this study, the proposed similarity is defined as the weighted average of similarity using item ratings and similarity using genre information. For similarity calculation using item ratings, it uses JAC and ACOS. To incorporate genre information, the similarity based on genres is defined by a formula that combines JAC with the distance between items based on their genre representations. JAC measures the similarity between the sets of genres associated with two items, while the distance component captures the dissimilarity between the genre representations of the items.

3. Proposed genre-based similarity measures

3.1. Proposed similarity measures

Unlike various similarity measures for UBCF, the proposed similarity measures calculate the similarity between users comparing users’ ratings by genres, not by items. It is possible to use the number of rated items per genre that can be used for similarity calculation because individual items are categorized by genre. To take into account the number of rated items per genre in similarity measures, the relative frequency of rated items for each genre is obtained as follows:

f_{u, g} = \frac{N_{u, g}}{N_{u}}

(2)

where

u

represents each user,

N_{u}

is the number of all rated items of user

u

and

N_{u, g}

is the number of items rated by user

u

for genre

g

. Next, the genre-specific rating for each user is defined as follows:

{\bar{r}}_{u, g} = \frac{\sum_{i \in g} \frac{r_{u, i}}{| G_{i} |}}{| G |}

(3)

where

i

represents each item,

r_{u, i}

is the rating of item

i

for user

u

G_{i}

is the set of genres to which item

i

belongs, and

G

is the set of all available genres. In this study,

| \cdot |

for a set represents the size of a set and thus,

| G |

is the total number of genres. Some items belong to multiple genres.

| G_{i} |

is introduced to calculate the mean rating of each genre. Moreover, few items do not have genre information depending on datasets and in a such case,

| G_{i} |

is set to

| G |

, which implies that items without genre information are assumed to belong to all genres. In addition,

{\bar{r}}_{u, g}

is normalized as follows so that the proposed similarity measures have a value between 0 and 1 regardless of the rating scale:

{\bar{r}}_{u, g}^{'} = \frac{{\bar{r}}_{u, g}}{r_{m a x}}

(4)

where

r_{m a x}

is the maximum value in the rating scale.

First, we proposed different similarity measures according to the types of genre-specific information used in the similarity calculation. Table 1 lists the proposed similarity measures. In names of similarity measures, $F$ indicates that the corresponding similarity measures use the frequency information and $R_{1}$ and $R_{2}$ indicate that the corresponding similarity measures use the rating information. The similarity measures using the rating information are further divided into two types: (1) using the original mean rating for each genre ( $R_{1}$ ) and (2) using the difference between the mean rating of each genre and the overall mean rating ( $R_{2}$ ). In the similarity measures using $R_{2}$ , ${\bar{r}}_{u}^{'}$ is the normalized average rating value of user $u$ for all rated items by user $u$ . The superscripts, $1$ and $2$ , represent the order of differences in genre-specific information between two users. For $R_{2}^{1}$ and ${F R}_{2}^{1}$ , 0.5 is multiplied to the second term (the summation part) to make this term within the range of 0 to 1. Similar to $R_{2}^{1}$ and ${F R}_{2}^{1}$ , 0.25 is multiplied to the second term for $R_{2}^{2}$ and ${F R}_{2}^{2}$ .

Table 1.

Description of the proposed genre-based similarity measures.

Name	Type	Equation
$F^{1}$	Absolute	$1 - \frac{\sum_{g \in G_{u} \cap G_{v}} \| f_{u, g} - f_{v, g} \|}{\| G_{u} \cap G_{v} \|}$
$F^{2}$	Square	$1 - \frac{\sum_{g \in G_{u} \cap G_{v}} (f_{u, g} - f_{v, g})^{2}}{\| G_{u} \cap G_{v} \|}$
$R_{1}^{1}$	Absolute	$1 - \frac{\sum_{g \in G_{u} \cap G_{v}} \| {\bar{r}}_{u, g}^{'} - {\bar{r}}_{v, g}^{'} \|}{\| G_{u} \cap G_{v} \|}$
$R_{1}^{2}$	Square	$1 - \frac{\sum_{g \in G_{u} \cap G_{v}} ({\bar{r}}_{u, g}^{'} - {\bar{r}}_{v, g}^{'})^{2}}{\| G_{u} \cap G_{v} \|}$
$R_{2}^{1}$	Absolute	$1 - \frac{1}{2} \frac{\sum_{g \in G_{u} \cap G_{v}} \| ({\bar{r}}_{u, g}^{'} - {\bar{r}}_{u}^{'}) - ({\bar{r}}_{v, g}^{'} - {\bar{r}}_{v}^{'}) \|}{\| G_{u} \cap G_{v} \|}$
$R_{2}^{2}$	Square	$1 - \frac{1}{4} \frac{\sum_{g \in G_{u} \cap G_{v}} {({\bar{r}}_{u, g}^{'} - {\bar{r}}_{u}^{'}) - ({\bar{r}}_{v, g}^{'} - {\bar{r}}_{v}^{'})}^{2}}{\| G_{u} \cap G_{v} \|}$
${F R}_{1}^{1}$	Absolute	$1 - \frac{\sum_{g \in G_{u} \cap G_{v}} \| f_{u, g} - f_{v, g} \| \cdot \| {\bar{r}}_{u, g}^{'} - {\bar{r}}_{v, g}^{'} \|}{\| G_{u} \cap G_{v} \|}$
${F R}_{1}^{2}$	Square	$1 - \frac{\sum_{g \in G_{u} \cap G_{v}} (f_{u, g} - f_{v, g})^{2} \cdot ({\bar{r}}_{u, g}^{'} - {\bar{r}}_{v, g}^{'})^{2}}{\| G_{u} \cap G_{v} \|}$
${F R}_{2}^{1}$	Absolute	$1 - \frac{1}{2} \frac{\sum_{g \in G_{u} \cap G_{v}} \| f_{u, g} - f_{v, g} \| \cdot \| ({\bar{r}}_{u, g}^{'} - {\bar{r}}_{u}^{'}) - ({\bar{r}}_{v, g}^{'} - {\bar{r}}_{v}^{'}) \|}{\| G_{u} \cap G_{v} \|}$
${F R}_{2}^{2}$	Square	$1 - \frac{1}{4} \frac{\sum_{g \in G_{u} \cap G_{v}} (f_{u, g} - f_{v, g})^{2} \cdot {({\bar{r}}_{u, g}^{'} - {\bar{r}}_{u}^{'}) - ({\bar{r}}_{v, g}^{'} - {\bar{r}}_{v}^{'})}^{2}}{\| G_{u} \cap G_{v} \|}$

The proposed similarity measures are computed using the relative frequency or average rating values of common genres between users. For a user pair $(u, v)$ , the computational cost is proportional to the number of shared genres, resulting in a time complexity of $O (| G_{u} \cap G_{v} |)$ , where $G_{u}$ and $G_{v}$ denote the sets of genres associated with users $u$ and $v$ , respectively. In the worst case, this becomes $O (G)$ , where $G$ is the total number of genres. By contrast, traditional user-based collaborative filtering methods typically calculate similarity over co-rated items, leading to a complexity of $O (| I_{u} \cap I_{v} |)$ , where $I_{u}$ and $I_{v}$ are the sets of items rated by the two users. Since the number of genres is generally far smaller than the number of items in real-world recommendation datasets, the proposed approach can offer substantial computational advantages. For example, the MovieLens datasets contain only 18 distinct genres despite including thousands of items.

3.2. Proposed rating prediction method

This study also proposed the rating prediction method utilizing genre information of items. This method estimates the rating of an unknown item using the average ratings of genres to which the item belongs instead of the overall mean rating of a user. The predicted rating of item $i$ in genre $g$ by user $i$ is defined as follows:

{\hat{r}}_{u, i, g} = {\bar{r}}_{u, g} + \frac{\sum_{v = 1}^{k} s i m (u, v) (r_{v, i} - {\bar{r}}_{v, g})}{\sum_{v = 1}^{k} | s i m (u, v) |}

(5)

Some items are categorized into multiple genres, so the final prediction can be determined by aggregating the predicted ratings for multiple genres. In this study, we tested two different approaches: the first one is weighted averaging using the relative frequency of genres as weights and the second is the maximum of the predicted ratings for multiple genres. Each method is defined as follows:

\begin{aligned} {\hat{r}}_{u, i}^{m e a n} & = \frac{\sum_{g \in G_{i}} f_{u, g} {\hat{r}}_{u, i, g}}{\sum_{g \in G_{i}} f_{u, g}} \end{aligned}

(6)

\begin{aligned} {\hat{r}}_{u, i}^{m a x} & = max_{g \in G_{i}} {\hat{r}}_{u, i, g} \end{aligned}

(7)

In these equations,

G_{i}

represents a set of genres to which item

i

belongs.

Figure 1 illustrates the overall framework of the proposed recommendation method. The conventional UBCF procedure is maintained, while the similarity computation stage is replaced by the proposed genre-based similarity measures and the final rating prediction stage can optionally employ genre-based aggregation.

Figure 1.

Overall framework of the proposed genre-based recommendation method.

Algorithm 1 summarizes the computational procedure of the proposed method, including preprocessing of genre statistics, similarity computation, neighborhood selection, and the two alternative rating prediction strategies.

The proposed rating prediction method estimates ratings for each genre an item belongs to and then aggregates them to compute the final rating. Consequently, the computation time increases linearly with the number of genres per item. However, since most content types—such as movies, music, books, and online marketplace products—are typically classified under a small number of categories, this increase in computation time is minimal. In fact, among the four datasets used in this study, the three datasets with expert-classified genre information have an average of fewer than two genres per item. Therefore, the additional computational burden of the proposed aggregation step is practically negligible in many real-world applications.

3.3. Hybrid similarity measures

The proposed genre-based similarity measures have the advantage that it is possible to calculate the similarity between more pairs of users compared to other similarity measures defined on the common rating items of two users. However, the proposed measures cannot fully utilize the ratings of items because they extract the features based on the genre from ratings of items. One possible way to address this issue is to combine the proposed measures in combination with other similarity measures defined on the common rating items of two users as follows:

s i m^{h y b r i d} = α \cdot s i m^{g e r n e} + (1 - α) \cdot s i m^{i t e m}

(8)

where

α

is the parameter to balance two similarities,

s i m^{g e r n e}

represents one of the proposed similarity measures that exploit the genre information of items and

s i m^{i t e m}

represents any similarity measure utilizing the rating of each item. However, when the hybrid similarity is defined as the sum of two similarity values, the influence of one of the two similarity measures on the hybrid similarity may be insignificant when the deviation of the two similarity measures is significantly different.

Figure 2 shows the distributions of similarity values obtained by some traditional similarity measures and the selected proposed measures for the MovieLens100K dataset which is described in Section 4.1. The distributions were estimated using kernel density estimation with a Gaussian kernel. As shown in Figure 2-(a), the distributions have various shapes depending on similarity measures. In Figure 2-(b), the distributions of similarity values computed by the genre-based similarity measures using the absolute difference are illustrated and most similarity values are greater than 0.9, except for $R_{1}^{1}$ . In particular, $F R_{1}^{1}$ and $F R_{2}^{1}$ have values very close to 1 for many pairs of users because the numerator of the second term includes the multiplication of the two factors between 0 and 1. In other words, the difference in the similarity values of $F R_{1}^{1}$ and $F R_{2}^{1}$ is insignificant compared to other similarity measures such as COS and MSD, which causes that the influence of $F R_{1}^{1}$ and $F R_{2}^{1}$ on the hybridized similarity is marginal. Moreover, this problem becomes more severe in the genre-based similarity measures using the squared difference than in those using the absolute difference.

Figure 2.

Distributions of similarity values for MovieLens100K. (a) Traditional similarity measures and (b) Genre-based similarity measures.

To overcome the problem that some genre-based similarity measures provide similarity values very close to 1 for most pairs of users, we suggest a rescaling method. All proposed similarity measures can be represented as $1 - X$ and the method to calculate $X$ is different from the similarity measures. The proposed rescaling method simply multiplies a constant, $s$ to $X$ as $1 - s \cdot X$ to increase the deviation in the similarity values. The key idea of the proposed rescaling method is to allow the values of the proposed similarity measure, except for the extreme values, to have a deviation similar to that of the other similarity measure.

To determine the appropriate $s$ , we used the fence of the boxplot to determine the outliers. The first step of the rescaling is to calculate the range of the inner fence of the boxplot for both $s i m^{g e r n e}$ and $s i m^{i t e m}$ . The upper and lower inner fence values ( $f_{u}$ and $f_{l}$ ) are defined as $f_{u} = Q_{3} + 1.5 I Q R$ and $f_{l} = Q_{1} - 1.5 I Q R$ where $I Q R$ represents the inter-quartile range defined as $Q_{3} - Q_{1}$ . Hence, the range of the inner fence is the same as $4 I Q R$ . Then, $s$ is computed as follows:

s = \frac{I Q R_{i t e m}}{I Q R_{g e n r e}}

(9)

That is, the range including most samples except for outliers defined in the boxplot is proportional to IQR, so

s

should be set to make the ranges of two distributions obtained from

s i m^{g e r n e}

and

s i m^{i t e m}

same.

Figure 3 shows the distributions of $s i m^{g e r n e}$ and $s i m^{i t e m}$ after rescaling. In Figure 3, the right $y$ -axis corresponds to the distribution of $F R_{1}^{1}$ . As shown in Figure 3, the deviation in $F R_{1}^{1}$ becomes comparable to the deviation in COS or PCC after rescaling.

Figure 3.

Distributions of similarity values after rescaling for MovieLens100K. (a) COS and (b) PCC.

4. Experiment

4.1. Data

For our experimental study, we used four datasets: MovieLens100K, MovieLens1M, Yahoo-Music, and Goodreads, which are the most frequently used datasets by researchers and developers in CF. Two MovieLens datasets were obtained from the webpage of GroupLens Research (URL: https://grouplens.org/datasets/movielens) and were used as they were. Yahoo-Music is provided by Yahoo Research (URL: https://webscope.sandbox.yahoo.com) and the dataset used in this study was the dataset that contains the genre information of songs (R2). In this study, only the first training dataset was used and songs without genre information were removed. Additionally, to ensure computational efficiency, we randomly selected 10,000 users from those who had rated more than 50 items. The Goodreads dataset, available on Kaggle (URL: https://www.kaggle.com/zygmunt/goodbooks-10k), contains user-generated tags for books, which we treated as genre labels. However, as these tags were inconsistently assigned and often unrelated to book content, we applied a filtering process: tags appearing in more than 2,000 books or fewer than 500 were removed to reduce noise. Despite this filtering, the genre information in Goodreads remains more ambiguous and less accurate than that of MovieLens100K, MovieLens1M, and Yahoo-Music, since the tags were manually assigned by users rather than being predefined genre categories. However, in real-world scenarios, obtaining perfectly accurate genre information is often impractical. To account for this, we focused on removing only tags that were highly inaccurate or redundant while preserving a diverse range of genre-related information.

The properties of these four datasets in terms of the number of users, number of items, number of genres, number of ratings and density are summarized in Table 2.

Table 2.
Description of datasets.

Dataset Users Items Genre Ratings Density (%)

MovieLens100K 943 1,682 18 100,000 6.3

MovieLens1M 6,040 3,706 18 1,000,209 4.1

Yahoo-Music 10,000 5,000 19 746,001 1.5

Goodreads 7,035 4,992 233 259,416 0.7

Dataset	Users	Items	Genre	Ratings	Density (%)
MovieLens100K	943	1,682	18	100,000	6.3
MovieLens1M	6,040	3,706	18	1,000,209	4.1
Yahoo-Music	10,000	5,000	19	746,001	1.5
Goodreads	7,035	4,992	233	259,416	0.7

4.2. Evaluation metrics

In this study, we mainly used three different types of evaluation metrics to compare the performance of different similarity metrics including the proposed measures. The first evaluation metric is the mean absolute error (MAE) which calculates the average error between the true ratings and predicted ratings. MAE is defined as follows:

MAE = \frac{\sum | r_{u, i} - {\hat{r}}_{u, i} |}{n}

(10)

where

n

is the number of predicted ratings in the validation set.

The second evaluation metric we used was F1, which is the harmonic mean of the precision and recall. Unlike MAE, F1 measures the performance of recommendations. To obtain F1, the precision and recall are computed first. Precision is the ratio of the number of items that are actually relevant among the items predicted relevant to the user to the total number of items that are predicted relevant to the user and recall is the ratio of the number of items that are actually relevant among the items predicted relevant to the user to the total number of items that are actually relevant to the user.

In this study, the set of items that are actually relevant to the user ( $R_{t r u e}$ ) and the set of items that are predicted relevant to users ( $R_{p r e d}$ ) are defined as follows:

\begin{aligned} R_{t r u e} & = {i | r_{u, i} > {\bar{r}}_{u}} \end{aligned}

(11)

\begin{aligned} R_{p r e d} & = {i | {\hat{r}}_{u, i} > {\bar{r}}_{u}} \end{aligned}

(12)

Then, precision and recall are defined as follows:

\begin{aligned} Precision & = \frac{| {i | i \in R_{t r u e} a n d i \in R_{p r e d}} |}{| R_{p r e d} |} \end{aligned}

(13)

\begin{aligned} Recall & = \frac{| {i | i \in R_{t r u e} a n d i \in R_{p r e d}} |}{| R_{t r u e} |} \end{aligned}

(14)

Using precision and recall, F1 is defined as follows:

F1 = \frac{2 \cdot Precision \cdot Recall}{Precision + Recall}

(15)

The last evaluation metric is the normalized discounted cumulative gain (NDCG), which is one of metrics used to evaluate the ranking quality.³⁰ NDCG is the ratio of discounted cumulative gain (DCG) of the recommended order to DCG of the ideal order. DCG and NDCG are usually used to evaluate the quality of recommendations with the top $N$ items. Let $n$ be the number of top items recommended to each user. Then, $D C G_{u} @ n$ represents DCG at $n$ for user $u$ and is computed as follows:

D C G_{u} @ n = \sum_{p = 1}^{n} \frac{{\hat{r}}_{u, p}}{\log_{2} (p + 1)}

(16)

where

{\hat{r}}_{u, p}

represents the predicted rating of an item at the top

p

I D C G_{u} @ n

, the ideal DCG at

n

is obtained using the same equation for

D C G_{u} @ n

but for the real top

n

item driven from the validation set.

D C G @ n

and

I D C G @ n

are the averages of

D C G_{u} @ n

and

I D C G_{u} @ n

for all users in the validation set, respectively. Then,

N D C G @ n

is obtained as follows:

N D C G @ n = \frac{D C G @ n}{I D C G @ n}

(17)

4.3. Experimental design

In our experiments, first, we investigated the superior similarity metric among the proposed similarity measures using two different rating prediction approaches. In addition, we also evaluated the performance of hybrid similarity measures combining the genre-based similarity measure and the other similarity measure that does not incorporate genre information varying with $α$ where $α$ is selected from ${0, 0.1, 0.2, \dots 0.9}$ . When $α = 0$ , the results correspond to the results obtained using the comparison similarity measures that do not utilize genre information to obtain similarity between users. Through these experiments, we investigated the conditions under which the proposed genre-based similarity measure achieved the best performance. Next, we compared the proposed similarity measures and hybrid similarity measures with the state-of-the-art similarity measures described in Section 2.

In addition, the number of nearest neighbors in the rating prediction, $k$ , was selected from ${10, 20, \dots, 100}$ except for the experiments to evaluate the performance of hybridization of similarity measures and calculate $N D C G @ n$ for all experiments. For $N D C G @ n$ , we set $k$ to 10. For the experiments on the hybrid similarity measures, $k$ was set to 100, which showed the best performance in terms of MAE and F1. To obtain the hybrid similarity values, we tested four simple similarity measures such as COS, PCC, MSD, and JMSD, and four recently developed similarity measures such as NHSM, RJAC, OS, and TMJ.

Similar to most researchers, for evaluation, we conducted 5-fold cross-validation by randomly choosing different training and test sets. For each fold, each dataset was divided into 80 % of the data for the training and 20 % for the testing.

All experiments were conducted using Python, and the proposed similarity and rating prediction functions can be accessed via this link (URL: http://bit.ly/4hTmg9C).

4.4. Experimental results

4.4.1. Performance of the proposed similarity measures

Figures 4 to 6 show the evaluation results for different genre-based similarity measures. In each subfigure of these figures, the left side displays the results for the genre-based similarity measures using the absolute difference while the right side shows the results for the genre-based similarity measures using the squared difference.

Figure 4.

MAE of the proposed similarity measures by the conventional rating prediction method. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

Figure 5.

F1 of the proposed similarity measures by the conventional rating prediction method. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

Figure 6.

$N D C G @ n$ of the proposed similarity measures by the conventional rating prediction method. (a) MovieLens100K, (b) MovieLens1M and (c) Yahoo-Music and (d) Goodreads.

Comparing the performance of the genre-based similarity measures using the absolute difference with the genre-based similarity measures using the squared difference, the absolute difference provides a better performance than the squared difference for MAE and F1. For MovieLens100K, MovieLens1M, and Goodreads, the similarity measures using both frequency and rating information generally show better performance in terms of MAE and F1. However, in the case of Yahoo-Music, the best performance was observed for $R_{2}$ , which only utilizes the mean-centered average ratings of genres. In addition, using the mean-centered average ratings of genres achieved better performance in MAE and F1 than using the average ratings of genres. In other words, considering user rating behaviors (considering ${\bar{r}}_{u}$ ) in the similarity calculation can enhance the accuracy of rating prediction and the quality of recommendations.

In terms of $N D C G @ n$ , the absolute difference slightly outperforms the squared difference for all the datasets and using the mean-centered average ratings of genres usually achieved better performance than using the average ratings of genres, similar to the results using MAE and F1. However, the similarity measures using only rating information without frequency information shows enhanced performance compared with the other similarity measures for $N D C G @ n$ , except for Yahoo-Music. In the case of Yahoo-Music, $F R_{2}$ shows higher values of $N D C G @ n$ than $R_{2}$ , except for the values of high $n$ ( $n \geq 90$ ).

Next, we compared the different rating prediction methods. Figures 7 to 9 show the evaluation results for the selected genre-based similarity measures ( $F$ , $R_{2}$ , and $F R_{2}$ ) depending on the rating prediction method. In each subfigure, the left side corresponds to the results using the similarity measures with the absolute different, whereas the right side corresponds to the results using the similarity measures with the squared difference.

Figure 7.

MAE of the proposed similarity measures depending on the prediction methods. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

Figure 8.

F1 of the proposed similarity measures depending on the prediction methods. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

Figure 9.

$N D C G @ n$ of the proposed similarity measures depending on the prediction methods. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

${\hat{r}}_{u, i}^{m e a n}$ shows a smaller MAE value than ${\hat{r}}_{u, i}$ regardless of the similarity measures, whereas ${\hat{r}}_{u, i}^{m a x}$ increases MAE values compared with ${\hat{r}}_{u, i}$ . These results are natural because, in neighborhood-based collaborative filtering, predicted ratings are typically obtained as weighted combinations of neighbors’ ratings and are not inherently constrained to remain within the original rating scale. Consequently, predicted values may exceed the valid rating range unless an additional calibration or clipping step is applied. This tendency can be more pronounced for ${\hat{r}}_{u, i}^{m a x}$ , since it selects the largest value among the predicted ratings for each genre and therefore has a higher probability of choosing an extreme or overestimated value.

Meanwhile, ${\hat{r}}_{u, i}^{m a x}$ achieves higher F1 values than ${\hat{r}}_{u, i}$ and ${\hat{r}}_{u, i}^{m e a n}$ . Although accurate rating estimation is important for obtaining a low MAE, maximizing F1 does not necessarily require numerically precise rating values. Instead, items that a user truly prefers should be assigned relatively higher predicted scores than items the user is less likely to prefer. When separate ratings are predicted for each genre of an item, the score corresponding to the user’s most preferred genre is likely to be relatively high, and this preference signal is directly reflected by ${\hat{r}}_{u, i}^{m a x}$ . As a result, relevant items are more likely to be ranked above non-relevant items, which can improve F1 performance.

A similar tendency is observed for $N D C G @ n$ , where ranking quality is more important than the absolute accuracy of predicted scores. By promoting items with stronger genre-specific preference signals to higher positions, ${\hat{r}}_{u, i}^{m a x}$ can produce superior ranking performance compared with the other prediction methods.

Therefore, the higher MAE of ${\hat{r}}_{u, i}^{m a x}$ should not be interpreted solely as weaker recommendation capability. Rather, it partly indicates that the predicted scores may require better calibration to align with the original rating scale. If an appropriate post-processing strategy such as clipping, normalization, or scale calibration is applied, ${\hat{r}}_{u, i}^{m a x}$ may potentially reduce MAE while preserving its advantages in F1 and $N D C G @ n$ .

In addition, ${\hat{r}}_{u, i}^{m e a n}$ increased F1 compared with ${\hat{r}}_{u, i}$ , so it can be concluded that ${\hat{r}}_{u, i}^{m e a n}$ is better than ${\hat{r}}_{u, i}$ , considering both MAE and F1. Therefore, in summary, while ${\hat{r}}_{u, i}^{m e a n}$ requires more computation time than ${\hat{r}}_{u, i}$ , it provides more accurate rating predictions and better recommendation performance. However, when the average number of genres per item is less than two, as observed in datasets such as MovieLens100K, MovieLens1M, and Yahoo-Music, the number of genre-specific ratings that need to be predicted remains below two on average. In such cases, the computational cost of ${\hat{r}}_{u, i}^{m e a n}$ is nearly identical to that of ${\hat{r}}_{u, i}$ , making ${\hat{r}}_{u, i}^{m e a n}$ the preferable choice.

However, determining whether ${\hat{r}}_{u, i}^{m e a n}$ or ${\hat{r}}_{u, i}^{m a x}$ is superior depends on the specific application objective. Since both methods have similar computational costs, their comparative usefulness is primarily determined by predictive performance under the target evaluation criterion. If numerically accurate rating estimation is the main goal, ${\hat{r}}_{u, i}^{m e a n}$ may be preferred because it generally provides more stable and better-calibrated predictions, which can lead to lower MAE values. Accurate rating prediction is also valuable because it often serves as a foundation for generating effective recommendation lists.

Nevertheless, in practical recommendation systems, users are typically exposed to ranked recommendation lists rather than the predicted rating values themselves. Therefore, when MAE and ranking-based metrics such as F1 or $N D C G @ n$ exhibit conflicting tendencies, greater emphasis may reasonably be placed on recommendation accuracy, since it more directly reflects user-facing performance. From this perspective, ${\hat{r}}_{u, i}^{m a x}$ can be more advantageous because it tends to assign relatively higher scores to items associated with a user’s most preferred genres, thereby improving the likelihood that relevant items appear in the recommendation list and are ranked near the top.

At the same time, if genre information is inaccurate or incomplete, ${\hat{r}}_{u, i}^{m a x}$ may be more sensitive to such errors because it selects the largest genre-specific predicted value, which can amplify noise from incorrect labels. Therefore, ${\hat{r}}_{u, i}^{m a x}$ is particularly suitable when genre metadata are reliable, curated, or carefully preprocessed to ensure sufficient quality.

4.4.2. Performance of the hybrid similarity measures

As observed in the results described in Section 4.4.1, one of $R_{2}$ and $F R_{2}$ achieved the best performance among the proposed similarity measures regardless of the datasets, and using the absolute difference was generally superior to using the squared difference. Therefore, we provided the results of the hybridization between the proposed similarity measure and the other similarity measure using $R_{2}^{1}$ and $F R_{2}^{1}$ . Moreover, in recommendation systems, the ultimate objective is to identify items that users are likely to select and present them in an effective ranked list. Although MAE remains an important metric for evaluating the numerical accuracy of predicted ratings, F1 and $N D C G @ n$ provide a more direct assessment of user-facing recommendation quality by measuring recommendation relevance and ranking effectiveness. For this reason, to ensure clarity and avoid overwhelming results that obscure key insights, we present only the cases where the traditional method is used and where the proposed approach employs ${\hat{r}}_{u, i}$ and ${\hat{r}}_{u, i}^{m a x}$ , which demonstrate superior recommendation performance over ${\hat{r}}_{u, i}^{m e a n}$ in terms of F1 and $N D C G @ n$ . This allows us to more effectively highlight the advantages of the proposed method.

Figures 10 to 12 provide the performance of the hybrid similarity measures using MAE, F1 and $N D C G @ 100$ varying with $α$ when the original rating prediction method was used. For MAE and F1, the results when the number of neighbors was 100 are only shown in Figures 10 and 11. For each subfigure of these figures, the left side represents $R_{2}^{1}$ and the right side represents $F R_{2}^{1}$ . The different lines of the subfigures were obtained using the different hybrid similarity measures obtained from the ordinary similarity measures that did not use genre information. In addition, the circle on each line represents the best case for each hybrid similarity measure. When $α$ is 0, the hybrid similarity measure corresponds to the ordinary similarity measure, whereas it is the same as the proposed similarity measure when $α$ is set to 1.

Figure 10.

MAE of the hybrid similarity measures: Orig. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music, (d) Goodreads.

Figure 11.

F1 of the hybrid similarity measures: Orig. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

Figure 12.

$N D C G @ 100$ of the hybrid similarity measures: Orig. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

In general, the proposed similarity measures outperform the most traditional similarity measures such as COS, PCC, and MSD, and OS, regardless of the evaluation metrics and the datasets. On the other hand, JMSD, NHSM, RJAC, and TMJ achieved lower MAE and higher F1 than the proposed measures, and JMSD and NHSM provided higher $N D C G @ 100$ than the proposed measures. In other words, the similarity measures that have recently been developed are superior to the similarity measures that use only the genre information of items. However, the best performance was mostly observed using the hybrid similarity measures.

In addition, Figures 13 and 14 show the results for the hybrid similarity measures by ${\hat{r}}_{u, i}^{m a x}$ in terms of F1 and $N D C G @ 100$ . As shown in these figures, when ${\hat{r}}_{u, i}^{m a x}$ was used, the proposed similarity measures showed better results than many comparison methods, compared to the case when the original rating prediction method was used. Especially, in terms of F1, the difference between the rating methods is larger than that of $N D C G @ 100$ .

Figure 13.

F1 of the hybrid similarity measures: Max. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

Figure 14.

$N D C G @ 100$ of the hybrid similarity measures: Max. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads.

Even when ${\hat{r}}_{u, i}^{m a x}$ was used to estimate unknown ratings, the proposed similarity measures did not outperform all comparison similarity measures. In most cases, they achieved lower F1 and $N D C G @ 100$ values than NHSM. However, the hybrid approach combining the proposed similarity measures with NHSM performed better than NHSM alone, except for the $N D C G @ 100$ evaluation results on Goodreads.

Table 3 summarizes the performance evaluation of the hybrid similarity measures by ${\hat{r}}_{u, i}^{m a x}$ when $α = 0.5$ . In this table, the column, “# better”, provides the number of comparison similarity measures that the hybrid similarity measures outperform. According to Table 3, except for Goodreads, the hybrid approach combining the proposed similarity with other similarities showed statistically significant improvements in F1 and $N D C G @ 100$ compared to using other similarities individually. On Goodreads, although the F1 values were better, there were more cases where the performance in NDCG@100 decreased. The main difference between Goodreads and other datasets is that, while genre information in the other datasets was determined by experts, on Goodreads, users themselves provided tags that were used as genre information. As a result, the accuracy of genre information on Goodreads is lower compared to other datasets. In such cases, when ${\hat{r}}_{u, i}^{m a x}$ is used, the accuracy of rating predictions may be lower. Moreover, if the maximum rating is selected based on incorrectly labeled tags, the performance of recommendations can also be diminished. The decrease in performance observed in NDCG@100 on Goodreads is likely due to mispredicted items being included in the list of the top 100 recommended items. Therefore, when using ${\hat{r}}_{u, i}^{m a x}$ for rating predictions, it is essential to ensure the reliability of the genre information beforehand.

Table 3.

Summary of the performance evaluation for the hybrid similarity measures by ${\hat{r}}_{u, i}^{m a x}$ at $α = 0.5$ .

Data	Proposed sim.	Metric	# better
Movielens100K	$R_{2}^{1}$	F1	8
		$N D C G @ 100$	8
	$F R_{2}^{1}$	F1	8
		$N D C G @ 100$	8
Movielens1M	$R_{2}^{1}$	F1	8
		$N D C G @ 100$	8
	$F R_{2}^{1}$	F1	8
		$N D C G @ 100$	8
Yahoo-Music	$R_{2}^{1}$	F1	7
		$N D C G @ 100$	8
	$F R_{2}^{1}$	F1	8
		$N D C G @ 100$	8
Goodreads	$R_{2}^{1}$	F1	8
		$N D C G @ 100$	3
	$F R_{2}^{1}$	F1	8
		$N D C G @ 100$	2

Moreover, in terms of computational cost, the proposed similarity measures require significantly less computation time than the comparison methods. Figure 15 illustrates the time required for similarity computation, showing that the proposed measures are considerably more efficient. In particular, compared to NHSM—the most computationally expensive method—the proposed measures require only a marginal amount of time. Unlike traditional similarity measures, which rely on individual item ratings, the proposed approach utilizes rating frequencies and average ratings by genre. Since the number of genres is typically much smaller than the number of items, this results in faster similarity computation. Consequently, the proposed similarity measures are not only computationally efficient but also suitable for integration with other similarity measures to enhance recommendation performance.

Figure 15.

Computation times for the similarity measures. (a) MovieLens100K, (b) MovieLens1M, (c) Yahoo-Music and (d) Goodreads

5. Conclusion

In this study, new genre-based similarity measures are suggested by utilizing item genre information. The proposed similarity measures exploit either the number of rated items per genre or the average rating of genres to obtain similarity between users. The number of genres is typically much smaller than the number of items, so the proposed similarity measures reliably calculate the similarity between users with a few co-rated items.

Among the proposed similarity measures, the methods based on both the relative rating frequency of genres and the average rating of genres generally outperformed the other variants. In particular, the mean-centered versions, namely $R_{2}$ and $F R_{2}$ , usually achieved better performance than their non-centered counterparts $R_{1}$ and $F R_{1}$ . In these methods, each user’s average rating is subtracted from the genre-specific average rating before similarity is computed. This adjustment appears to improve recommendation performance because users often exhibit different rating tendencies: some users consistently assign higher ratings, whereas others use a more conservative rating scale. Without accounting for such user-specific rating bias, two users with similar relative genre preferences may appear dissimilar simply due to differences in their overall rating levels. By removing the individual mean effect, $R_{2}$ and $F R_{2}$ can more accurately capture relative genre preferences and thereby produce more reliable user similarity estimates. In addition, using the absolute difference was generally more effective than using the squared difference.

In terms of the rating prediction approach, estimating the rating of an unrated item as the maximum value among the predicted ratings for each genre achieved higher F1 and $N D C G @ n$ than the other rating methods, although it also increased MAE. This result suggests a trade-off between numerical rating accuracy and recommendation effectiveness. MAE remains an important metric because accurate and well-calibrated rating prediction can support better recommendation decisions. However, users in practical recommendation systems are typically exposed to ranked recommendation lists rather than the predicted rating values themselves. Therefore, F1 and $N D C G @ n$ , which directly evaluate recommendation relevance and ranking quality, provide a more user-oriented assessment of recommendation performance. From this perspective, the prediction method based on ${\hat{r}}_{u, i}^{m a x}$ can be considered an effective approach for top- $N$ recommendation. At the same time, its relatively higher MAE indicates that additional calibration strategies may further improve numerical prediction accuracy while preserving its ranking advantages.

Furthermore, we suggested the hybridization of two different similarity measures: one uses item genre information and the other does not use item genre information. A scaling method to match scales of different similarity measures was proposed to properly calculate the similarity using the hybrid similarity measure. Through this method, it is possible to obtain reliable hybrid similarity measures and a comprehensive study revealed that combining the similarity measures can enhance the recommendation performance. Considering the short computation time in the similarity calculation by the proposed measures, it can be seen that the benefits of the hybridization are greater than the effort involved in additional computational cost to obtain similarities by the proposed similarity measures.

Future studies may extend the proposed framework in several directions. First, the hybridization of different similarity measures could be further improved by adaptively adjusting their weights according to factors such as the number of co-rated items. Second, the current scaling method relies mainly on the interquartile range of similarity distributions, and more robust scaling strategies incorporating additional distributional statistics may yield better performance. Third, the proposed method assumes that item genre metadata are sufficiently reliable. Although this assumption is often reasonable in domains where metadata are curated by providers, user-generated tagging environments may contain inaccurate or incomplete labels. Future research could therefore investigate confidence-aware genre weighting schemes or robust similarity measures that explicitly account for metadata uncertainty.

Another limitation of this study is that the proposed model does not explicitly consider temporal changes in user preferences. In practice, users’ interests and rating behaviors may evolve over time, causing their preferences for specific genres to shift. Since the primary focus of this study was to examine how genre information can enhance conventional similarity measures, the proposed method was developed under a static-preference assumption. Future research could extend the model by incorporating temporal dynamics, such as time-decay weighting or recency-sensitive similarity adjustment, to improve recommendation accuracy and adaptability in dynamic environments.

Footnotes

Acknowledgement

This study was supported by the Research Program funded by the SeoulTech(Seoul National University of Science and Technology).

ORCID iD

Kyoungok Kim

Funding

This study was supported by the Research Program funded by the SeoulTech (Seoul National University of Science and Technology).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Bobadilla

Ortega

Hernando

, et al. Recommender systems survey. Knowl Based Syst 2013; 46: 109–132.

Da’u

Salim

. Recommendation system based on deep learning methods: a systematic review and new directions. Artif Intell Rev 2020; 53: 2709–2748.

Kasula

Yenugula

Konda

, et al. Federated learning with secure aggregation for privacy-preserving deep learning in iot environments. In: 2025 IEEE conference on computer applications (ICCA), 2025, pp.1–7. IEEE.

Khan

Mansha

Khan

, et al. Collaborative filtering based online recommendation systems: A survey. In: 2017 International conference on information and communication technologies (ICICT), 2017, pp.125–130. DOI: 10.1109/ICICT.2017.8320176.

Jeong

Lee

Cho

. Improving memory-based collaborative filtering via similarity updating and prediction modulation. Inf Sci (Ny) 2010; 180: 602–612.

Sun

Kong

. A comparison of several algorithms for collaborative filtering in startup stage. In: Proceedings. 2005 IEEE Networking, Sensing and Control, 2005, 2005, pp.25–28. DOI: 10.1109/ICNSC.2005.1461154.

Birtolo

Ronca

Armenise

, et al. Personalized suggestions by means of Collaborative Filtering: A comparison of two different model-based techniques. In: 2011 Third world congress on nature and biologically inspired computing, 2011, pp.444–450. DOI: 10.1109/NaBIC.2011.6089628.

Zhong

. Unified collaborative filtering model based on combination of latent features. Expert Syst Appl 2010; 37: 5666–5672.

Vadakkethil

Polimetla

Alsalami

, et al. Mayfly optimization algorithm with bidirectional long-short term memory for intrusion detection system in internet of things. In: 2024 Third international conference on distributed computing and electrical circuits and electronics (ICDCECE), 2024, pp.1–4. IEEE.

10.

Bobadilla

Serradilla

Bernal

. A new collaborative filtering metric that improves the behavior of recommender systems. Knowl Based Syst 2010; 23: 520–528.

11.

Kim

El-Saddik

. Collaborative error-reflected models for cold-start recommender systems. Decis Support Syst 2011; 51: 519–531.

12.

Sun

Zhang

Dong

, et al. Integrating triangle and jaccard similarities for recommendation. PLoS ONE 2017; 12: e0183570.

13.

Ahn

. A new similarity measure for collaborative filtering to alleviate the new user cold-starting problem. Inf Sci (Ny) 2008; 178: 37–51.

14.

Lam

, et al. Addressing cold-start problem in recommendation systems. In: Proceedings of the 2nd international conference on Ubiquitous information management and communication, 2008, pp.208–211.

15.

Adomavicius

Tuzhilin

. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 2005; 17: 734–749.

16.

Ghazanfar

Prugel-Bennett

. A scalable, accurate hybrid recommender system. In: 2010 Third international conference on knowledge discovery and data mining, 2010, pp.94–98. DOI: 10.1109/WKDD.2010.117.

17.

Aamir

Bhusry

. Recommendation system: state of the art approach. Int J Comput Appl 2015; 120(12): 25–32.

18.

Khoshgoftaar

. A survey of collaborative filtering techniques. Adv Artif Intell 2009; 2009: 421425.

19.

Resnick

Iacovou

Suchak

, et al. GroupLens: An open architecture for collaborative filtering of netnews. In: Proceedings of the 1994 ACM conference on computer supported cooperative work, 1994, pp.175–186. CSCW ’94. New York, NY, USA: Association for Computing Machinery. DOI: 10.1145/192844.192905.

20.

Shardanand

Maes

. Social information filtering: Algorithms for automating “Word of Mouth”. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1995, pp.210–217. CHI ’95. USA: ACM Press/Addison-Wesley Publishing Co. DOI: 10.1145/223904.223931.

21.

Herlocker

Konstan

Borchers

, et al. An algorithmic framework for performing collaborative filtering. In: Proceedings of the 22nd annual international ACM sigir conference on research and development in information retrieval, 1999, pp.230–237. SIGIR ’99. New York, NY, USA: Association for Computing Machinery. DOI: 10.1145/312624.312682.

22.

Jamali

Ester

. TrustWalker: A random walk model for combining trust-based and item-based recommendation. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining, 2009, pp.397–406. KDD ’09. New York, NY, USA: Association for Computing Machinery. DOI: 10.1145/1557019.1557067.

23.

Koutrika

Bercovitz

Garcia-Molina

. FlexRecs: Expressing and Combining Flexible Recommendations. In: Proceedings of the 2009 ACM SIGMOD international conference on management of data, 2009, pp.745–758. SIGMOD ’09. New York, NY, USA: Association for Computing Machinery. DOI: 10.1145/1559845.1559923.

24.

Liu

Mian

, et al. A new user similarity model to improve the accuracy of collaborative filtering. Knowl Based Syst 2014; 56: 156–166.

25.

Bag

Kumar

Tiwari

. An efficient recommendation generation using relevant jaccard similarity. Inf Sci (Ny) 2019; 483: 53–64.

26.

Gazdar

Hidri

. A new similarity measure for collaborative filtering based recommender systems. Knowl Based Syst 2020; 188: 105058.

27.

Zhang

Song

. A collaborative filtering recommendation algorithm based on item genre and rating similarity. In: 2009 International conference on computational intelligence and natural computing, 2009, Vol. 2, pp.72–75. DOI: 10.1109/CINC.2009.219.

28.

Barman

Hasan

Roy

. A genre-based item-item collaborative filtering: Facing the cold-start problem. In: Proceedings of the 2019 8th international conference on software and computer applications, 2019, pp.258–262. ICSCA ’19. New York, NY, USA: Association for Computing Machinery.

29.

Lin

Zhang

Jiang

, et al. Improvement of similarity coefficients based on item rating and item genre. In: 2019 International conference on wavelet analysis and pattern recognition (ICWAPR), 2019, pp.1–6. DOI: 10.1109/ICWAPR48189.2019.8946453.

30.

Järvelin

Kekäläinen

. Cumulated gain-based evaluation of IR techniques. ACM Trans Informat Syst (TOIS) 2002; 20: 422–446.

A new user similarity model using item genre information for collaborative filtering

Abstract

Keywords

1. Introduction

2.1. Collaborative filtering problem

2.3. Genre-based similarity measures

3. Proposed genre-based similarity measures

3.1. Proposed similarity measures

4.1. Data

Table 2. Description of datasets. Dataset Users Items Genre Ratings Density (%) MovieLens100K 943 1,682 18 100,000 6.3 MovieLens1M 6,040 3,706 18 1,000,209 4.1 Yahoo-Music 10,000 5,000 19 746,001 1.5 Goodreads 7,035 4,992 233 259,416 0.7

4.4. Experimental results

4.4.1. Performance of the proposed similarity measures

Footnotes

Acknowledgement

ORCID iD

Funding

Declaration of conflicting interests

References

Table 2.
Description of datasets.

Dataset Users Items Genre Ratings Density (%)

MovieLens100K 943 1,682 18 100,000 6.3

MovieLens1M 6,040 3,706 18 1,000,209 4.1

Yahoo-Music 10,000 5,000 19 746,001 1.5

Goodreads 7,035 4,992 233 259,416 0.7