Hybrid Recommendation System

  • Recommendation System is a type of algorithm that uses consumers’ tastes, purchase history, reviews, and ratings to determine how much they will want certain products/services in the future. It is actively used by major companies like Google, Amazon.. etc, and active research is being conducted in academia, too. Some of the basic but widely used recommender includes Content-based Filtering (CBF) or Collaborative Filtering (CF). However, this post will focus on Hybrid method, so let’s briefly explain the two methods via commonly used example - movie recommendation.
  • Content-based Filtering(CBF): Every movie has its own ‘genre’. For example, the genre of Harry Potter is fantasy and that of Fast and Furious is action. Of course, not all movies have only one genre - some movies can have elements for SF and drama genre simultaneously. Therefore, we can divide the target movies based on what genre each movie has. The algorithm that uses distances(which are based on the characteristics of item’s ‘content’ - e.g. genre) between products is called CBF. CBF has the advantage of being relatively easy to implement and not necessarily requiring information about users, but it has the disadvantage of requiring domain knowledge and presenting only safe recommendation.
  • Collaborative Filtering(CF): Unlike CBF, CF is based on user behavior. Going back to the example of a movie, CF can be implemented based on the rating that users have made for movies(items). Based on the rating given by three users for five films, Batman, Avengers, Shrek, Conjuring, and Mission Impossible, it can also be seen as an algorithm to see what rating each user will rate for the films they have not yet seen. There are several techniques used in CF. In high dimensions, factorization techniques such as Matrix Factorization and Factorization Machine are useful, and other methods using Cosine similarity or (Pearson) Correlation coefficient can be solutions for the curse of dimensionality. Unlike CBF, CF can have a good starting point in that it does not require domain knowledge and is based on user behavior, but it has the disadvantage of having a Cold start (not easy to judge due to lack of information for new users) or Sparsity problems.

Now, let’s get to know about the Hybrid Recommendation System. Hybrid method, as the name suggests, is a mixture of methodologies such as CBF and CF. Research questions that are primarily addressed in this area are as follows:

  • Cold start: As previously mentioned in the above CF description, the issue of how to make recommendation for new users/new products is still being discussed. Various solutions have been introduced, for example, using a probabilistic model to extract latent features of an item, and then use these latent features to rate the corresponding item even if it does not have much history. Another method is to use CBF only for those who have a cold start problem when using CF.
  • Data Sparsity: This problem usually arises from the fact that users only respond to limited items. This results in Sparse user-item interaction matrix, which has the disadvantage of making it difficult to distinguish similar items/users. Several solutions have also been discussed, for example, the problem can be either mitigated by applying factorization to user-item-domain(triadic) relationship or by using user-item rating as a precursor to other missing values, referring to the rating of a target item for similar users.
  • Scalability: Scalability issues are in fact actively discussed in a variety of other fields as well as recommendation systems. In the case of recommendation systems, various solutions are suggested, such as making the interaction term workable through the Factorization Machine, or using similarity to select Nearest & Fastest Neighbors. In the process, models such as Naive Bayes and SVM are being combined with CF, and further research on Scalability is expected to be conducted in the future.

So how does the Hybrid method combine methodologies such as CF and CBF to solve the above research questions? There may be many ways to combine, but here I will introduce five commonly used techniques.

  • Weighted: Weighted hybrids are the most commonly used method, aggregating output scores of various recommendation techniques into weighted linear functions to make final recommendations. The simplest example is the recommendation based on a weighted linear combination of CF and CBF rating scores. The weights are determined on per-user basis for the optimal mix and then combined later. In addition, papers on weighted hybrids such as techniques created by weighted combinations of user-user, user-tag, and user-item relations are frequently published.
  • Feature combination: This is a method of putting the output of one recomender into the original feature data and then use other recomenders in this extended data to produce final results. Looking at the simple CF-CBF hybrid, we use CF as the final model, but combine the output of CBF to the original data in advance - before running CF. These combination methods have the advantage of being able to reduce sensitivity to the possible sparsity of the initial data. There are other papers except CF-CBF hybrids, such as performing recommendations by increasing data using CF on the first stage, clustering on the second one, and finally, CBF.
  • Cascade: Cascade is a kind of staged recommendation process. The first recommended technique is used to specify coarse ranking (goal: sort items) for the target items, and the second technique is used to refine the candidate set produced in the previous step. Cascade is order-sensitive as the stages are divided. This implies that the results of CF-CBF and CBF-CF can be very different. As an example of Cascade hybrid, the first classification is conducted through SVMs, and the second analysis of user personality (estimating the probability for a user to have the same personality as other users based on the rating given by users to items) is used to recommend the final music. Of course, there can be another combining method: you can use CF in both stages, but in the first stage, neighbors(based on similarity) will be found, and in the second, Bayesian estimator will be used for recommendation.
  • Switching: Switching is a method using different recommendation techniques depending on whether the certain criteria that we set in advance has met or not. For example, the CF-CBF method is a method of using CBF when it is not possible to give sufficient recommendations through CF methods, such as the emergence of new users while using CF. A number of techniques can also be used with CF or CBF, so strategies such as CF1-CF2 can also be used.
  • Feature augmentation: Feature augmentation: This is a hybrid method that seems to be a mixture of Cascade and Feature combination, which first proceeds with the item preference/classification and reflects this during the second recomender technique. For example, there may be a way to use multidimensional clustering in the first stage and CF in the second. First of all, the user/item data is divided through clustering, and clusters with similar features are erased while the remaining clusters are placed as CF modules. The item-item similarity is then calculated, resulting in the final result of CF based on this and the rating deviations for neighbors. It may be somewhat confusing with Cascade or Feature combination, but personally, I think it differs slightly from these methods in that the results are reflected later as a kind of feature and that the classification/prediction is refined and reflected later rather than extending the original data itself.

So far, we have explored issues and several methodologies for Hybrid RecSys. Now, for a more detailed example, let’s review a paper using Hybrid Method and wrap it up.

  • The paper is intended to adequately hybrid CF and CBF to capture both Scalability and Accuracy. CF and CBF were used for each step, and when computing the last score, is seems to have combined weights for the methodology differently. To summarize the process as follows.

    • Step 1: Data filtering -> Similarity indicators such as Pearson Correlation coefficient are utilized to obtain and filter the Nearest & Farthest neighbors of Users (CF). This makes it easier to analyze large scale data. (Scalability)

    • Step 2: DTB learning(CBF) -> Obtain a decision boundary for the items purchased by the previously selected target users. First of all, if target users bought the item, 1 will be allocated and if fastest neighbors bought it, 0 will be assigned. In this phase, we use classification techniques such as LDA, QDA, SVM, etc.. to conduct binary classification between target groups and fastest neighbors groups. In other words, the goal is to calculate the boundaries between the two classes.

    • Step 3: HYRED prediction -> Step 1 calculated the correlation between users and Step 2 showed the distance between the item and the boundaries. In Step 3, these CF and DTB scores are properly combined after normalization, allowing us to predict user i’s expected rating for an item j based on the ratings recorded by user i’s KNNs, etc. Each CF/DTB score and combination formula presented in the paper are shown below.

      (S_Ti: Similarity btw user T and user i & f: Distance btw decision boundary and item j) \(S^{\mathrm{CF}}\left(I_{j}\right) =\frac{\sum_{i \in P_{N}} S_{T i} y_{j i}}{\sum_{j} \sum_{i \in P_{N}} S_{T i} y_{j i}}, \quad S^{\mathrm{DTB}}\left(I_{j}\right) =\frac{f\left(I_{j}\right)}{\sqrt{\sum_{j} f\left(I_{j}\right)^{2}}}\\ C\left(I_{j}\right)=S^{\mathrm{CF}}\left(I_{j}\right)+\lambda^{*} S^{\mathrm{DTB}}\left(I_{j}\right) \quad \forall_{j} \in P_{N}\)

    In this paper, the author argued that depending on how we give weight, we can properly combine the advantages of the two methodologies to produce improved performance. Also, thankfully, it allowed us to learn more specifically about how the Hybrid method can be implemented. I think that implementing various Hybrid models such as dividing steps according to the nature of the given data and setting the methodology suitable for each step will not only achieve good results but also broaden the insight into the Recommendation System.

  • References :

    • Çano, E., & Morisio, M. (2017). Hybrid recommender systems: A systematic literature review. Intelligent Data Analysis, 21(6), 1487-1524.
    • Choi, S. H., Jeong, Y. S., & Jeong, M. K. (2010). A hybrid recommendation method with reduced data for large-scale application. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(5), 557-566._