Improving the quality of recommender system algorithm using associative analysis methods

FB Consult specializes in the development, implementation, and support of full-featured CRM solutions for banks, insurance, commercial and industrial, pharmaceutical companies. A customer relationship management system (CRM-system) is an information system designed to collect and process customer data. The data obtained from this system can be used in a recommendation system, helping managers to determine the needs of customers more accurately. Understanding the diverse insurance needs of the population and comparing them with related products offered by insurance companies makes insurance more effective and makes insurance companies more successful. Earlier, FB Consult developed an analytical platform that includes services for recommendations and time series analysis. The objective of the study is to test the impact of the affinity analysis algorithm for the F2-score metric-evaluation of the recommendation algorithm based on collaborative filtering and cluster analysis of data. The article describes the developed algorithm, which consists of 2 stages. At the training stage, which takes a long time, but is carried out only when there is a significant change in customer data, a recommendation model is created. First of all, customers are divided into clusters based on metadata using the EM algorithm, and a list of the most popular products is generated for each cluster. This is necessary to solve the cold start problem. In addition, customers are divided into clusters according to shopping lists in order to further speed up the collaborative filtering algorithm, since customers from another cluster will not be close to the customer for whom the recommendation is calculated, and the association rules are calculated using the Apriori algorithm. As a result, the model consists of a list of the most popular products for each cluster, a customer classifier by metadata, a customer classifier by shopping lists, customer lists divided into clusters by shopping and a list of found association rules. The recommendation phase is for each customer and therefore must be fast. If the customer does not have purchased products yet, then he is classified by his metadata and receives a recommendation from the list of popular products for his cluster. Otherwise, the customer is classified according to the shoppinglist, then, using collaborative filtering, the closest customers are found among the customers of his cluster and recommendations are formed on the basis of their purchases. In addition, if a customer has a cause for a previously found association rule in the purchased products, he is recommended its effect along with recommendations based on purchases of similar customers. Testing and analysis of the effectiveness of the developed algorithm was carried out on the data of insurance company. The data includes 30 thousand customers and 21 types of products from 2010 to 2020. As a result of testing, it was revealed that the proportion of correctly found products for recommendation among the products that needed to be recommended increased, but also the proportion of recommended products that were clearly not necessary for recommendations (were not removed from the customer during testing) increased. Should take into account that these could be products that should be recommended to customers, but that they have not purchased yet. In this article, a study was carried out of the impact of affinity analysis on the recommendation algorithm. The main result of this work is to improve the F2-score metric in comparison with the basic implementation of the recommendation algorithm. With the help of affinity analysis, you can generate not only positive, but also negative association rules. In future work, it is planned to investigate the use of such rules in order to reduce the likelihood of recommending products that are contained in the effect of these rules, thereby increasing the accuracy of the system.