Classification of the User's Intent Detection in E-commerce systems – Survey and Recommendations

Published Online December 2020 in MECS DOI: 10.5815/ijieeb.2020.06.01

Nowadays, customers are more and more demanding. In e-commerce, browsing products in a self-service manner is not enough, and customers expect a personalized experience. According to the Forrester report [48], personalized experience directly impacts the e-commerce business's effectiveness. Measuring customer experience is usually done by explicit feedback like internet pools or feedback forms. However, this explicit feedback has a low response rate and is not commonly used in e-commerce stores [64]. Measuring user experience from implicit feedback still needs research to be successfully applied in production systems. Even though customer experience is not easily measurable [21, 59], its impact is visible on business goals like customer satisfaction, loyalty, and trust. Challenging business goals: increase sales, customer lifetime value, lower marketing costs still lack research. Despite that, the customer experience becomes a battleground for competition [26, 64].

A natural way to improve the experience is to understand the customer. A method to understand the customer is intent detection. In this article, we classify methods that are used to recognize user intent in e-commerce. In our work, intent detection is the process of estimating the customer's intentions, inferring them through observed actions or from the context of the environment [49]. The result of intent detection is creating a customer profile model. This model can contain scores for short and long interests, propensities, personal preferences, and similar. We discuss customer profile model details in section 3.

In this article, we review the literature for the problem of estimating customer intent for e-commerce. We selected articles from 2015 until 2019 to overview the most recent research in the field. We found that there is no proper classification of the intent detection methods nor a comparison of available approaches. To answer that, we propose classification and discuss the limitations of the existing solutions. We analyzed the intent detection methods from the perspective of the following qualities:

• -    Personalisation. Modern methods can find intent for specific individuals, in contrast to older approaches, which focus on groups or segments. Also, finding intent for anonymous users where the customer profile usually does not exist and responding correctly to their short-term needs is one of the fundamental problems in e-commerce [75].

• -    Adaptation to the dynamic changes. It becomes crucial to adapt quickly to context change and user intent drift. User intent can change between different web site interaction sessions [20]. Adaptation to context changes is one of the primary goals in personalized, session-aware recommendations.

• -    Context information: Intent detection methods should leverage as broad context information as possible to increase prediction accuracy. We discuss context information types and their impact on accuracy in section 5.

• -    Scalability and performance. The ability to scale out to many customers is a crucial problem in the continuously growing e-commerce market.

The article is divided into four parts, each focusing on different aspects of intent detection:

• -    Section 2: types of intent detection: we discuss different solutions and their limitations for customer intent detection.

• -    Section 3: Personalization context and customer profile: We describe personalization methods on a single customer level and calculate customer profile.

• -   Section 4: Dynamic changes in user interests: We analyze different modeling dynamics in customer interests.

• -   Section 5: Attributes for customer intent modeling: we analyze features most important for personalized intent

• 2.    Types of Intent Detection

prediction.

Intent detection is a new and challenging issue in modern e-commerce. To approach this problem, many methods are used. Unfortunately, there is no classification of the methods nor comparison of available approaches in the ecommerce domain. Intent detection in literature is predominantly implemented by recommendation systems [56]. Other types of intent detection, like customer lifetime value or propensity prediction, are commonly used in production systems but usually not considered as an intent type for a single customer. In this section, we propose the following four clear classes of intent detection:

• -    Next-Item Recommendation: Models intent for the next customer action.

• -    Market Basket Analysis: Models intent of customer for buying an additional product.

• -    Propensity Modelling: Models behavioral intent on the e-commerce site.

• -    Customer Lifetime Value: Models the customer's intent for time, how long he will stay with the e-commerce site.

• 2.1.    Next-Item Recommendation

• 2.2.    Market Basket Analysis (MBA)

• 2.3.    Propensity Modelling

The next-item recommendation can be the most intuitive approach for predicting user intent. Many algorithms and approaches exist to recommend or predict the next item [56, 91]. Subsequent user interactions live at the heart of making personalized recommendations [90]. With the information about what will be the next move of the customer, an e-commerce system can apply a variety of personalization techniques to improve customer experience. The growing amount of choices that consumers face every day assigns high importance to such personalization.

Under the “Next-item recommendation” intent detection class we decided to include many algorithms which find one or N next items with the highest probability of the user interaction. The list of included approaches includes the Next-Item Recommendation, Top-N recommendation, Collaborative Filtering and Contextual Bandits, which results in a high number of literature references for Top-N recommendation class [10, 13, 19, 20, 28, 34, 40, 50, 52, 65, 68, 69, 74, 75, 79, 84, 85, 90, 95]. In the majority of analyzed articles, authors propose Recurrent Neural Networks (RNNs) to predict the next items in the session. The input to RNNs is a sequence of customer actions from the current session. On output, we get one or more next customer actions. This approach is similar to a machine learning application to the Natural Language Translation. We provide more details of this approach in the section 3.1.

Another common technique is Collaborative Filtering (CF) [67]. It discovers affinities to items which the customer has not interacted with yet (Top-N recommendations). Fang et al. [12] propose a combined item-based and user-based model to improve CF recommendation quality. Wu et al. [85] reproduced CF with the Feedforward Network to improve performance and scalability issues common for this method.

Despite the popularity of Recurrent Neural Networks and Collaborative Filtering in literature, there are traces of less commonly used but promising methods. Figueiredo et al [13] have used the semi-Markov random walk model on user activity logs to predict the probability of subsequent activities. Tang and Wang [69] propose Convolutional Network for the Next-Item Recommendation. The authors stacked a sequence of original customer action logs into an “image”, then applied convolutional filters to get the sequence on Top-N recommendations. The solution to the Contextual Bandit problem was proposed by Bouneffouf et al. [3] predict the relevant item for the user, taking into account his contextual information.

It is the affinity analysis method used to understand the purchase behavior of customers. It answers the question of which products are bought together. Thanks to this information e-commerce systems can elaborate the probability that customers are going to buy particular products from their purchase history. Although many e-commerce systems implement the MBA, its algorithms have problems in scalability, manual association rule definition and dynamics in data causing rule definition change over time [18, 33]. The most common methods for MBA are Association Rule Mining (ARM), Apriori and Frequent Pattern Growth [92]. Kaur and Kang [33], Pillai and Vyas [51], and Raorane et al. [54] use ARM with moderately good results. Pillai and Vyas [51] find ARM better than Apriori or Frequent Pattern growth because it can better address the rare item problem. Even though also recently, Apriori is used for simple manual analysis of the existing transaction data [38].

Propensity Modelling is widely used in e-commerce systems [58]. Typically, Propensity Modelling includes a propensity to convert, a propensity to churn, and propensity to unsubscribe. Propensities are used to model behavioral intent: viewer or buyer on an e-commerce site, as proposed by Loyola et al. [45].

The most common methods to calculate propensity scores are Logistic Regression, Tree-Based Methods, Support Vector Machines and Neural Networks [82]. Specifically, Recurrent Neural Networks are remarkably efficient in predicting propensity scores [1, 39, 61, 62]. Lang and Rettenmeier [39] and Wu et al. [87] show methods where feature engineering is not necessary because RNNs can learn them automatically. Loyola et al. [45] use encoder-decoder architecture, which was initially designed for machine translation. Tavakol and Brefeld [70] classify session topics by product attributes: category, gender, color and price. White et al. [83] model category attribute as a session topic, which allows predicting propensity score for each category. Wang and Zhang [78] propose the propensity score as a timefunction, which allows choosing the right time for the e-commerce system to obtain recommendations.

• 2.4.    Customer lifetime value (CLTV)

• 3.    Personalization Context and Customer Profile

Customer lifetime value (CLTV) is a prediction of monetary value attributed to a future relationship with a customer. The part of the method which can be beneficial for intent detection is Survival Analysis. Based on methods of calculating CLTV, the e-commerce system can infer how long a customer is going to use it. The most common method for CLTV is the Pareto/NBD (negative binomial distribution) model [29]. However, the most promising are methods based on deep learning like Deep Reinforcement Learning [72].

We define the intent detection output as creating a model for each individual user mainly to predict their incoming information [47]. The model for an individual user in literature is called "customer profile." Intent detection methods usually store in customer profile interest scores like propensities to convert short and long interest products or categories. The data collected in the customer profile is the input for the methods which improve user experience. Ecommerce systems create customer profiles where they also keep a transactional data like orders, delivery addresses and billing information. The customer profile is also a place for personal preferences, satisfaction levels, purchasing behavior, loyalty and similar. Moreover, keeping data in a clear and consistent format is imposed by General Data Protection Regulation (GDPR), especially the right to be informed [41]. In addition to activity logs, the user’s personal preferences can have a significant impact on customer intent. The customer profile is usually created by processing individual web visitors' activities undertaken during the interaction session. For example, users with different intents will visit separate categories and view assorted items. Their browsing activities, the frequency of page visits, and the time spent on each category can vary broadly [66]. Analyzing user activities with the context improves the profile further [2]. In a different context, the same actions can result in different intents.

In this section, we propose four separate types of user’s profile modelling and methods for creating this type of profile in the reviewed literature. The three types are:

• -   Session-Based Profile: which identifies the user by recent actions in the session.

• -   Interest Scores: which stores scores for the set of pre-defined preferences and intent types in the user profile.

• -    Latent Variables Profile: which identifies the user profile as a set of latent variables for Collaborative Filtering and Content-Based Filtering.

• 3.1.    Session-Based Profile

• 3.2.    Interest Scores

In e-commerce where most of the traffic is new anonymous customers, historical preference data is hardly achievable, no user ID exists, and recommendations should be based only on previously consumed items within the same session.

However, clickstream data can provide useful signals of ratings [64]. Wang et al. [77] show that user behavior clustering can be done on clickstream to detect different types of behavior. Su and Chen [66] note that the sequence of visits to a product webpage positively correlates to users’ interests. They found that users will firstly choose a product they most prefer.

All these findings are reflected in the number of literature references. In the majority of the reviewed literature the authors look for the information about customers and their intents in the activity logs [1, 10, 19, 20, 28, 37, 39, 40, 52, 53, 61, 68, 74, 75, 93, 95].

Technically, the user’s activity logs are time series. Recurrent Neural Networks (RNN) proved to be very efficient at processing sequential data [56]. Application of RNNs to the sequential data stemmed from language processing and text translation. Recommender-Systems extend language processing algorithms by adding more contextual information. One of the unique contextual attributes for Recommender-Systems, which is not that important in language analysis is the time. Further contextual information also improves prediction accuracy. For instance, Phuong et al. [50] propose a way to combine Collaborative Filtering (CF) and session-based profile to incorporate more context into the model. Smirnova and Vasile [65] include context information in the RNNs in two ways: first, by conditioning item representation on the context, then by conditioning the hidden dynamics of the RNN. Twardowski [75] embedded items (products) as a set of attributes to improve the session-based profile. We focus on the time and another contextual aspect in more detail in the succeeding sections.

Quadrana et al. [52] the same as Donkers et al. [10] use RNN with Gated Recurrent Unit (GRU) to model customer profiles from recent interactions. Sheil et al. [61] prove that Long Short-Term Memory (LSTM) gives excellent results as well. Besides, Sheil et al. [61] show the advantage of RNNs over classic algorithms: RNNs automatically discover features without manual modelling. However, Tan et al. [68] show that GRU outperforms the LSTM. Tang and Wang [69] propose a Convolutional Neural Network for analyzing clickstream events. They embed a sequence of new items into an “image” in the time and latent spaces and learn sequential patterns as local features of the image using convolutional filters. The sequence of user interactions is stacked into “image” and then apply convolutional filters to get the sequence for Top-N recommendations. Chen et al. [5] use memory-augmented neural networks introducing a per-user memory matrix to store historical user records and create user embeddings based on that. Wang et al. [79] propose an attention layer to improve prediction accuracy for long sessions.

The session-based profile is not interpretable because it hides the details in the hyper-parameters of the algorithm. Predictions that are hard to explain are challenging to explain to consumers who are asking for transparency. An example: customers can ask how algorithms determine their individual user experience [39]. The user's awareness of the GDPR law is rising, and we can expect that questions based on the right to be informed will be frequent. Because of that, the user profile should be as human-readable as possible to follow GDPR constraints [41].

In this approach e-commerce systems store predefined affinity scores in a customer profile. For example, a score from 0 to 1 expresses how much the customer desires features of the product like color, size. The value can also be calculated for behavioral attributes. One example can be customer price sensitiveness. We can correlate the score to the frequency of visits to the product pages, and the further to users’ interests [66]. Park and Chang [47] propose a Weighted Individual Interest score (WAII) of the product feature:

WAAl = a ( x с + a2 x b + a^x p + a^x i

where c is a click, b: add to basket, p: purchased, i: selected interest.

Diwandari et al. [9] propose a similar approach, with the additional factor of time spent on the item page. Sheil and Rana [60] classify users to clicking, browsing, buying groups. Su and Chen [66] define the user’s interest as a set of product categories that the user has visited. Tavakol and Brefeld [70] set the score for session topics, which they define as specific product attributes: color, gender, category, and price. Toth et al. [73] classify classes: purchase, abandon the shopping cart, and browsing-only. Gasparetti [15] proposes to use an item description to find keywords relevant to the user. It is worth mentioning that the interest score approach is better for model interpretability than other approaches described in this article, hence follows GDPR constraints [41].

• 3.3.    Latent Variables Profile

In this approach, a latent vector of the user's preferences is calculated based on implicit or explicit feedback to find similarities between user and item. The similarity metric has different proposals like Vector Cosine-Based similarity [14, 30]. Tay et al. [71] and also Hsieh et al. [22] use the memory-based network to learn the distance between user and item.

The typical problem formulation is as follows: for the parameters: p number of items (products), n number of item features and, i the number of users. We have the following relation:

R(«) =( pU)) Tx\ П)

where R(i) is the user rating, P(i) is the user’s latent variable vector, and X(n) is a vector of item properties. The problem of finding P(i) is to find a latent vector which minimizes error for all rated items:

minpa>

-У ((Р<‘))^тХ<^п> - R^)) 2     "•'

X(n) is also a latent variable in Collaborative Filtering, but in Content-Based Filtering is defined manually together with the item [2]. After calculating P(i), it is easy to get the rating for items the user has not seen before. Content-Based Filtering can estimate ratings for products similar to the ones the user has seen before. Specifically, a short-term content-based user profile is created that contains the brand and category information of items that the user has recently viewed. On the other hand, Collaborative Filtering identifies similar users and recommends based on what they liked. Collaborative Filtering is very popular in the reviewed literature [2, 8, 24, 25, 34, 37]. Collaborative Filtering implementations perform very poorly for methods that rely on user profiles such as Matrix Factorization or Item-K Nearest Neighbors. Hidasi et al. [20], Devooght and Bersini [8] and also Ko et al. [34] find that RNNs perform much better for Collaborative Filtering problems. Despite its popularity, Collaborative Filtering is suffering from the "early rater problem", so it requires many ratings before can recommend items to the users. Phuong et al. [50] and Hu et al. [23] propose a way to combine CF and session-based profile where user boosts results returned by session-based recommendations. Jannach et al. [27] propose a technique which places recently viewed items on the top of the recommendation list and appends the remaining items. Moreover, Jannach et al. [27] re-rank the items returned by a baseline recommender based on content features.

The problem with Collaborative Filtering and Content-Based Filtering for e-commerce systems is that both approaches require an existing customer profile to recommend items or predict user intent. Compared to Session-Based profile, Latent Variables profile is more manageable to explain for GDPR “right to explanation” questions.

• 4.    Dynamic Changes in User Interests

In e-commerce, customer interests drift over time. This drift is usually defined as the interest change of the items (products) and categories. For instance, the interest score highly depends on variables like item popularity, seasonality, and more. Another example is the change in the customer's propensity to buy or convert based on their behavioral patterns like mood. These changes can affect the customer's profile scores even without any user's interaction with the system. In order to keep intent detection accuracy at an acceptable level e-commerce systems need to update and maintain customer profiles, taking into consideration individual web visitors' activities to reflect any changes in their intents and preferences [59]. It is worth noting that the systems can infer dynamics only after they are observed [84]. Interest scores drifts mentioned above can depend on the many context attributes such as time, location, weather, companion [93]. For the same browsing behavior, users might have different intents, so it is important to also incorporate context into methods of updating profile [63]. Recommender systems traditionally assume that user-profiles and item attributes are static, so typically in the e-commerce system models are re-calculated frequently to reflect current trends [33]. Dai et al. [6] try to capture user preference dynamics using embeddings technique. They propose the method of predicting how user-item affinity score will evolve to, for example, predict the right time when the system recommends the item. Tan et al. [68] suggest training a model on the entire dataset, and then the trained model is used to initialize a new model, which is only trained using only a more recent subset of the data. Zhang et al. [89] suggest a more flexible framework to capture the heterogeneity in the duration of users' interests for the time-dependent recommendation using the hidden semi-Markov model. However, it seems this method is not scaling for real use case scenarios.

In this section, we extracted four types of user preference dynamics models. Which are:

• -    Interest Scores Decay: which decreases interests score over time.

• -    Periodic Interests: which tries to catch seasonality in user intents.

• -    Short and Long-Term Interests: which differentiates between general preferences and short-time intents.

• -    Popularity-Based Interests: which correlates user intent with item popularity dynamics.

• 4.1.    Interest Scores Decay over time

• 4.2.    Periodic Interests

• 4.3.    Short and Long-Term Interests

Typically, it is implemented as a monotonic decreasing function in order to express the changes of user’s interests over time. Gasmi et al. [14] propose an exponential form of the decay function:

-t time weight =-------------

1 I             t ^^                                                                      (4)

where Interest(t d ) is the Interest score at the time of last evaluation caused by the user’s activity. Hwangbo et al. [25] propose to apply data decay to reflect the seasonality of interests better. Twardowski [75] represents sessions by the time-decayed sum of event vectors associated with the actions performed by the user. Usage of data decay is questionable because a lot of potentially important information is lost. Campos et al. [4] get better prediction results when they do not use time decay.

Seasonality is extreme in e-commerce domains such as fashion. All products can be entirely different depending on the season of the year [25]. Moreover, shorter periodicity is observed by Gasmi et al. [14] and Campos et al. [4] including weekends, day of the week and even time of day. Campos et al. [4] and also Romov and Sokolov [57] obtain recommendation improvements by using time-dependent data partitioning such as morning, evening and weekend, workday. Wang et al. [80] use an attention layer to capture periodic interests. Guha et al. [17] define periodic interests as habits: actions that users regularly take, for example, reading a favorite blog or news site, checking stock prices. Jing and Smola [31] propose the RNN model built on Long-Short Term Memory cells to estimate when a user will return to a site and what their future intents will be. They prove that proposed architecture has high accuracy for recurring sessions. Zhang et al. [90] use a self-attention mechanism to capture sequential patterns and use it to model a user's recent interactions. Liu et al. [42] analyze user clickstreams to detect Common Visitor Paths.

Guha et al. [17] define interest as a task. A task can be short term like buying a hat when after the purchase the user is not any more interested in hats or can span multiple sessions. Although long-running tasks can be very long, it can take weeks or months. Long-time interests may cross months or years, and often do not have an end. Examples include the following sports teams, celebrities, or TV shows.

Li et al. [40] and also Quadrana et al. [53] define Short-Term interests as recent actions and show that they are the most important for prediction accuracy. At the same time, they show that modelling Long-Term interests can personalize recommendations which further improve accuracy. White et al. [83] define Short-Term interests as an "interaction context". They obtained the best accuracy when recent actions were used to predict interests in the following hour. Kumar et al. [37] use neural attention to model the Long-Term Interests of the user with promising results. Du et al. [11] integrate Long and Short-Term interests but modify the model online when a new user's interaction happens.

• 4.4.    Popularity-Based Interests

• 5.    Attributes for Customer Intent Modelling

The research shows the correlation between the popularity of the products and their attractiveness for the users. Marketeers stimulate product popularity by creating campaigns, sales and discounts. Jannach et al. [28] find that popular items increase recommendation accuracy: customers intent to buy popular products. Also, Jannach et al. [28] find that conversion rates for discounted items were 18 times higher than for non-discounted ones. Romov and Sokolov [57] find that also the popularity of categories has a significant impact on prediction accuracy. Jannach et al. [27] implemented a non-personalized baseline strategy which ranks the items based on the number of times they have been viewed or purchased in the training set. Park and Chang [47] and also Gholamian et al. [16] propose to calculate factors which affect individual interest depending on item popularity.

As we discussed in the previous sections, detecting customer’s intent can be significantly supported by recommendation systems. Various researchers demonstrate that the recommendations systems quality can be improved by leveraging context information [76]. By context information, we understand all data types with potential correlation to user intent. Liu et al. [42] show that incorporating contextual information is very important for behavior modelling. Experimental results show that models which incorporate additional context information, outperform those solely based on historical interactions [91]. Context plays an essential role in making recommendations since users’ behaviors are exceedingly affected by their current circumstances [55]. Contexts can play many roles in recommendation systems. For instance, an e-commerce system can employ different contexts as different domains or get different insights for the same user’s behavior for another context.

As we discussed in section 3, RNNs are widely used for modelling customer profiles from activity logs. However, RNNs have difficulty in modelling contextual information [42]. Wu et al. [86] show that models fail to select useful features from available context features and proposed a context features auto-encoding algorithm based on the regression tree. Kong et al. [35] hypothesize that there are multiple contexts which can lead to current action and proposed statistical method on how to identify it.

In this section, on the base of analyzed literature, we propose seven types of contextual information. They all are proven to improve recommendation accuracy:

• -   Activity Logs: data about recent customer actions.

• -   Item Features: data about the item (product) attributes browsed by the customer.

• -    Item Ratings: data about how customers rated an item.

• -   Interaction Time and Dwell Time: time aspect of the recent customer actions

• -   Discounts, Sales, Item Popularity: metrics of items (products).

• -   User’s Personal Data: information used to identify the customer.

• -   User’s Behavioral Data: different types of user interactions with the product not captured by the activity logs.

• 5.1.    Activity Logs

• 5.2.    Item Features

• 5.3.    Item Ratings

• 5.4.    Interaction Time and Dwell Time

• 5.5.    Discounts, Sales, Item Popularity

Every e-commerce system collects information about customer actions and transactions. Typical operations collected are search, click, view the item, add to cart, purchase. The activity logs are the most data source for recommendation systems. Most of the analyzed articles use common attributes from the Activity Logs.

In every interaction with the e-commerce system, the customer is looking for items with some features: brand, category, gender, color, and others. Intuitively, the item attributes can determine customer intent to some extent. Tuan and Phuong [74] show how to improve prediction accuracy by combining views with multiple content features, item ID, name, category of products or timestamp. Jannach et al. [28] find that the brand, price level, category and color are significant features to improve recommendation accuracy. White et al. [83] use the item category as the main model feature. Gasparetti [15] proposes to use the item description to find keywords relevant to the user.

Ratings can be explicit when the user rates the item (thumbs up, stars, and others), or implicit where the system implicitly rates user experience according to the user-item interactions. Explicit feedback on item (product ratings) is tough to obtain because of the low response rate. Most of the analyzed literature process ratings from user interactions. Only few incorporate explicit ratings [2, 66, 86].

Interaction Time is defined as every time aspect of the user-item interaction, which improves prediction accuracy. Zhu et al. [23] and Donkers et al. [10] show that capturing time intervals between actions can improve prediction accuracy. Diwandari et al. [9] show that for recurrent sessions time between the first and the last visit is also an essential factor. Smirnova and Vasile [65] show that timestamp in seconds broken into discrete features: month, hour and day of the week can improve prediction accuracy. Romov and Sokolov [57] find that the number of seconds between first and last click on an item, duration of the clicks on an item and its categories also have an impact on the recommendation prediction accuracy.

In analyzed literature, special attention is paid to the Dwell Time. The Dwell Time measures how long the user was in interaction with the item. Typically, an e-commerce system measures the Dwell Time as a time spent on the product page. E-commerce systems can use Dwell Time as an essential metric to measure user satisfaction and interests [4, 88, 94]. Incorporating Dwell Time as context information positively improves recommendation accuracy [7, 61]. The duration for which users stay on a product page is positively correlated to their interests [66]. Different click’s dwell times among click actions indicate different kinds of feedback information. White et al. [83] propose a threshold of 30 seconds above, which marks the page as containing useful content. Liu et al. [43], on the other hand, say that they cannot merely use a click dwell time threshold (30 seconds) to distinguish between relevant and irrelevant results. Yi et al. [88] try to normalize the Dwell Time across different devices and contexts. Dallmann et al. [7] find in the RecSys Challenge 2015 dataset some exciting artefacts: the two peaks. One with very short dwell time and one at approximately 35 seconds, which they encode as discrete classes of user feedback.

In many reviewed articles, authors use the time as a random variable used to predict next user-item time. Dai et al. [6] and Wang et al. [81] use the time to predict when the next action happens. Wang and Zhang [78] use Hazards Model in Survival Analysis to predict when is the right time for the system to make recommendations of the right product. Zhou et al. [94] use the Fast-Slow layer for very short sessions and an attention layer for noisy sessions to predict the Dwell Time of the next user action. Krishna et al. [36] and Zhou et al. [94] predict the next action and its duration, which can be used to propose more interesting actions to the customer. Jing and Smola [31] model RNNs using Long-Short Term Memory cells to estimate when the user will return to a site and what will be their future interest. Some researches use time as a variable to predict when the next action happens [6, 81], when it is the right time for a system to make recommendations of the right product [78], what is the dwell time of the next user action [94], the next action and its duration [36; 94], when users will return to a site and what will be their future interest [31].

These are essential factors which impact user intent. As we noted in section 4, there exists a correlation between the popularity of the products and their attractiveness for the users. Wu et al. [84] note that when a movie receives awards or nominations, there is an evident rise in predicted ratings, which matches the data trend. Jannach et al. [28] find that popular items increase recommendation accuracy: customers intent to buy popular products. Also, Jannach et al. [28] find that conversion rates for discounted items were 18 times higher than for non-discounted ones. Romov and Sokolov [57] find that also the popularity of categories has a significant impact on prediction accuracy. Wu et al. [84] note that when a movie receives awards or nominations, there is an evident rise in predicted ratings, which matches the data trend.

• 5.6.    User’s Personal Data

• 5.7.    User’s Behavioural Data

• 6. Conclusion and Future Work

  This survey study aims to review existing literature on methods applicable to customer intent detection in ecommerce to help new researchers build a broad understanding of the field. We analyzed nearly 100 recent articles from 2015 to 2019. We found no precise classification of the intent detection methods in e-commerce or other domains. To address that, we propose the following four clear classes of intent detection: Next-Item Recommendation, Market Basket Analysis, Propensity Modelling, and Customer Lifetime Value. We classify current literature in four aspects, which we believe are the most important for e-commerce system implementation: Types of Intent Detection, Personalization Context and Customer Profile, Dynamic Changes in User Interests, and Attributes for Customer Intent Modelling.

The typical personal data collected by the e-commerce system is age, gender, time and space, social data, location, occupation [32]. These data allow us to easily segment customers. Matsubara et al. [46] propose using additional context data as access devices, HTTP/document referrer and similar. GDPR restricts usage of the personally identifiable data. Users not often want to share personally identifiable data with an e-commerce system which applies primarily to the Personal Data. The reluctance of sharing the Personal Data makes it hard for an e-commerce system to obtain explicit consent to allow the processing of User’s Personal Data.

This type of data is not very popular in the reviewed literature but remains very interesting. Agrawal et al. [1] propose to acquire data like scroll rate on every page, clicks, taps, scrolls or more complicated inputs.

Even though intent detection is a prerequisite to personalization, and personalization directly impacts the ecommerce business's effectiveness [48], we found no holistic approach that adequately addresses our proposed qualities: personalization, adaptation to the dynamic changes, context information, scalability and performance. The best suitable methods with the potential to support majority qualities mentioned above are next-item recommendations and propensity modeling. Next-item recommendations and propensity modeling support the qualities in the following way:

• -    Personalization: Next-item recommendations and propensity modeling return sets of scores indicating interest in a specific product or category. These scores can be directly applied to individual customer profiles. We think that the set of scores can properly represent customer intent, but this still needs further research. Moreover, the scores are human interpretable, and keeping a customer profile in a clear and consistent format is imposed by GDPR [41].

• -    Adaptation to the dynamic changes: Many implementations of the next-item recommendations and propensity modeling can leverage time-dependent data to reflect periodic interests. An item’s popularity score can be modeled as an attribute of an item to and ingested by the method to follow popularity-based interests changes. Next-item recommendations and propensity modeling methods are successfully used to model short and longterm interest as separate models.

• -    Context information: Implementations of the next-item recommendations and propensity modeling represent a user and item as a set of properties. This is the opposite of the Collaborative Filtering method, where the user is usually represented as a single identifier. Set of properties representation allows leveraging a broad set of context information with a proven impact on the intent detection accuracy.

• -    Scalability and performance: Modern techniques like user and item embeddings are used in next-item recommendations and propensity modeling algorithms. Item embeddings approach reports the best scalability comparing to all other methods analyzed in this work.

In the analyzed literature, we have not found any research which defines the customer profile model that can holistically represent the customer intent. We see an important research area in customer intent modeling. Besides, we also see a potential in researching the intent detection approach based on the next-item recommendation and propensity modeling algorithms. We think that the new research should focus on a method to appropriately address our proposed qualities: personalization, adaptation to the dynamic changes, context information, scalability and performance.