An Interactive Cart with Analytics Recommendation and Tracking-iCART

Published Online April 2020 in MECS

Shopping carts are used in big malls or supermarkets to carry large amount of groceries around the area to look for items required. These are very useful mechanical tools to carry a large amount of items, but some technologies can be used to make them a lot more useful. Here, we have used deep learning models to identify the customer using a camera and using his previous shopping history we are recommending him appropriate items he might want to buy. Along with face recognition the age and gender of the personorcustomer can also be extracted to suggest or predict the items he might want to buy. A recommendation system is built to identify the products the customer might want to buy which works based on the inputs received from deep learningmodel.

One very important aspect of supermarkets is the staff who are present to answer the customer queries. but, the supermarket has to hire the sufficient number of staff, which increases the cost of management for the supermarket. iCart provides the solution of this problem using a Chatbot which is integrated in the display given to them. They can type any type of queries they want in the chatbot and it will give the relevant answers. The display provides the customers with a way to enter the items they want to buy.

The supermarkets have a lots of stuff, and sometimes the customers have to search a lot for the products they want to buy. iCart provide an innovative solution to this problem. Using Indoor positioning system we can give exact path to the products the customer has entered into the system using the interface given with the shopping cart. The Indoor positioning system uses Bluetooth sensors to identify the stalls in the market. The raspberry pi keeps detecting the Bluetooth devices and based on the signal strength, it decides when the iCart trolley is present at which stall. The indoor positioning system uses a modified version of dijkstra’s algorithm. The algorithm is customized to work with the layout of the supermarket. The customer can add items in the cart and on each addition the algorithm calculates the shortest distance with which the customer can go through all the stalls which contains these items. The integration of all this technology is the main difficulty in implementing a system with so many features and complexity. This paper is aimed at explaining all the steps involved in making the iCart system, including the individual systems involved and also the integration of all the technologies involved. This paper will provide the implementation done for making iCart.

• II.    Related Work

Many smart cart solutions are already proposed like You- Chiun Wang and Chang-Chen Yang in the paper Intelligent Shopping Trolley (IST) System by WSN to Support Hypermarket IoTService [1] proposed a smart cart which uses zigbee routers for indoor positioning system and a centralized server to answer all the queries via a LCD screen. They use a simple rule based algorithm for recommendations which can be greatly improved by using machinelearning.

Pritha N, Sahana S et al., Apr-2018 in their paper Smart Trolley System for Automated Billing using RFID and IoT[3] proposed a smart trolley for automated billing which incorporates even weight scanners and smart checkpoints. The smart checkpoints and weight scanners makes the process of billing more complex. But, they provide a validation for total number of items in the cart.

Suganya.R, Swarnavalli. N et l., March 2016 in their paper Automated Smart Trolley with Smart Billing Using Arduino[4] proposed a smart cart system in which a mobile app is used to find the items the customer wants to buy. The carts are associated with RFID tags which must be scanned by the customer and items must be put into it. It generates the bill for the customers automatically and customer doesn’t have to stand in theline.

In [6] the focus is on identifying the behavior of customer who is interested in buying things online. The main focus is to recommend the products that are interesting to him. In e-commerce customers usually fail to find the right product from many online offers. In this paper author tried to find best classifier for customer based on his behavior. The results shows decision tree table gives highest accuracy.

In [7] the regression technique and boosting techniques are implemented to process the huge data. This paper tries to predict sales of retail stores using various ML techniques. The results shows boosting technique gives high accuracy than regression technique.

In [8] as time is more important. In this paper authors used BOLT ESP8266 based IOT technique to save time of customer. They used scanner in the trolley which scans items and display the bills on the display panel and sends the same information about number of items and bill amount to billing section which will save the time of waiting near billing counter.

In [9] RFID readers are installed to maintain the list of all products and LCP display is used in the cart to identify the items purchased by customer and also display the available items in the mall and mobile device is installed in the cart so that customer can make the billing without waiting for queue and all the customer processed information is stored in database for further analysis.

In [10] an intelligent system is designed for only edible objects like vegetables and fruits because for these objects RFID tag or bar code need to stuck on each item. So the deep learning technique is used to measure the weight of each edible object which is attached in cart and customer uses camera for scanning and billing the item.

In [11] smart billing system is designed using hybrid technology which includes Zigbee and RFID. Zigbee is based on IEEE 802.15 used of long distance communication. RFID tag is available on all items in shopping mall. Shopping cart is installed with PID(product information device) EEPROM, Zigbee, RFID. Product information is saved on EEPROM and sent to billing counter using Zigbee module. EEPROM data will be accessed for billing which will save the time of customer.

In [12] they compared the traditional machine learning with enhanced deep learning algorithms. The focus is to study the behavior of customer using deep learning. Six month data was collected which is sparse and high dimension. Decision tree (DT) and random forest (RF) machine learning algorithms are compared with deep belief networks (DBN) of deep learning method with three hidden layer. Result shows for DT and RF with 64 percent and 72 percent accuracy and DBN with 86 percent accuracy.

In [13] edge computing is used for computing tasks which is used for computing zillion bytes of data. Comparative study was done using deep neural network with three layers and tanh as activation function. The main focus is to use AI and edge computing for computing large data.

In [14] overview of advanced machine learning and IoT is used to facilitate analytics. Summary of Deep learning models are discussed. The challenges of deep learning and characteristics of IoT is reviewed. DL model with design factors and mobile edge is discussed.

In [15] machine learning is used for collection and analyzing large dataset. IoT application has become smarter in all aspects. The data collected using IoT devices is used in modern era for computation. This paper focus on ML techniques and IoT applications for smart transportation system.

• III.    Components Oficart

• A.    Indoor Navigation

Perhaps the most nettling task of any customer in the supermarket is foraging for the required items. Some items may be placed in inconspicuous locations leading to the customers assuming that the item is not present in the supermarket, this is a loss for the customer as well as the owner. In order to surmount this issue iCart implements indoor positioning system. It makes use of a 2D map of the supermarket constructed on the user interface, the customer is allowed to type in his desired item, and the IPS provides the shortest possible route to that item on the 2D map by making use of dijkstra's algorithm. In case the item is not present in the supermarket, the system will notify the customer about its absence saving the customers valuable time and energy which would have been squandered searching for theitem.

• B.    Product Recommendation

Providing good recommendation of products will not only boost the business of the supermarket employing icart system but, will also make the shopping experience a lot better for the customers. A recommendation bar is provided at the bottom of the user interface. Generating recommendations for a user is done using the techniques of data analytics. iCart cleverly extracts customers important features such as his/her age and gender, and passes these features into a recommendation model to generate appropriate real time recommendations for the customer.

• C.    Chatbot

A Chatbot is an interactive system capable of natural language processing and carrying out necessary actions, it is an essential component of icart. The Chatbot acts as a virtual agent eliminating the need of all the human agents who often roam around the mall or supermarket to assist the customers. A user can ask any doubt regarding the products or general question about the supermarket as though he/she is asking to a human agent and the chatbotwouldbe able to answer all the queries. If the user is not satisfied with the answer provided by the chatbot, a human agent can be sent a request to go onsite and resolve theissue.

In order to implement Indoor Navigation it is necessary to find the customer’s location within the supermarket first.

One possible way to achieve this task is to make use of the GPS coordinates, but there are two limitations with GPS technology which will not allow us to proceed with this method, they are:

• 1)    The GPS signals are impinged by the roof of the supermarket, this introduces errors in theresult.

• 2)    The accuracy of GPS is in terms of several meters, this is not applicable in supermarkets where the accuracy is expected to be in termsof centimeters.

Another solution could be to employee trilateration using Bluetooth modules.

Although trilateration is an efficient method to find out the position of the customer in an obstruction free environment, because it obeys inverse square law as shown in Figure 1, there exists a major issue in a real time environment of a supermarket where there are a lot of obstructions, because the RSSI values of the Bluetooth modules will disobey the inverse square law as shown in Figure 2, due to the presence of various objects between the transmitter and receiver. This leads to egregious errors in the values of the customer coordinates calculated. Hence, this method is not apropos to a supermarket environment.

• IV. Implementation

The iCart system consists of three main components, namely chatbot, recommendation system and indoor positioning system. Additionally, a user interface is required to integrate chatbot and recommendations module of the system. The user interface is rich in functionality in that it shows the layout of supermarket as a map, where guidelines are succinctly provided to gauge user towards the product in the shopping cart.

Additionally, we will also need some hardware components like a shopping cart to integrate our display, raspberry pi and camera.

• A. IndoorNavigation

Fig. 2. Plot of RSSI vs Distance under real time conditions

Fig. 3. Setup of IPS in a supermarket

The final and real time solution for indoor positioning within a supermarket will involve utilization of real time behaviour of RSSI values with distance and hacking some generic features of a supermarket.

There are two important facts to consider while approaching the real time solution. Firstly, supermarkets are places to sell items and make profits, the owner would notbe interested in squandering any space in the supermarket by leaving it empty. Therefore, there would be as many possible item stacks as there could be, covering the entire supermarket as shown in Figure 3. Secondly, although, the RSSI values of Bluetooth modules disobey the inverse square law in real time, it still exhibits a decreasing trend which can be clearly observed in Figure 2, this implies that closer an object gets to the Bluetooth module, higher will be the RSSI value. Considering the above two factors we can have a setup in which a Bluetooth module is attached to each and every item stack present in the supermarket. Also, all trollies contains an RSSI signal strength sensor which will detect the Bluetooth address and signal strength of each and every Bluetooth module attached in the supermarket as shown in Figure 3. If the signal strength of a Bluetooth module crosses a particular threshold value, then the customer location will be updated to that Bluetooth modules location and the changes can be seen on the user interface. Since Bluetooth modules are attached to all the stacks present in the supermarket, now one can expect the accuracy in terms ofcentimeters.

After the indoor positioning is done, the next step in indoor navigation is to find the shortest possible routes from source to the destination. The most facile method of achieving this task would be to employ the dijkstra's algorithm. Here, each and every item stack can be considered as nodes, the direct path from one stack to the other as edges connecting the two nodes and distance between these nodes will act as the weights of the connecting edges. A graphical representation of Figure 3 is shown in Figure 4. The dijkstra's algorithm is given below.

Algorithm dijkstra's(graph,source):

for each vertex v in graph:

distance[v]     ←     infinity previous[v] ← undefined

Q ← set of all nodes in graph

W[] ← weight of all pair of connected nodes ingraph distance[source] ← 0

while Q is notempty:

u ← node with in(distance[]) delete u from Q for each neighbour v of u:

if(distance[v] > distance[u] + weight[u, v]) distance[v] = distance[u] + weight[u, v] previous[v] = u return previous[]

Fig. 4. Supermarket represented as a graph

Fig. 5. Sample display of shortest route between 2 nodes

Once the IPS is setup and the graph is prepared, the indoor navigation system will be ready to use. When the customer queries for any item, the indoor navigation system will obtain the customer’s current location (source node) from IPS and item location (destination node) from the supermarket database, it runs the dijkstra's algorithm with these two parameters and find the shortest possible route to that item, this path will be shown in the user interface. A sample snapshot of user interface is shown in Figure 5, the customer is present near itemstack 3, the item is present in stack 7 and the blue path represents the shortest route from customer to item.

• B.    Product Recommendation

In order to recommend the products to an individual customer, the system first extracts the essential features of the customer which influences his/her choice of items to purchase. To find out these features data visualization is done on a survey carried out in a supermarket, the inference from the visualization is that there is a correlation between item purchased and features such as month of the year, age and gender of the customer. A visualization of number of leather jackets purchased against the month, age and gender is shown in Figure 6, Figure 7 and Figure 8, from these Figures we can conclude that a 25 year old male customer is likely to purchase leather jacket in the month of December.

Fig.7.Sampledisplayofshortestroutebetween2nodes distribution of purchases over age

After deciding on the model parameters, the Naive Bayes classifier is employed to recommend the items. The recommendation algorithm is writtenbelow:

Algorithm recommendations(a, g, m, n,P[]):

prediction_probs[] ← P (P i|a⋀ g ⋀ m) sorted_predictions ← descending_sort(prediction_probs)

best_n_recommends ← sorted_predictions[0:n] return best_n_recommends

Where, a is age of customer, g: gender of the customer, m: month, n: number of recommendations and P[] is the list of all products.

The top 10 ranked items from theNaive Bayes classifier will be displayed on the user interface as recommendations to a customer.

Since, the recommendation algorithm uses customer features such as age and gender, it is necessary to have infrastructure to extract these features. For this, a camera is mounted on the trolley which takes the image of the customer while he/she is using the trolley and sends this image to AWS as a request to extract all the required features. The flow chart of feature extraction using AWS is shown in Figure 9.

• C.    Chatbot

In order to implement a chatbot a machine learning toolkit for conversational AI called RASA is used. RASA consists of two components RASA NLU and RASA CORE. RASA NLU is used for natural language understanding, it uses support vector machines for intent classification and conditional random fields for entity extraction. RASA CORE is used for dialogue state tracking; it implements a recurrent neural network (RNN) with LSTM in order to track the state of the conversation.

RASA NLU is used for natural language understanding, it uses support vector machines for intent classification and conditional random fields for entity extraction. RASA CORE is used for dialogue state tracking, it implements a recurrent neural network (RNN) with LSTM in order to track the state of the conversation. To develop a chatbot, the capabilities of RASA NLU and

RASA CORE are combined, as they work synchronously together. The data flow diagram of the implementation of the chatbot is shown in Figure 10.

Fig. 9. Flow chart for feature extraction from image of face

Fig. 10. Data flow diagram of chatbot using RASA

• V.    Results

Indoor Navigation System:

In order to evaluate the indoor navigation system we use the graph represented in Figure 11. Tabular columns are constructed which compares the distance travelled by a customer to reach a particular item using dijkstra’s algorithm against the average distance needed to be travelled to reach the item considering all possible paths. Here we assume distance between two stacks is one unit.

Fig. 11. Supermarket represented as a graph

Table 1. Evaluation of dijkstra's algorithm

Source	Destination	Dijkstra’s Path	Average Path
2	1	1	3
3	1	2	3.5
4	1	1	3
5	1	2	2.66
6	1	3	3
1	2	1	3
3	2	1	3
4	2	2	2.66
5	2	1	2.33
6	2	2	2.66
1	3	1	3
2	3	2	3.5
4	3	1	3
5	3	2	2.66
6	3	3	3
1	4	1	3
2	4	2	3.5
3	4	1	3
5	4	2	2.66
6	4	3	3
1	5	1	3
2	5	1	3
3	5	2	2.66
4	5	1	2.33
6	5	2	2.66
1	6	1	3
2	6	2	3.5
3	6	1	3

From table 1, It is clear that the distance travelled is reduced to a greater extent using dijkstra’s algorithm, even for a graph with only six nodes.

Recommendation System:

In order to evaluate the recommendation engine, each tuple in test dataset is passed to the recommendation algorithm with the number of recommendations “n” set to 25 for all the inputs. The result is positive, if the item purchased by the customer in the test dataset is present in the list of recommendations produced by the algorithm. The efficiency of the algorithm is the calculated as shown in equation (1):

Г =

number of positive results size of dataset

The efficiency of the recommendation algorithm using equation (1) was found to be 0.7, i.e, 70%.

Chatbot:

Although a chatbot covers a number of use cases, to evaluate the chatbot a single use case is considered in which a customer complaints to the bot that he/she could not find particular item, and it would be helpful if that product was bought to the supermarket in the future. The Thechat bot responds to this complaint by apologizing to the customer for the inconvenience caused as shown in Figure 12, it also inserts that item into the missing items database as shown in Figure13.

Fig. 12. Conversation between customer and bot

ITEM		DAY
	Filter	Filter
1	mango	155498983...
2	mango	155498984...
3	mango	155498984...
4	mango	2019.04,11
5	mango	2019,04,11
6	banana	2019,04,11

Fig. 13. Missing Item updated in database

• VI.    Conclusion

The Supermarket chain is a highly established chainof supermarkets and malls. In spite of this, loopholes exist in the current system. These loopholes can be eradicated to better improve the customer relationships with the supermarket owner and also customer satisfaction can be increased likewise. Among the most important stakeholders of a supermarket ecosystem are the owner and the customer buying products from the mart. By providing a system that helps solve existing loopholes by utilizing minimum resources, the product handles many complexities on the part of the owner and the customer alike. Bluetooth technology oneof the most critical variables the supermarket owners strive to optimize, is increased in the process.Since the product targets niche technology, many problems exist as research problems. A better cost effective and accurate solution to indoor positioning system will aid in improving results served to customers by the product. Also improvements can also be achieved by scaling the system across supermarkets. This will lead to accumulation of lot of data in a central location. This data can then be fed into machine learning models utilized by the product to fine tune results of recommendation fed to the customer. To conclude, the product has rich branches for further extension, which will impact the supermarket ecosystem in considerable ways so as to increase revenue and maintain competence among other types such as E Commerce.