Experimental validation of contextual variables for research resources recommender system

Published Online April 2018 in MECS

It has become a very time-consuming effort for the academic community to locate useful and relevant information on the internet due to the increase in the amount of digital information on the Internet. Recommender system deals with this problem of information overload by filtering relevant and personalized information fragment out of a large amount of dynamically generated information [1, 2]. Recommender systems are beneficial in different domains such as movies [3, 4], music, news [5, 6], scientific libraries [7, 8], e-commerce [9, 10] etc. In the domain of research, there is an exponential increase in the number of research resources available on the internet hence; recommender systems assist researchers in finding research resources of interest quickly and accurately [3]. Research resources domain has become a much focused area [11, 12, 13] for recommender systems because of their two major and very important features which can be harnessed by recommendation techniques. These features include the text of the research paper and the citation web which links a research resource to other similar research resources. The text of different research resources can be analysed for similarity or by finding relevant research resources based on their linking to other research resources. An author must cite appropriate and relevant previous studies to assist readers to have in-depth knowledge of a particular research resource [14].

Fig.1. A contextual rating recommendation model [9].

Also, context-aware recommendation technique [15] further enhances recommendation of research resources by modeling and predicting researcher’s preferences through the introduction of contextual information (variable) into the recommendation process as an explicit additional class of data [16, 17]. Incorporating contextual information improves the prediction accuracy of recommendation systems and also increases the performance of recommendation systems [18, 19]. Fig. 1, is an example of a context-aware recommendation rating model where the rating given by user U1 on item I1 at a specific context C11 is equal to 3.

User x Item x Context ^ Recommendations. (1)

In “(1),” Context specifies the contextual information (variables) associated with the specific domain of application of the recommender system [20]. This contextual information can be classified into the following categories as in Fig. 2, Environmental context, User context, Item context and device context [21]. Environmental context captures information such as: time and date, noise level, weather current location, direction. User context can have the following information: interest, goals, age, activities, country, language, profession and knowledge. In device context, information like connection speed, screen size, operating platform, manufacturer and brand type are relevant. For item context, information here could include anything that characterizes an item in a specific domain. For example, in research resources domain, an article can be characterized by its major sections, such as introduction, review and methodology.

Fig.2. Categories of contextual information

In CARS, context can be incorporated into recommender system by three major approaches which include pre-filtering, post-filtering and contextual modeling [22, 8]. Pre-filtering approach uses contextual information to select the most relevant data, and then it uses any of the 2D traditional recommender system for generating the final recommendations while post-filtering approach ignores context information in the input data, any traditional recommender system is first used as input, and trained on non-contextual data before using context to generate the final recommendations. Contextual modeling approach uses contextual information directly in the recommendation function as an explicit predictor of a user’s rating for an item.

• A. Context Modeling in CARS

Context modeling in CARS models and predicts user’s interest by integrating relevant contextual information

(variables) into the recommendation process as explicit additional categorical data [21, 23] as shown in fig. 3. These long-term interest are normally expressed as ratings and modeled as a function of items, users (authors) and context. For example, in our application to recommend citations to users, where users (authors) and research resources (articles) are described as relations having the following attribute:

Articles: the set of all the articles that can be recommended to users (authors) which can be expressed as:

Articles ( article_id, Title, keywords, pub_date )

User: the individuals that seek recommended which can also be expressed as:

Authors ( author _id, author_ name, author_email ) Context consists of the life-stages of an article which can be expressed as:

Stages (introduction, review, methodology ).

Fig.3. Context modeling technique

Formally, assuming d1,d2, ......, dn are dimensions in which two of these dimensions are User and Item, and others are context. Each dimension di is a subset of a Cartesian product of some attributes Aij; ( j = 1,…. ,ki), i.e., di C Ai1 xAi2 x: : : xAiki , where each attribute defines a domain of values with one or more attributes as a key that uniquely define the other attributes. In some situations, a dimension can be defined by just one attribute, and in such situation ki is expressed as 1. In our application where we have three dimensional recommendation space Author ^Article ^Stages, the Author dimension is defined as Author C author_ name x author_email which is made up of a set of authors that have different names and e-mails. Likewise, the Article dimension is defined as Article C Title x keyword x pub_date which is made up of a set of articles that have different titles, keywords and publication dates. Also, the Stage dimension can be expresses as StagesC introduction ×review× methodology and consists of the list of citations in the various stages. For example, given dimensions d1,d2,........,dn, the recommendation space for these dimension can be defined as Cartesian product S = d1 *d2 x xdn. Also, assuming R is a rating domain representing the ordered set of all possible rating values. Then, the rating function is can be expressed over the space d1 *d2 x xdn as R: dix...... xdn ^ Rating [q]. In our application with Author × Article× Stages, For instance, the rating function R can be defined over a recommendation space, Author × Article× Stages by specifying how much author a∈ Author liked article r ∈ Article at the stage s ∈ Stages, R(a, r, s ). The rating function R is usually defined as a partial function since it is only the initial set of values that are known. The goal of recommender system usually is to make the rating function R total by predicting the unrated items according to user’s interest.

The use of context is very crucial in finding reliable recommendations on the internet; therefore, there is a need for qualitative research to experimentally validate the contextual variables to be used in any context-aware recommendation systems to enhance predictive accuracy and efficient recommendations, especially in research resources domain. Most of the past studies seem to have failed to give cogent attention to validate experimentally the contextual variables used in designing their recommendation systems [24, 25, 26, 27]. Hence, this paper attempts to experimentally validate the contextual variables in the domain of research resources recommendation system, in order to avoid the use of inappropriate context that could have undesirable effects on recommendation accuracy and user satisfaction. Also, to sensitize Context-Aware Recommender System (CARS) researchers on the need to validate context before their use in recommendation systems.

The remainder of the paper is structured as follows. Section II summarizes some related work in context-aware recommendation systems. Section III presents the experimental design of our CARS model and the experiment conducted to evaluate the proposed model is detailed in Section IV, and finally, Section V contains the conclusion and some future direction.

• II.    Related Work

Context-aware systems and recommender systems [28] focus on one central goal, they suggest to user relevant information or services selected from a large amount of dynamically generated information. The difference is that, in context-aware systems [15], the selection is based on the user’s context which allows the users to trust in the recommended items [29, 30] while the recommender systems rely on the user’s interest. These two systems are complementary to each other, hence the need for their integration. [31] explores how contextual information can be used to create intelligent and useful recommendation system. They provide an overview of the multi-faceted notion of context, different approaches for incorporating contextual information into recommendation process and also an illustration of how to use the approaches and different application areas where different types of contexts area exploited. Contextual information can be acquired explicitly, implicitly or by inferring [32]. [33] reviewed and pointed out the need for thorough investigations into the concept of contextual information from the user’s viewpoint in research resources domains.

• [34]    demonstrated that various types of contextual information can be inferred with a reasonably high degree of accuracy in some applications by using different data mining methods, such as Naive Bayes classifiers and Bayesian Networks. [35] incorporated contextual dimensions (such as time, companion, and weather) into the recommendation process and then used machine learning techniques to provide recommendations in a restaurant recommender system. [36] proposed a novel approach for the user's satisfaction modeling based on incorporating the user’s contextual information into the rating prediction and consideration of previous users’ rating history. In [37], location was used as the contextual information for recommending music tracks that are suitable at a particular point of interest (POI) for users. On the other hand [38] used time, temperature, weather and some other contextual information to suggest appropriate music to different users. [24, 39, 40] used citation context information to recommend top N ranked paper for researchers. Despite all these models, [41] observed that how to incorporate contextual variable into recommendation system and which variables should be considered appropriate are still research questions. He also discovered that in the recent time, most academic research focuses more on the development of reliable CARS algorithms but ignore the identification of appropriate context and the interpretation of contextual effects. Since contextual information is a very crucial factor that determines the accuracy of recommendation [42], there is need to thoroughly and carefully selects and experimentally validates them to know their viability before use.

• A. Common Issues in CARS

Despite the success of CARS in various domains, both in research and practice, they are still faced with some fundamental issues and challenges which are discussed below.

Data sparsity: After the relevant data has been selected based on specified context using pre-filtering approach, the data sparsity level increases within the system. Therefore, making relevant recommendations then becomes practically impossible for the system.

Exact context: Pre-filtering techniques use exact context for filtering which could be too narrow to generate accurate and reliable recommendations. Sometimes, with exact context, enough data may not be available to match such exact context and hence, it will not be possible to generate any recommendations for the specific context. The practical solution to this kind of issue is to generalize contextual information [19].

Predefined context: In nearly all CARS, contexts are always predefined, which can cause serious problem especially when it comes to temporal contexts. Predefined temporal contexts can be too general for some users. Also, in a relatively long and general context, it is possible for user’s preference to change significantly. Where the context is not also general enough, CARS might find it difficult to obtain sufficient information to learn the user’s preference. Aside these, there may be several generalized contexts. Therefore, it is very important to determine which context to use and the methods to use in generalizing the context. One option is to use a manual, expert-driven technique [32].

• III.    Experimental Design

This section introduces the data sets and the approach used in designing the experiment.

Datasets Description: The datasets were extracted from the ACM database and include 500 research papers comprising of 250 journal articles and 250 proceeding articles from the domain of Computer science (Information Retrieval and Recommender System, Cloud Computing, Software Engineering, data management and visualization, Data Mining and Computational Intelligence). The research papers extracted were from

2011-2015 (a period of five years), 10 articles were selected from each year for journal articles and proceeding articles respectively.

In the process of writing a research report, a researcher needed to consider three very important stages which include introduction, review, and methodology. We chose these three stages as the contextual variables to be validated. The constraint within some of the papers is that some of these variables come under different names, the introduction was referred to as background, the literature review was called related works, previous works or studies etc. also, the methodology was sometimes referred to as design or our approach. Some heuristic methods were used to structure the data to appropriate model for analysis (Table 1). That is, the citation count in different stages of all the articles were extracted and normalized.

Table 1. Structure of the Research Resources

Author name	Article type	Intro. Citation count	Review citation count	Methodology citation count
Author 1	journal	---
Author 2	Proceed.	---	---
		---
---	…	---	---
Author N	…	---

• IV.    Methodology

Analytical tools (SPSS and Microsoft-excel) were used to determine the variability between the citations of articles both in the journals and proceedings for the three contextual variables; Introduction, Review, and methodology. T-test provides information about the results of the comparisons between the means of journal articles and proceeding articles. The significance level provides the most important element of reporting the t -test, it indicates the level of difference observed between the means whether greater than would be expected by chance (typically p < .05). T-test is also used to report the degrees of freedom (98) and the t-value (1.96). For a dependent samples t -test, the degrees of freedom will always be N – 1. For an independent samples t -test the degrees of freedom will always be N – 2. It should be noted that t -tests with degrees of freedom that is more than 100 will always be significant if the value meets or exceeds 1.96. This is a useful point in understanding the necessary critical value of a t -test for it to reach statistical significance. In this work, t-test analysis, uses the means and standard deviations, to computed and compare the variability between articles in journals and proceedings using the formula in “(2),”

s 2 = standard deviation for sample 2 (articles in proceedings)

n 1 = total number of values in sample 1(articles in Journals)

n 2 = total number of values in sample 2 (articles in proceedings)

Standard deviation is a measure of dispersion which indicates how much a sample data is spread out. Specifically, it shows how much data is spread out around the mean. The standard deviation s which is the root means square deviation of the sample value from their mean % for both proceeding and journals is computed as “(3),”

s

=I-

P=1 (X i -X) n-1

where, xi = is the given sample values

% = mean n = total number of samples taken.

t =

% 1 -% 2 IS12. S22 ’ \ »1  712

The results of the t-test and p-test analysis for the two samples, proceeding and journals are presented in Table 2.

% 1 = mean of sample 1(articles in Journals)

% 1 = mean of sample 2(articles in proceedings)

s 1 = standard deviation for sample 1(articles in Journals)

Table 2. Results of t-test and p-test for Journal and Proceeding

FIELD	CONTEXT	ARTICLE TYPE	MEAN	SD	T	P	REMARK
RS/IR	Introduction	Journal	10.28	10.12	0.619	0.537	NS
		Proc.	9.14	8.20
	Review	Journal	19.80	17.35	2.347	0.021*	S
		Proc.	13.26	9.33
	Methodology	Journal	6.54	8.47	0.687	0.494	NS
		Proc.	5.56	5.48
CC	Introduction	Journal	9.60	7.63	2.374	0.020*	S
		Proc.	6.22	6.57
	Review	Journal	29.62	32.39	3.044	0.003*	S
		Proc.	14.92	10.83
	Methodology	Journal	7.90	12.74	1.892	0.061	NS
		Proc.	4.28	4.56
SE	Introduction	Journal	13.84	9.96	2.454	0.016*	S
		Proc.	9.16	9.09
	Review	Journal	20.64	16.12	0.359	0.721	NS
		Proc.	18.78	32.98
	Methodology	Journal	7.34	9.24	1.576	0.118	NS
		Proc.	4.82	6.51
DBM/V	Introduction	Journal	12.58	9.68	3.383	0.001*	S
		Proc.	6.66	7.71
	Review	Journal	28.44	13.65	8.938	0.000*	S
		Proc.	8.10	8.52
	Methodology	Journal	11.36	9.02	0.774	0.059	NS
		Proc.	6.96	13.57
DM/CI	Introduction	Journal	12.48	7.98	3.549	0.001*	S
		Proc.	7.00	7.45
	Review	Journal	29.76	17.95	6.695	0.000*	S
		Proc.	9.44	11.76
	Methodology	Journal	9.30	13.73	1.994	0.049	S
		Proc.	5.26	4.12

N=50, degree of freedom (df) =98, NS-not significant, S-significant, when P>0.05=NS and when P<0.05 then S

From Table 2, there was significance variability between the citations of the contextual variable (Introduction) in journals and proceedings in the domains of CC (t=3.044; p=0.003), SE (t=2.454;p=0.016), DBM/V (t=3.383; p=0.001), DM/CI (t=3.549; p=0.001) at 0.05 level in each case while there was no significance variability in the citations of the contextual variable (Introduction) of articles in journals and proceedings of RS/IR since (t=0.619; p=0.537). Also, there was a significant variability between the citations of contextual variable (CV) (review) in both journals and proceedings in the domains of RS/IR (t=2.347; p=0.02), CC (t=3.044; p=0.003), DBM/V (t=8.938; p=0.000), DM/CI (t=6.695; p=0.000) at 0.005 level in each case. The variability of citations in the review contextual variable of articles in journals and proceedings of SE domain, however, did not show a significance variability since (t=0.359; p=0.721). Finally, there was no significant variability between the citation of the contextual variable (Methodology) articles in journal and proceedings in the domains of RS/IR since (t=0.687; p=0.494), CC (t= 1.892; p=0.061), SE (t=1.576; p=0.118), DBM/V (t=0.774; p=0.059), at 0.05 level in each case. There was, however, a significance variability between the citations of the contextual variable

(Methodology) in articles in both journals and proceedings in the domain of DM/CI since (t=1.994; p=0.049). Figure 2 shows the different variety based on the different domains of study.

Fig.4. Variability in contextual variables (citations) between Journals and Proceedings in the five different domains.

Generally, from fig. 2, irrespective of the domain of article, journal articles have higher variability in their citations at introduction contextual variable, very significant variability between the articles in the review contextual variable and high variability in the methodology contextual variable than the articles in the proceeding under the three contextual variables as seen in the fig. 3. The reason for this high variability between journal and proceeding citations could be attributed the following reasons.

Fig.5. Overall view of variability in contextual variables (citaions) between Journals and Proceedings

Conference articles are short communication medium for presenting ongoing research work which may later be submitted for publication in journals [43]. The traditional reviewing process in conference publications does not usually follow the standards and quality of journal publication review [44] such as adding more than one full round of refereeing and a final editing stage. This may account for the lesser number of citations in different sections of the proceeding articles. On the other hand, journal articles are papers that are thoroughly appraised by experts in the same field as the author and it completes the work of conference articles by offering full proofs, comprehensive experimental results and referencing. Generally, journal article attends to the weaknesses of conference articles. This may account for why more citations are generally included in different sections of journal articles than conference articles as shown in figures 2 and 3.This informs us on how to incorporate these contextual variables for the two samples (proceeding and journal) based on the variability observed during recommendation design in context-aware recommendation process.

In context-aware recommendation system research, context is always chosen arbitrarily by researchers. This, in turn, leads to the generation of weak CARS. Also, considering the fact that context is central to the recommendation system’s effectiveness and reliability, there is need to thoroughly and carefully select and experimentally validate any contextual variables to be used in CARS. Currently, several CARS recommendation algorithms [45, 46, 27] have been developed but very few research went back to discuss the means and importance of experimentally validating the context in the different domain of applications of CARS. In this paper, we have used statistical models to experimentally establish the feasibility and authenticity of the variables before going ahead to use them in CARS. We found out that there is variation in the citations of journal articles and proceeding articles. In the three contextual variables, introduction, review and methodology, journal articles have more citations in the different sections compare to proceeding articles. Also, for both journal and proceeding articles, the variations in the three contextual variables are highest in the review, then higher in the introduction and lowest in the methodology. With the experimental validation information, that is, we know the variation in citations of journals and proceedings at different sections in the body of the respective articles, and then the contextual variables could be incorporated into CARS through any of its techniques (pre-filtering, post-filtering, and contextual modeling).

Future Direction -In this paper, we have used statistical models to determine the feasibility and authenticity of the contextual variables (Introduction, review, and methodology) proposed to be used in context-aware research resources recommendation algorithm. We, however, hope that future works will gradually move to deeper levels to harness the strength of different machine learning algorithms to improve on our work. Also, considering the importance of context-aware recommender systems, we propose that more research should be geared toward developing standard algorithms that could automatically detect and extract contextual information that is relevant to different application scenario instead of randomly selecting these variables.

• [1]    D. Gavalas, C. Konstantopoulos, K. Mastakas, and G. Pantziou “Mobile recommender systems in tourism,” Journal of Network and Computer Applications , 2014, vol. 39, pp.319-33.

• [2]    F. O. Isinkaye, Y. O. Folajimi, and B. A. Ojokoh, “Recommendation systems: Principles, methods and evaluation,” Egyptian Informatics Journal, 2 015, vol.16 no. 3 pp. 261-273.

• [3]    F. M. Harper and J. A. Konstan, “The movielens datasets: History and context,” ACM Transactions on Interactive Intelligent Systems (TiiS) , 2016, vol. 5, no. 4, pp. 19.

• [4]    C. A. Gomez-Uribe and N. Hunt, “The netflix recommender system: Algorithms, business value, and innovation,” ACM Transactions on Management Information Systems (TMIS) , 2016, vol. 6, no. 4, pp. 13.

• [5]    D. Doychev, A. Lawlor, R. Rafter and B. Smyth, “An analysis of recommender algorithms for online news,” In CLEF 2014 Conference and Labs of the Evaluation Forum:   Information Access Evaluation Meets

  Multilinguality, Multimodality and Interaction, 15-18 September 2014, Sheffield, United Kingdom .

• [6]    R. Ren, L. Zhang, L. Cui, B. Deng, and Y Shi, “Personalized financial news recommendation algorithm based on ontology,” Procedia Computer Science , 2015, vol. 55 , pp. 843-851.

• [7]    M. O. Omisore and O. W. Samuel, “Personalized Recommender System for Digital Libraries,”

International Journal of Web-Based Learning and Teaching Technologies (IJWLTT) , 2014, vol. 9, no. 1, pp. 18-32.

• [8]    O. C. Santos, M. Saneiro, J. G. Boticario and M. C. Rodriguez-Sanchez, “Toward interactive context-aware affective educational recommendations in computer-assisted language learning,” New Review of Hypermedia and Multimedia , 2016, vol. 22, no. 1-2, pp.27-57.

• [9]    S. S. Li and E. Karahanna, “Online recommendation systems in a B2C E-commerce context: a review and future directions,” Journal of the Association for Information Systems , 2015, vol. 16, no. 2, pp. 72-107.

• [10]    W.Q Qwaider. “A personalized E-learning Portal D2L Recommender system,” Journal of Engineering and Applied Sciences, 2017, vol.12, no.8, 2084-2087.

• [11]    S. Feyer, S. Sieber, B. Gipp, A. Aizawa and J Beel, “Integration of the Scientific Recommender System Mr. DLib into the Reference Manager JabRef,” In European Conference on Information Retrieval ,April 2017, pp. 770-774.

• [12]    F. J. Cabrerizo, J. A. Morente-Molinera, I. J. Pérez, J. López-Gijón and E. Herrera-Viedma, “A decision support system to develop a quality management in academic digital libraries,” Information Sciences , 2015, 323 , 48-58.

• [13]    B. Amini, R. Ibrahim, M. S. Othman and M. A. Nematbakhsh, “A reference ontology  for profiling

scholar’s background knowledge in  recommender systems,” Expert Systems with Applications, 2015, vol. 42, no.2, pp. 913-928.

• [14]    H. Liu, X. Kong, X. Bai, W. Wang, T. M. Bekele and F. Xia, “Context-based collaborative filtering for citation recommendation,” IEEE Access , 2015, vol. 3 pp. 16951703.

• [15]    G. Nesrine, B. Naouar, B. Ahlame and Z. “Arslane,” Improving the Proactive Recommendation in Smart Home Environments: An Approach Based on Case Based Reasoning and BP-Neural Network,” International Journal of Intelligent Systems and Applications , 2015,vol. 7, vol. 7,pp. 29-35

• [16]    M. J. Barranco, J. M. Noguera, J. Castro, and L. A. Martínez, “Context-aware mobile recommender system based on location and trajectory," In Management intelligent systems , Springer Berlin Heidelberg, 2012, pp. 153-162.

• [17]    M. A.Domingues, M. G. Manzato, R. M. Marcacini, C.V. Sundermann, and S. O. Rezende, “Using contextual information from topic hierarchies to improve context-aware recommender system,”. In Pattern Recognition (ICPR), 2014 22nd International Conference, 2014, pp. 3606-3611.

• [18]    U. Panniello, A. Tuzhilin, M. Gorgoglione, C. Palmisano, and A. Pedone, “Experimental comparison of pre-vs. post-filtering approaches in context-aware recommender systems,” In Proceedings of the third ACM conference on Recommender systems , 2009, pp. 265-268.

• [19]    G. Adomavicius, A. Tuzhilin, “Context-aware recommender systems,” In Recommender systems handbook , Springer US, 2011, pp. 217-253.

• [20]    S. D. Seifu and S. Mogalla, “A Comprehensive Literature Survey of Context-Aware Recommender Systems,” International Journal of Advanced Research in Computer Science and Software Engineering , December 2016, vol.

• 6,    vol. 12. pp. 40-46

• [21]    B. Hidasi and D. and Tikk, “General factorization framework for context-aware recommendations,” Data Mining and Knowledge Discovery , 2016, vol. 30, no. 2, pp.342-371.

• [22]    T N.Nguyen and N. A. Te, “Towards context-aware recommendations: Strategies for exploiting multi-criteria communities,” In Collaborative Computing, Application and Worksharing (Collaborecom), 9th International Conference , October 2013, pp. 105-114..

• [23]    C. Pan and W. Li, “Research paper recommendation with topic analysis,” In Computer Design and Applications (ICCDA), 2010 International Conference , June 2010, vol. 4 pp. V4-264.

• [24]    B. Kim and J. Lee, “Improved Post-Filtering Method Using Context Compensation,” International Journal of Fuzzy Logic and Intelligent Systems, 2016, vol. 16, no. 2, pp. 119-12.

• [25]    J. Tang and J. Zhang, “A discriminative approach to topic-based citation recommendation,” In Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer Berlin Heidelberg. April 2009, pp. 572-579.

• [26]    V. C. Ostuni, T. Di Noia, R. Mirizzi, D. Romito and E.D. Sciascio, “Cinemappy: a context-aware mobile app for movie recommendations boosted by dbpedia,” In Proceedings of the 2012 International Conference on Semantic Technologies Meet Recommender Systems & Big Data, Nov. 2012, vol. 919, pp. 37-48.

• [27]    T. H. Soliman, S. A. Mohamed and A. A Sewisy, “Developing a mobile location-based collaborative Recommender System for GIS applications,” In Computer Engineering & Systems (ICCES), 2015 Tenth International Conference, Dec. 2015, pp. 267-273.

• [28]    S. A. E. M. Mohamed, T.H.A. Soliman and A. A. Sewisy, “A Context-Aware Recommender System for Personalized Places in Mobile Applications,” International Journal of Advanced Computer Science & Applications , 2016, vol. 1 no.7 pp. 442-448.

• [29]    R. Sharma, S. Vinayak and R. Singh, “Guide Me: A Research Work Area Recommender System,” International Journal of Intelligent Systems and Applications , 2016, vol. 8, no. 9, pp.30-37

• [30]    P. G. Campos, F. Díez, I. Cantador, “Time-aware recommender systems: a comprehensive survey and analysis of existing evaluation protocols,” User Modeling and User-Adapted Interaction. 2014, vol.24 no.(1-2), pp. 67-119.

• [31]    V. Codina, F. Ricci and L. Ceccaroni, “Distributional semantic pre-filtering in context-aware recommender systems,” User Modeling and User-Adapted Interaction , 2016, vol. 26, no. 1, pp. 1-32.

• [32]    G. Adomavicius, A. Tuzhilin, “Context-aware recommender systems,” In Recommender systems handbook, Springer US, 2015, pp. 191-226.

• [33]    K. Verbert, N. Manouselis, X. Ochoa, M. Wolpers, H. Drachsler, I. Bosnic and E. Duval, “Context-aware recommender systems for learning: a survey and future challenges,” IEEE Transactions on Learning Technologies , 2012, vol.5, no.4, pp.318-335.

• [34]    Z. D. Champiri, S. R. Shahamiri and S. S Salim, “A systematic review of scholar context-aware recommender systems,” Expert Systems with Applications , Feb. 2015, vol.42, no. 3, pp.1743-1758.

• [35]    C. Palmisano, A. Tuzhilin and M Gorgoglione, “Using context to improve predictive modeling of customers in personalization applications,” IEEE transactions on knowledge and data engineering , 2008, vol. 20,no. 11, pp.1535-49.

• [36]    K. Oku, S. Nakajima, J. Miyazaki and S. Uemura, “Context-aware SVM for context-dependent information recommendation,” In Mobile Data Management, 2006. MDM 2006. 7th International Conference, May 2006, pp.

109-109.

• [37]    M. Kompan and M. Bieliková, “Context-based satisfaction modelling for personalized recommendations,” In Semantic and Social Media Adaptation and Personalization (SMAP), 2013 8th International Workshop, Dec. 2013, pp. 33-38.

• [38]    M. Kaminskas and F. Ricci, “Location-adapted music recommendation using tags,” In International Conference on User Modeling, Adaptation, and Personalization , Springer Berlin Heidelberg, July 2011, pp. 183-194.

• [39]    H, S. Park, J. O. Yoo and S.B Cho, “A context-aware music recommendation system using fuzzy bayesian networks with utility theory,” In International Conference on Fuzzy Systems and Knowledge Discovery, Springer Berlin Heidelberg, Sept. 2006, pp. 970-979.

• [40]    Q. He, D. Kifer, J. Pei, P. Mitra and C. L. Giles, “Citation recommendation without author supervision,” In Proceedings of the fourth ACM international conference on Web search and data mining , Feb. 2011, pp. 755-764.

• [41]    W. Huang, Z. Wu, L. Chen, P Mitra and C. L. Giles, “A Neural Probabilistic Model for Context Based Citation Recommendation,” In AAAI , Jan. 2015, pp. 2404-2410.

• [42]    Y. Zheng, “A revisit to the identification of contexts in recommender systems,” In Proceedings of the 20th International Conference on Intelligent User Interfaces Companion , March 2015, pp. 133-136.

• [43]    H. Liu, H. Zhang, K. Hui and H. He, “Overview of context-aware recommender system research,” In 3rd International Conference on Mechatronics, Robotics and Automation (ICMRA 2015), 2015b, pp. 1218-1221.

• [44]    Y. H. Zhang and X. Jia, “Republication of conference papers in journals?,” Learned Publishing, 2013, vol. 26, no. 3, pp.189-196.

• [45]    Vrettas G, Sanderson M. Conferences versus journals in computer  science .  Journal of the Association for

  Information Science and Technology . 2015, 66(12):2674-2684.

• [46]    M. Jiang, P. Cui, R. Liu, Q. Yang, F. Wang, W. Zhu and S. Yang, “Social contextual recommendation,” In Proceedings of the 21st ACM international conference on Information and knowledge management, Oct. 2012, pp. 45-54.

• [47]    R.  Dias  and M. J. Fonseca,  “ Improving music

  recommendation in session-based collaborative filtering by using temporal context,” In Tools with Artificial Intelligence (ICTAI), 2013 IEEE 25th International Conference, Nov 4, 2013, pp. 783-788.

Nigeria (CPN)) and Association for Computing Machinery (ACM). She is currently a visiting Ph.D. scholar at the Laboratory for Knowledge Management, Politecnico di Bari, Italy.

Yetunde O. Folajimi is Research Fellow at Northeastern University, Boston, USA and a Senior Lecturer (currently on leave) at the Department of Computer Science University of Ibadan, Ibadan, Nigeria. She specializes on Artificial Intelligence and its application on recommender systems, serious games and intelligent tutoring. She has published over 30 articles in both reputable National and International Journals and cponferences. She is a Fellow of British Computer Society and she belongs to many international associations including Association for Computing Machinery (ACM), ACM SIG-AI, ACM SIG- CSE, Association for the Advancement of Computing in Education (AACE), Nigeria Computer Society, Computer Professionals Registration council of Nigeria.