Comparative Study of Supervised Algorithms for Prediction of Students’ Performance

It is essential for every educational organization to facilitate high quality education to their students. Performance of student in academic is major concern for every institute as it linked to job opportunities and reputation of institution. One of field related to dealing with processing and analyzing of all educational data is educational data mining (EDM). EDM develops methods to understand student and their environment of learning [1]. It also helps to predict patterns that can be helpful to improvement of student performance. Prediction of student’s academic performance is a difficult task because it depends on various demographic, socio-economic and past-academic factors. In this paper, the attributes responsible for affecting the academic performance of the student and the students’ grades for three different datasets are determined.

In literature, different data mining (DM) algorithms and machine learning (ML) algorithms have experimented for this problem. Machine learning algorithms as said ‘learn’ from given data, discover hidden patterns and provide predictions, which allow engineers, researchers and scientists to make a reliable decision. Machine learning is broadly divided as supervised, unsupervised, and reinforcement learning. In supervised learning is done using training data which is analyses and builds model to perform predictions for training set. Classes or target variable is labelled in this case. DM and ML techniques are widely applied in field of analytics and predictions. The research work in [2-8] make use of such algorithms which are Logistic Regression, J48, Decision Tree, Support Vector Machine (SVM), NB, Random Tree, ANN, K-Nearest Neighbour (KNN), MLP, and Random Forest. In few cases, other algorithms are also used such as association rules [9], and clustering [10]. NB Tree is used in [11] for predicting status of student, length of study and GPA. Techniques such as REP Tree, PART, Decision Table, Decision Stump, and JRip [12-13] are used for student performance prediction. Results show that the algorithms that perform well for predicting grades are Random Forest, J48, CART, NB, KNN and SVM. To the best of our knowledge, the experimentations are not conducted on C5.0, which is an advanced version of C4.5 (also called as J48) for predicting grades of students. Hence, these seven algorithms are used for grade prediction.

Attribute selection is critical task in every DM and ML algorithm. Performance of algorithm depends on type of data it consists. Adding and removing of certain attributes can also change the performance of algorithm. For educational researches, data can be demographic, academic, and behavioural data. Most of cases demographic and academic data is used for student performance prediction.

• •    Factors such as age, gender, annual income, parents’ occupation, parents’ education, are included [2, 4].

• •    Academic data such as subject marks, previous examination marks are included in [2, 14],

• •    Previous semester marks, participation in activities are included in [3, 12],

• •    GPA, subject marks and assignment marks are included in [11, 15]

• •    Exam scores, absences and attendance is used in [2, 7].

• •    Consideration of behavioural data is done in dataset of Kalboard 360 available on Kaggle used in [5, 7].

• •    Comment based data is used in [16] for every lesson taught and various attributes are retrieved using text

classification on those comments and [17] comprehends questionnaire based input variables.

In the literature, many machine learning algorithms tested for the problem. In most of cases in related work, the input parameters that are related target are not identified, such as in [2-3, 5]. Also, it is found out that the algorithms those are applied in previous work [7-8, 28, 40] are processed without fine tuning which doesn’t pushes limits to know how far an algorithm can accurately predict results.

The research paper presents application machine learning on three different datasets. Performance of algorithm is analyzed by changing values of parameters. Results of decision tree are compared with other algorithms in literature.

The major research objectives of this paper are,

• 1 . To find attributes having more influence on target variable using correlation. By using correlation coefficient it is expected that the attributes that are closely related to grades i.e. those who impact on academic performance most are to be identified.

• 2 . To apply C5.0, J48, NB, Random forest, KNN, SVM, and CART. Using comparison of results of these seven algorithms we determine which of the algorithms can accurately predict the results when fine tuning is applied.

• 3 . To study effect of tuning parameters on accuracy of classifiers.

2.    Related Work

Evaluation of these results are done using various measures such as precision, recall, True Positive Rate (TPR) and False Positive Rate (FPR).

Sections below are divided into 6 parts. Literature review on student performance prediction is described in section II. Section III contains problem formulation for current work and previous work. Methodologies used are mentioned in section IV. Results are discussed in section V and section VI explains conclusion.

Research for predicting students’ academic performance has been done for various kinds of datasets and using numerous methods. Datasets can be of type e-learning, university data, college data and the variety of methods applied are statistical, data mining techniques and machine learning algorithms.

A review of the various algorithms used and their accuracies obtained to solve student performance prediction presented below. In [3] algorithms such as Logistic Regression, SVM, Decision Tree, NB, NN, and KNN are applied. Experimentation is conducted by considering all input variables and attributes filtered through feature selection. Performance of KNN is found out to be better when all attributes are considered. SVM and Logistic Regression perform well for dataset with feature selection process. Conversely in the work mentioned in [18] has that Logistic Regression performs poor along with NB, but KNN has better accuracy results. The dataset in [3] and [18] differs in terms of number of records as well as [3] has previous academic data and [18] has course grades along with psychometric factors. In [19] using e-learning data and behavioural survey of student using such e-learning platform Logistic Regression model is applied to predict the failure of student in a particular course. Dataset considered is presumably small on which accuracy obtained is 73.7%. Logistic Regression, NN and Random Forest are used in [20]. All of these techniques provide low level of correctness in results, such weakness is overcome by inclusion of uncertain classes. Similar to [19], work in [21] follows same algorithm with almost same amount of records having accuracies of 78.6 % and 78.8%. In addition with algorithms used in [3], the work proposed in [22] has BayesNet and SMO where prediction of failure of a student in particular course is performed. Experimentations are conducted by taking into consideration dataset with and without filtering, discretization and rebalancing. Without filters performance of algorithms is lower than expected. With applying filters, most of algorithms has enhancement in their performance. Decision Trees, Random Tree and Random Forest have been giving best potential. In [23] the 3rd semester performance is carried out using Decision Tree and Random Tree. Results show that RT achieves 94.4% accuracy followed by J48 with 88.37%. Predictions are achieved high in [12], their work also convey that data mining techniques are not limited by size of datasets. [24] has multiclass classification performed by using algorithms such as RF, DT, SVM, NB, Boosting Trees and Bagging Trees. Above 2000 student records are considered for the prediction process. Their work focuses on obtaining results for degree level performance of students. The Random Forest achieves an accuracy of 96.17% which is best among the other algorithms implemented. In cases [2, 5, 24], Random Forest, J48, and NB tend to predict the results more accurately. For SVM, the results achieved are remarkable, as it has maximum F1-measure value after DT [4]. SVM and KNN both are to be found suitable for student academic predictions as mentioned in [6]. The algorithms CART and C5.0 are rarely used as per the best of our knowledge. Literature survey conveys that trees and Random Forests perform best for classification of grades of students.

The following literature discusses different kinds of outcomes predicted using ML algorithms. Most of the cases Grade Point Average (GPA) or cumulative GPA is predicted. The outcome can be either binary class such as pass or fail in particular subject or semester, successful or unsuccessful to complete graduation or degree otherwise target attribute can be multiclass such as in [8] end semester percentage is converted to five classes which are best, very good, good, pass and fail. The research work in [3, 4, 19, 21, 22] focus on acquiring predictions for a particular course. Classification performed is binary class, student will pass in the course or get failed is predicted. Whereas, [18] predict that whether student performs poor in academics or is a strong achiever, based on GPA values. Final GPA are determined in [15] by using demographic data, high school information, and family financial status. The research work in [25] predicts that whether a student will obtain his engineering degree or not using student academic data and background information and same with the case of [20]. Unlike these research papers the work mentioned in [26] predicts whether a student will obtain excellent grade, good grades, get passed or just get failed in a course with grades predicted to be from scale of 0 to 10 for final exam of a course. Similar to this, in [23] it is predicted either student will get 3rd semester performance as below average, average, above average or excellent. In [12] predicts the score of a course of students to be low, high or medium.

Impact of different attributes on the performance of student performance is reviewed and presented. Most of the research works have included correlation methods to find out the influencing factors. Models those are trained on students of less age provided good results in [18]. The type of registration to University and income of student’s family are found to be correlated achievement of student. The four major attributes that highly affect the performance of student found in [25] are First Year University GPA, CC BP transfer credit hours, first fall credits GPA and CC BP transfer GPA. Here all the attributes related to academic and student’s background were considered. The work in [21] where e-learning data is taken for process, the significant predictors are found out to be date of first login to LMS, mode of study, previous academic performance record, and weighted average marks. For the results in [23], where 3rd semester performance is predicted, it is revealed that 2nd semester results, leadership and drive qualities correlate a lot with output variable. For [27], the ability to understand and handle basic subjects influence a lot on final result of degree. In [28], experimentations are conducted using dataset from UCI machine learning repository. On comparison with target variable it is found out that weekday alcohol consumption, romantic relationship and parents’ education do affect student’s performance. GPA, Participation rule, Test average, Lab test average, Assignment submit attendance, Final grade are considered as best attributes in [29] for undergraduate student data. In [30] where techniques such as Cluster Analysis and Association Rule Mining were used found a pattern that frequent occurrence of seven courses {MTH 111, STA 122, MTH 122, MTH 121, CSC 111, BIO 111, CSC 121} in failed students’ data. These courses are found crucial for academic performance of a student. Survey presented in [31] for student performance concludes that CGPA and internal marks are important attributes. Conversely, evaluations done in [32] mention that grades do not necessarily affect outcome achievement and direct assessment has positive impact on student performance.

Table 1. Literature review