Formalizing logic based rules for skills classification and recommendation of learning materials

The problem of students’ skills evaluation whether in formative or summative learning systems is not uncommon in literature. But what is not common is the diagnosis and classification of skills of open-ended answer entries into a system for the purpose of assessment and recommendation of materials requisite to the level of students. Several systems have been designed over time to assist in learning, teaching and assessments (LTA). Whilst some LTA support career path recommendation after assessments e.g. [1], others categorise students for learning via skills assessment that are based on multiple choice question technique or item rating e.g. [2, 1]. Unlike existing systems that employs rating items or multiple-choice test techniques, the Preassessment System of this study that is agent based, takes as inputs open-ended answer entries and classifies students’ skills for learning materials. As with natural reasoning in which humans engages the use of acquired facts to make decisions; logic based assertions, precisely, first order logic (FOL) are strong and reliable knowledge base representation models for the act of reasoning in intelligent systems [3]. However, the main challenges according to [3] in their use of FOL model in context-aware application are: how to model, represent and classify context, and then how to reason about it.

This paper is a description of a system of FOL based rules for the diagnosis and classification of skills, and the recommendation of learning path. The paper supports the application of the educational principle of chunking [4, 5] in the diagnosis of students’ skills for learning material recommendation in the learning of SQL: a subject that has been described as difficult. The structure of this paper continues with related work in Section II, and Section III SQL learning, teaching and assessment systems. Section IV presents the hypothesis of our logic based classification model, and the strategies for supporting students to successful learning. In Section V is the implementation of the logic based rules and the data gathered so far, and Section VI conclusions and further work.

• II.    Related Works

Classification is feature, instance or attribute learning. It is when features (inputs or training set) that are symbolised in a system have corresponding class labels (i.e. outputs) to predict. These features can be continuous, categorical or boolean [6]. Classification consists of taking input vectors or data and deciding which N classes they belong after running them through a classifier(s) [7, 8]. Classification, involves one form of reasoning over some tasks. Reference [9] describes classification as a data mining technique that maps data input into predefined groups or classes. That the technique is a supervised learning approach which requires a set of training data to generate rules for classifying the test (unknown) data into predetermined groups or classes [9,10]. There are many models to classification, but choices are guided by the kind of data or problem at hand. For instance, in [9], the ID3/C4.5 technique was demonstrated in the classification of students’ grades or scores into a pass or fail nodes in a decision tree on the platform of Weka. In [11] a Neural Net (NN) model was developed for CPGA prediction that could support students to plan their further semester study. The NN model has two tasks: the prediction task that predicts student performances and a classification task that classifies students into groups. Case based reasoning (CBR) is also a type of classification technique that was combined with multi-agents in [12] to gather information about students and categorises them based on their knowledge level and learning preferences. CBR is a method in which concrete previous experience is applied to solve current and similar problem situations.

In contrast to CBR approaches where a current problem is interpreted as a previous one based on similarities or differences (classification CBR), or where a new solution is adapted based on the past, stored or existing solutions (problem CBR) [13]; the approach taken in this work is a model of FOL rule-based approach to reasoning by a classifier agent. This is where domain specific rules are specified as antecedents for a body of conclusions that is applied in a classification process [14, 7, 8]. This is because, we believe that the rule-based approach is more decisive to address the errors that are liable to be made by students in their responses to questions from the system that will in the end make accurate recommendation for their learning gap. In addition, because the answer inputs to the system are open ended, as such answers submitted by students to the pre-assessment system may also not be similar. While most classification systems are the CBR, decision trees and support vector machine; this paper considers an agent based classifier in students’ learning. The act of classification in the study was not about the grouping of nodes in an ontology tree. But the collection of information about the skills status of students and recommendation of appropriate or a set of appropriate learning materials based on the available skills and related information to the system. The decision process in which students are categorised is through conditionaction rules.

• A. Condition-Action Rule

In a classification system, decision rules are the fundamental knowledge that are compared and matched with available information or known facts, and subsequently utilised by the system to perform the act of classification or conclusions. Rules of this nature have two component parts: the left-hand side known as the antecedent, condition, premise or situation, and the righthand side part referred to as the consequent, action, conclusions, response, or prediction e.g. [14]. This is the logical structure of a rule based system where a classification system is given a reasoning task about some available knowledge or concepts in order to draw conclusions about some incoming data. In [15] such methods can be used for learning concepts : In AI (artificial intelligence), a concept is treated as a formal definition or predicate. For most of these systems to work, [15] states that in a learning system the following assumptions (that we have elaborated) are valid:

• •    Conditions which are basic predicates for testing a state must be specified in advance: This is preparing rules that must be satisfied as preconditions for the system or a component of the system.

• •    The predicates are the essential part of the language or formalism for task representation: All the variables in the environment should be gathered for adequate representation in the system.

• •    There must be something―set of rules―to learn: For a system to make decisions, a set of rules must be specified according to the environment and variables in the problem.

• •    The training set is clean or devoid of noisy relations: In that case, the data used for preparing the rules for the system must be unambiguous to be suitable to match the incoming unknown data or information.

• •    The training set should contain counter-examples: All examples (or facts) that may be available to a system may not be similar. Some may be positive and others negative. Rules should be stated to cover both positive and negative facts.

• •    Basic predicates can be partitioned into independent group: Different variables that are related can be grouped in one rule.

• •    Within each group, the predicates are mutually exclusive and cover all cases: No case of classification must be missed. Otherwise, this would result in the misclassification of an object.

A rule based system have IF THEN rules, where the set of are needed to be matched and satisfied before the part is triggered. In a system that supports teaching and learning, performance observation are therefore the prelude to determine learning path .

Over some given tasks in learning, recommender systems for formative assessment needs to have a suitable collection of predictable performances of student skills in order to suggest accurate learning path. With the boolean logic passed(N) or failed(N) predicates where N represents the decisions made on any leafnode after preassessment, our agent based system adapts to the changing knowledge levels of students in a specific domain of SQL using FOL rules.

A database is a repository of information organised in such a way that it can be accessed, managed and updated easily. A database is created, stored and maintained on a database management system (DBMS). SQL (Structured Query Language) is the dominant database language [16]. In [17] SQL is a formal declarative database programming language that comprise data manipulation keywords such as select, from, where, delete, insert, into, update, set, on, and join to mention a few. The skills in

SQL are challenging and students have many difficulties learning them [18].

In the perspective of [19] learning and mastering of these skills is a difficult process that requires considerable practice and effort on the part of students. One of the challenges faced by students is mapping a statement of problem given in natural language into the information that is required from the database in an appropriate SQL statement; this [19] further stated is not easy. Another difficulty is students’ misunderstanding of the basic elements of SQL and first order logic and the relational data model in general [20]. Recent studies have also shown the factors affecting students’ failure in programming. As one of the factors, [4] in their descriptive study identified inadequacy in the time dedicated to programming courses, that this consequently heralds to students’ inability to match the degree of difficulty required in programming skills. Other factors attributed are lack of higher order thinking on the part of students and incomplete or inappropriate teaching processes [21], as well as the problem of selecting the correct teaching method or tool [22, 4].

To support students with the learning of SQL and determine individual students’ SQL query formulation skills, systems such as the AssesSQL test software [19, 23] have been developed. Their research examined the difficulty faced in the assessment of students’ SQL query skills, and encourage students to use structured query language as software professions. For assessment, the system present questions to student, expects students to enter query solution to the questions. The AssesSQL query content covers only the SELECT statements.

In the LEARN-SQL tool, [16] implemented a strategy that objectively allows the evaluation of the correctness of the solution to a question given by a student by providing automatic correction to queries by comparing the students’ solution to all existing valid solutions in the system. The system, tests, feedback and grade students in their learning of SQL. The LEARN-SQL was developed and comprised statements such as the SELECT and UPDATE queries. This is from the backdrop of previously development SQL systems whose content only covered the SELECT statements [16]. There also exists a number of sites that provide tutorials to students on SQL learning. Examples are "w3schools.com/sql" [24], "Beginner SQL Tutorial" [25] and "SQLCourse.com" [26] that have lots of modules from which a student would make a choice in order to start learning; and the "SQLzoo.net" [27] that provide supports through multiple choice (objective type) quizzes. While these systems provide ability for students to run queries or take quizzes, they do not provide assistance for errors or incorrect queries. Besides, for students to learn some higher level skill, relative prerequisite ought to have learned. While [21] anchored their prerequisites on general problem solving skills and logical thinking, our prerequisite is based on previously learned knowledge or lower level concepts to a higher concept in the same subject of SQL. Thus our agent based Pre-assessment System, preassesses students, feedback to them, keep students’ SQL queries to questions, timestamp every activity, and finally make recommendation for learning materials. The system carries out diagnosis on the learning gap of in students’ learning between what wants to learn (called the desired_Concept D) and some prior learning (called prerequisites C). The SQL query modules covered in the system are SELECT, INSERT, DELETE, UPDATE, JOIN and Union.

• A.    Recommender Systems for E-Learning

From the preceding section, SQL is a difficulty subject that poses challenges to students’ learning. Thus one perceived approach to tackling this difficulty is through recommendation for learning materials after some prior skills test and assessment by a formative assessment system. In that regards, there are already existing intelligent tutoring systems (ITS) that provides support for a given level of adaptability. Reference [28] states that such system must be able to present materials at a level of difficulty and detail suited to the state of knowledge of the student, and to do so, the system must know and follow the student’s changing knowledge. This can be achieved by a set of carefully planned rules [15] where a set of outputs are provided for some given set of inputs. This amounts to integrating supervised classification technique into ITS development, aim at making accurate class predictions that suits an individual student’s need and level of knowledge.

Recommender systems for adaptive learning propose and prescribe content and items that centres around the learning needs of students. This is quite different from recommender systems for buying products; this is because learning is an effort intensive task that requires more time and interaction on the part of students compared to commercial transactions [29]. Furthermore, learners rarely achieve a final end state based on the fact that there are levels in learning. Instead of buying a product and owning it, learners achieve different levels of competences that have various levels in different domains. In such situation, what is important is identifying the relevant learning goals and supporting learners to achieving them (p.6).

In the views of [30] adaptive or personalised learning tends to model learners' learning path, activities and educational resource. To this end, several e-learning recommender systems have been proposed. In [30] for instance, a standalone quasi-summative assessment model was proposed to boost instruction process and customisation of learning path. In the model, students are graded based on some learning activities using a model of equation, and the adaption on the students’ preferences and effort spent on course. Should a learner fail an activity, it means the competence needed has not been completely acquired; and this could hinder further learning.