The Impact of E-assessments System on the Success of the Implementation Process

Published Online December 2012 in MECS

The revolution of the Internet and Telecommunication (ITC) Technologies have had very clear impacts on all aspects of our lives. One of these aspects is Learning. Recently, we have witnessed the utilization of the ITC technologies in the processes of teaching and learning which are taking place in many universities across the Globe. E-Learning is becoming one of the learning standards in many western universities, which is spanning all learning activities [1-5]. These days, many universities in the Middle East and particularly in Jordan are following the same footsteps of western universities in adapting the processes of e-learning.

The importance of this research comes from the fact that the management of students evaluation and assessment is one of most vital operations in any academic setting. This issue is already taking a lot of concerns in the public and private universities. One of the main reasons is the increase in the number of students attending these universities, which is also affecting the number of students enrolled in classes. Students assessment and evaluation is becoming extremely difficult using conventional methods such as the use of papers, which is also time consuming [6]. Moreover, the process of preparing such assessments is becoming very costly in terms of human resources, time, papers, photocopying and ink, etc. Probably, one of the most concerns in the process of student evaluation is the procuring of subjectivity and fairness, and minimizing human errors.

The availability of e-assessments via e-learning systems such as Blackboard, WebCT, and Moodle allowed universities to overcome these problems. In matter of fact, the utilization of e-learning systems have widely opened the doors to many benefits [7-8]. Moreover, the adaptation of e-assessments by universities should enhance and promote their reputation among each others. In this research, an attempt to explore the effect of providing the necessities for carrying out e-assessments successfully at the university. The college of IT at Isra University has adapted the e-learning system and electronic assessment in many of the courses for several years [9-10]. Recently, it has been spread to other faculties which they use mainly electronic assessments. A sample of 246 students from 5 colleges has been used in the research.

Section two of the paper presents the research problem; research aims, methodology used, research sample, the tools and variables, and then the reliability of the tools. In section three the collected data is statistically analyzed and the results are discussed. In Section four an elaboration on testing the hypotheses is presented, and finally the conclusion is shown in section five.

• II.    RESEARCH PROBLEM

  The adaption of e-assessments is considered a good indication on promoting student evaluation practices [1113]. Many public and private universities in Jordan have applied e-assessments systems many years ago, and Isra University is one of them.

Isra University has adapted the Moodle (Modular Object Oriented Dynamic Learning Environment) elearning system. Moodle is an open source course management system that supports a modern social constructionist pedagogy [6]. During the preliminary investigation, students and system administrators were interviewed to learn about the benefits and the flaws using the e-learning system.

Despite the success of adapting e-assessments, it has been noticed that there are some problems and obstacles that have faced all people who are involved in this process. These issues need to be explored and analyzed in order to come up with the right proposals and recommendations.

• A.    Research Hypotheses

The researchers have adapted two main research assumptions, which are as follows:

• 1.    Based on the opinion of the students sample, there are no effective relationship of statistical indication at (a≥0.05) between the extent of availability of the requirements for applying the e-assessments system and the process of applying the system. The requirements are computers, programs, computer networks, administrators with expertise, skills and knowledge of preparing such assessments.

• 2.    There are no effective differences of statistical indication at (a≥0.05) in the attitudes of the sample students in terms of the following variables: the availability of computers, programs, computer networks, administrators with expertise, skills and knowledge of preparing such assessments, signs of success in the process of applying electronic assessments, and the problems that are faced by the students and the administrators when working on such systems.

• B.    Research Aims

• 1.    Presenting a theoretical framework about the notion and importance of applying e-assessments in academic

• 2.    Measuring the effect on the availability of the following:

settings. The research is focused on the academic settings at Isra University in particular and at other Jordanian and regional universities in general.

• a.    System Infrastructure: Computers, programs, and computer networks.

• b.    People: Administrators and others who are responsible for the preparation and carrying out electronic assessments.

• c.    Computer Skills: The ability to take e-assessments successfully and follow ups.

• 3.    Measuring the level of differences in the attitudes of the sample students in the following variables:

• a.    The availability of computers, programs, computer networks, administrators with expertise, skills and knowledge of preparing such assessments.

• b.    Signs of success in the process of applying electronic assessments.

• c.    The problems that may be faced by the students and the administrators when working on such systems.

• C.    Research Methodology

The research have adapted a field-base, descriptive, analytical methodology relies on questionnaires for collecting data. Data is processed using statistical analysis to determine the accuracy of the hypothesis. Interviews with the systems administrators were also conducted by the researchers during the process of preparing the questionnaires. Lecture reviews have been conducted during the process of constructing the theoretical framework and also to learn more about other similar studies which was one of the main resources for gaining important information to be used in this research.

• D.    Research Sample

This research was conducted on a random sample of students from different colleges which all of them were subjected to electronic assessments, the faculties were: the college of Financial and Administrative Sciences, the college of IT, the college of Nursing, the college of Education, and the college of Literature, as shown in table 1.

As for the students, a random sample fitting with size has been used, where the questionnaire has been handed out to those who used the e-assessment.

Table 2 shows the personal attributes of the individuals involved in this research sample.

Table 1 DISTRIBUTION OF THE RESEARCH SAMPLE

College Name	Actual Number of Students	Sample Size	Number of Distributed questionnaires	Number of returned valid for analysis
Administrative Sciences	650	72	62	62
IT	400	38	38	38
Nursing	160	23	23	23
Education	900	86	86	82
Literature	500	48	42	42
Totals	2610	257	257	246

Table 2 PERSONAL ATTRIBUTES OF STUDENTS IN THE RESEARCH SAMPLE

N	College Name	Gender				Age						Academic Year
		Male		Female		18 - 21		22 - 25		26 and more		Year 1		Year 2		Year 3		Year 4		Year 5
		Q	%	Q	%	Q	%	Q	%	Q	%	Q	%	Q	%	Q	%	Q	%	Q	%
1	IT	22	57.9	16	42.1	21	55.3	14	36.8	3	7.9	-	-	9	23.7	18	47.4	9	23.7	2	5.3
2	Education	25	59.05	17	40.5	26	61	11	26.2	5	11.9	22	52.4	11	26.2	6	14.3	3	7.1	-	-
3	Literature	55	67	27	32.9	56	68.3	17	20.7	9	11	55	67	10	12.2	7	8.5	9	11	-	-
4	Nursing	8	34.8	15	65.2	8	34.8	8	34.8	7	30.4	3	13.3	2	8.7	6	24.1	12	52.2	-	-
5	Administrative Sciences	45	73.8	16	26.2	40	65.6	13	21.3	8	13.1	27	44.3	9	14.8	11	18	14	23	-	-

• E.    Research Tool and Variables

The variables that have been adapted by the researchers for the students questionnaires are as follows:

• 1.    Dependent Variables:    Variables for the

• 2.    Independent Variables: Indicators for applying the system successfully, variable for problems facing students when taking electronic assessments.

requirements, these are: Infrastructure, Administration Personnel, Computer Skills needed for the preparation of electronic assessments.

• F.    The Reliability of the Research Tool

We should note that the questionnaires used in this research were reviewed by experts in the field and their notes were taken into consideration and were implemented. Some paragraphs in the questionnaires were edited and other paragraphs were rephrased to insure the balancing of the content of all paragraphs.

The coefficients of reliability (Cranach’s Alpha) has been used in the Students questionnaires to measure the internal consistency for all variables in the research sample as shown in table 3.

We should note that the coefficients of reliability for the research variables in the students questionnaire are all high enough to carry out this research.

TABLE 3 THE COEFFICIENTS OF RELIABILITY FOR THE RESEARCH VARIABLES IN THE STUDENTS QUESTIONNAIRE

Paragraph Number in the Questionnaire	Name of Variables	Coefficient of Reliability
(10-6)	Infrastructure variable	%57,8
(16-11)	Systems Administration variable	%80.4
(22-17)	Computer Skills variable	%83
(29-23)	Indicator of Success variable	%83.6
(30-36)	Problems facing students variable	%80.8

• III.    STATISTICAL ANALYSIS AND DISCUSION OF RESEARCH RESULTS

In order to analyze the research variables in the students questionnaire and to test the validity of the hypotheses, the Statistical package (SPSS) has been used to extract the arithmetic means and standard deviation for each of the sample questionnaire and research variables in addition to the use of the coefficients of Cranach's Alpha to measure the reliability of the questionnaire, and to perform multiple regression analysis and sequential testing of the validity of the research model and to measure the impact of independent variables on the dependent variable and to identify any independent variables with the most impact on the dependent variables in addition to the use of (X2) to measure the variation in trends of the research variables.

• A.    Analysis of the students answers to the system infrastructure variable

Results shown in table 4 indicate that there is general agreement on all paragraphs of this variable by the research sample, that is all students in all colleges combined. They all agreed that the infrastructure of networks, computer hardware and software necessary to implement the e-assessments system are available. The overall arithmetic mean of their answers is 3.82%, which is higher than the arithmetic mean of measurement tool of 3 with standard deviation of 0.76. These answers came to confirm their consent on their colleges that they all provide the necessary computer hardware, sophisticated database which provides secured questions assessments, the presence of sophisticated software to implement the system, and provide laboratories equipped with the latest devices and advanced servers to carry out e-assessments successfully.

• B.    Analysis of the students answers to the availability of specialized personnel to administer the system variable

12

4.24

0.86

4.04

1.1

4.09

1.04

3.47

1.44

4.42

0.82

4.13

1.04

13

4.21

1.04

4.21

0.89

3.96

1.09

3.86

1.45

4.32

0.81

4.12

1.03

14

4.13

0.96

4.23

0.79

3.95

1.01

3.69

1.32

4.32

0.9

4.09

0.99

15

4.05

0.92

3.42

1.32

3.74

1.22

3.43

1.4

3.83

1.17

3.73

1.21

Arithmetic mean S.D.

4.07

0.58

3.95

0.62

3.91

1.8

3.57

0.08

4.12

0.53

3.96

0.72

Results shown in table 5 indicates that there is general agreement on all paragraphs of this variable by all members of the research sample on the provision of professional staff responsible for the preparation and implementation of the e-assessments system. The overall arithmetic mean of their answers was 3.96, which is higher than the average measurement tool standard deviation of 0.72. This means that all students who took the questionnaire agreed that those responsible for the preparation and implementation of the e-assessments system are highly skilled and they do provide help and assistance to students during the assessments periods.

TABLE 4 THE ARITHMETIC MEAN AND STANDARD DEVIATION FOR STUDENTS ANSWER ON THE INFRASTRUCTURE VARIABLE

Number of Paragraph	IT		Education		Literature		Nursing		Administrative Sciences		Mean and SD for all Faculties
	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.
5	3.84	1.15	3.67	1.24	4.03	0.96	3.60	1.15	4.06	0.96	3.92	0.06
6	4.36	0.78	4.19	0.86	4.07	0.93	3.65	1.15	4.24	0.69	4.14	0.88
7	3.89	0.98	3.88	1.04	3.76	1.04	3.43	1.37	3.86	0.88	3.80	1.03
8	3.65	1.09	3.57	1.15	3.75	1.0	3.43	1.37	3.75	1.07	3.67	1.09
9	3.76	1.14	3.11	1.4	3.69	1.12	3.60	1.43	3.52	1.17	3.55	1.23
Arithmetic mean S.D.	3.9	0.71	3.7	0.78	3.86	0.74	3.54	1.08	3.89	0.63	3.82	0.76

TABLE 5 THE ARITHMETIC MEAN AND STANDARD DEVIATION FOR STUDENTS ANSWER ON THE SYSTEM ADMINISTRATION VARIABLE

Number of Paragraph	IT		Education		Literature		Nursing		Administrative Sciences		Mean and SD for all Faculties
	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.
10	4.05	0.69	3.9	0.82	3.87	1.03	3.65	1.15	3.96	0.75	3.93	0.89
11	3.71	0.86	3.9	0.79	3.87	0.92	3.3	1.35	3.86	0.78	3.8	0.92

• C.    Analysis of the students answers for the variable of computer skills

Results shown in table 6 indicate that there is a general agreement on all paragraphs of this variable by students of all colleges combined, to provide the skills and capabilities of students to deal with computers and take eassessments, where the arithmetic mean for their answers is 3.75, the highest average measurement tool of 3 with standard deviation of 0.73.

This means that students have the ability and skills to deal with the computers and have the ability to perform assessments electronically. As for phrases 19 and 20, their results indicate disagreement, where arithmetic mean of less than average measurement tool, which means that students of all colleges agreed that they are not subjected to pre-assessments pilot before taking the real assessments, also the university never gave seminars to educate and rehabilitate students before.

• D.    Analysis of the students answers for the variable of the success on the implementation of the system

The results described in table 7 indicate that there is a general agreement by the students that the implementation of the e-assessments system has succeeded through several indicators and this agreement included all students in all colleges combined with arithmetic mean of 3.83, and higher than the measurement tool of 3 with standard deviation of 0.86. All members of the research sample agreed that the eassessments achieved the objectivity, allowed the students to easily perform the e-assessments, get the results instantly, in less time and efforts compared to traditional assessments. But noted that the students from the college of Nursing did not agree with the paragraph 23 which states that the e-assessments gave the students an opportunity to re-take the assessment in the case of absence.

• E.    Analysis of the students answers for the variable of system problems

The results shown in table 8 indicate that there is a general agreement by all members of the research sample of all colleges combined, that there are some problems faced the students when taking e-assessments. The arithmetic mean for all answers is 3.39, which is higher than the average measuring instrument tool with a standard deviation of 0.88.

TABLE 6 THE ARITHMETIC MEAN AND STANDARD DEVIATION FOR STUDENTS ANSWER ON THE COMPUTER SKILLS VARIABLE

Number of Paragraph	IT		Education		Literature		Nursing		Administrative Sciences		Mean and SD for all Faculties
	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.
16	4.76	0.85	4.28	0.8	4.29	0.97	3.69	1.29	4.45	0.67	4.34	0.89
17	4.92	0.27	4.3	0.86	4.04	1.05	3.56	1.44	4.32	0.9	4.25	1.01
18	3.65	1.45	3.02	1.24	3.2	1.31	3.08	1.27	3.29	1.42	3.25	1.3
19	2.84	1.46	2.73	1.28	2.86	1.36	2.34	1.19	2.81	1.5	2.78	1.38
20	2.71	1.25	2.73	1.26	2.86	1.22	2	1.12	2.55	1.33	2.66	1.26
21	4.6	0.67	4.41	1.02	3.93	1.16	3.39	1.4	4.32	0.76	4.12	1.06
Arithmetic mean S.D.	3.91	0.52	3.53	0.67	3.53	0.77	3.01	0.97	3.63	0.6	3.57	0.73

TABLE 7 THE ARITHMETIC MEAN AND STANDARD DEVIATION FOR STUDENTS ANSWER ON THE SUCCESS OF SYSTEM IMPLEMENTATION VARIABLE

Number of Paragraph	IT		Education		Literature		Nursing		Administrative Sciences		Mean and SD for all Faculties
	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.
22	3.65	1.23	3.9	1.28	3.78	1.12	2.95	1.6	4.04	1.21	3.77	1.26
23	3.52	1.46	3.66	1.4	3.7	1.1	2.82	1.52	4.11	1.08	3.69	1.29
24	4.5	0.88	4.07	1.15	4.03	1.02	3.91	10.34	4.5	0.86	4.61	1.03
25	4.15	4.09	4.09	1.14	3.81	1.19	3.13	1.57	4.47	0.94	4.01	1.22
26	4.02	1.1	4.26	1.01	3.91	1.03	3.60	1.15	4.39	0.75	4.08	1.01
27	3.76	1.21	3.9	1.07	3.6	1.25	3.08	1.23	4.13	0.99	3.67	1.18
28	3	1.61	3.4	1.39	3.31	1.39	2.69	1.36	3.73	1.32	3.32	1.43
Arithmetic mean S.D.	3.75	0.77	3.9	0.87	3.74	0.81	3.17	1.11	4.2	0.7	3.83	0.86

TABLE 8 THE ARITHMETIC MEAN AND STANDARD DEVIATION FOR STUDENTS ANSWER ON THE SYSTEM PROBLEMS OF SYSTEM VARIABLE

Number of Paragraph	IT		Education		Literature		Nursing		Administrative Sciences		Mean and S.D. for all Faculties
	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.	Arithmetic mean	S.D.
29	3.52	1.33	3.28	1.4	3.76	1.22	2.95	1.33	3.19	1.48	3.43	1.38
30	2.78	1.33	3.07	1.29	3.4	1.19	3.04	1.14	3.06	1.35	3.13	1.27
31	3.57	1.3	3.19	1.21	3.35	1.25	1.13	1.39	2.98	1.21	3.24	1.26
32	3.02	1.36	3.28	1.4	3.67	1.19	2.47	1.2	3.4	1.34	3.33	1.35
33	3.81	1.44	3.73	1.26	3.87	1.15	3.21	1.56	4.04	1.18	3.28	1.28
34	3.23	1.6	3.09	1.26	3.56	1.2	2.86	1.39	3.49	1.37	3.34	1.35
35	3.42	1.17	3.23	1.07	3.62	1.21	3.04	1.22	3.34	1.35	3.46	1.16
Arithmetic mean S.D.	3.34	0.98	3.27	0.93	3.6	0.82	2.96	0.8	3.4	0.83	3.39	0.88

• IV.    Testing Hypotheses

• A.    Testing the First Hypotheses

The results of the regression analysis shown in table 9 indicate that the value level of significance (F) was equal to 0.000 and is less than 0.05, the moral value. It also indicate that the independent variable was able to explain 0.312 only from the approved variation in the variable level of the successful implementation of the eassessments system. this means rejecting the hypothesis nihilism which states that "There is no effect relationship is statistically significant at the level of (0.05 ≥ a) between the availability of the requirements when applying the e-assessments system and the success of the implementation process” and accept the alternative hypothesis which states that there is a relationship.

For the purpose of knowing which variables has the most influence, Regression Analysis has been used in testing the best of the hypothesis which measures the influential relationship for each independent variable with the existence of other independent variables. The results of the analysis given in table 10 indicate that the variable for providing professional staff was excluded from the model, and the other two variables about the infrastructure and computer skills were both remained.

As the value of the correlation coefficient 0.477 and the value of the regression coefficient 0.228 which means that the independent variables have an effect as the values of level of significance for the two variables are 0.026 and 0.000 where both are less than 0.05. This indicating the rejection of both secondary hypotheses that state there is no relationship effect between each of the Infrastructure variable and Skills variable, and accepting the alternative hypothesis which states that there is a relationship between them and the indicators of success of the implementation process and accept the premise subsidiary of no effect relationship between the availability of professional staff and the success of the implementation process of being left out of the model.

• B.    Testing the Second Hypotheses

There is no statistically significant differences at (0.05 ≥) level in the trends of the research sample at Isra University for the following variables: Availability of the requirements needed to implement the e-assessments system which is: Infrastructure (computers, software, and networks). Professional staff responsible for the preparation and implementation of the e-assessments system, and the necessary skills and capacity to deal with the computer and the setup and configuration of eassessments, and indicators of the success of the implementation of the e-assessments system, and the problems they face when implementing e-assessments system.

• C.    Test results for students

  The results of analysis of X² test and described in table 11 indicate that there is variation in the trends of the research sample on all variables of the sub-hypotheses which are (Infrastructure, Staff, Skills, Problems, and Indicators of Success) where the level of significance for each of them are, respectively 0.000 , 0.000, 0.000, 0.000, 0.000 is less than 0.05. The level of moral for the requirements variable combined 0.0005, which is too less than 0.05. These results also came to confirm the rejection of all sub-hypotheses relating to all research variables. There any difference in students' attitudes about all the variables and this result was different from the results of aromatic means in the descriptive statistics which indicated an agreement in trends among them, but remains hypothesis testing is more reliable because it gives the most accurate results.

TABLE 9 MULTIPLE REGRESSION ANALYSIS FOR EFFECT RELATIONSHIP BETWEEN THE AVAILABILITY OF REQUIREMENTS AND SUCCESSFUL IMPLEMENTATION OF THE SYSTEM

Source of Variation	Squares of Deviation	Degrees of Freedom	Average Deviation	F	Moral Sig	R	R2
Slope	57.06	3	19.02	36.62	0.000	0.559	0.312
Remainders	125.0678	242	0.519
Total	182.737	245

TABLE 10 SEQUENTIAL RESULTS OF THE REGRESSION ANALYSIS OF THE INFLUENTIAL RELATIONSHIP BETWEEN THE AVAILABILITY OF REQUIREMENTS AND THE SUCCESSFUL IMPLEMENTATION OF E-ASSESSMENTS SYSTEM FOR STUDENTS ANSWERS

Independent Variable	R	R2	T	Level of Moral Sig
Infrastructure	0.477	0.228	2.23	0.026
Availability of Computer Skills			5.193	0.000

TABLE 11 MULTIPLE REGRESSION ANALYSIS FOR EFFECT RELATIONSHIP BETWEEN THE AVAILABILITY OF REQUIREMENTS AND SUCCESSFUL IMPLEMENTATION OF THE SYSTEM

Model	Infrastructure	Specialized Personnel	Computer Skills	All Requirements	System Success	System Problems
X2	108.927	212.244	169.415	132.61	130.024	155.610
Degrees of Freedom	16	21	23	42	26	26
Level of Moral	0.000	0.000	0.000	0.000	0.000	0.000

• V.    Conclusions

The results have shown the majority of members of the research sample agreed on the availability of all the requirements of the e-assessments system, i.e. the computerized infrastructure, the professional staff responsible for preparation and implementation of the system, in addition to the skills of students to use the system. The results also shown that the answers of the students to accept the alternative hypothesis that there is a relationship effect between the availability of the requirements for applying the e-assessments system and the success of the implementation process, and reject the hypothesis nihilism there is no effect relationship between.

The results of the regression analysis have shown that both variables of the infrastructure and the provision of skills and abilities of students to take e-assessments have an impact on the success of the implementation of the system, and rejected the hypothesis nihilism there is no relationship between them. The results indicate that there are differences in the trends of personnel research sample of students about all the variables of the hypotheses (infrastructure, provide professional staff, skills and abilities, indicators of success, and the problems facing the implementation process) and rejected hypotheses Sub nihilism to them.