Instructor’s Performance:A Proposed Model for Online Evaluation

Published Online October 2013 in MECS

The higher education institutions have so many constraints and barriers in providing quality education to the students. They have made continuously asses various performances for improvement and stakeholder satisfaction. The major stakeholders of the educational processes are generally identified as: students, instructors and management. The students and instructors play a vital role in the development of the educational business activities. The student’s satisfaction reflects the quality maintenance in the institution.

The students’ satisfaction shall be measured with respect to various perspectives. One such perspective is instructor’s teaching, learning and assessment methods satisfaction. Various studies were conducted with respect to different outlooks to measure the impacts and needs of such satisfaction measures. Students are also one of the determinant factors of instructor’s success evaluation. The student’s evaluation reports sometimes may be biased due to student’s knowledge background, emotion and mentality. In some situations, the instructor’s involvement and classroom approaches plays an important role in biased student’s satisfaction results.

The instructor’s teaching pedagogies and methods have very high influences on student’s satisfaction. The instructor’s have made to select various teaching styles to cope up with the current environmental development. The instructor needs to provide a clear and efficient learning environment to the students. The instructor’s have made to use different assessment tools and techniques to measure the student’s knowledge acquisition level.

This teaching, learning and assessment method satisfactions should be conducted in unbiased environment with more students’ interaction. Based on the number of students and courses taught in the semesters, the instructors need to be evaluated for their performances. This evaluation should give proper and reflective results. The students’ satisfaction on instructor’s classroom performances should clearly indicate the teacher’s attitude, technology usages, student motivation techniques and classroom management practices.

Currently, the higher education institutions are using different instruments and methods to collect and measure the instructor performances. Some colleges are using traditional pen – paper questionnaire distribution and manually evaluating the performances. Some of the institutions in developed nations use online technique to collect data collection.

In this paper, we have developed a model online system to collect the data systematically from the students, analyze the collected data and prepare the report. The basic idea of this paper is to provide a logical design to online instructor performance evaluation system. We have also provided some basic screenshots of the expected system. We have taken Sur University College, Sultanate of Oman as a sample for the development of the system. As the author’s are either involved directly in this process, the ideas were developed from the current techniques and methods followed.

The paper has been arranged as follows: The related literatures were analyzed under Section 2. The data collected from various resources were analyzed and clearly described under Section 3. Section 4 describes the findings of the analysis and the proposed model of the system. The logical model of the system has been described and suggested under Section 5. The conclusion has been stated in Section 6. Section 7 shows the future scope of the study.

• II.    LITERATURE REVIEW

  Zhang Zhiyu et al (2010) studied about the teaching evaluation management system and designed a sample system. They introduced the design idea and implementation methods of the teaching evaluation management system. They said that the teacher’s evaluation is the most important part of the teaching management work of the institution. According to the authors, this evaluation will help in supervising and responding to the teaching situation and teaching effectiveness [1].

McFallet al (2002) suggested that the serious instructional development requires continuous fine grained feedback from students. They created a general purpose web based data collection and processing system. The system generates forms to collect data and stores the responses in a database. The system is very flexible to use HTML and additional processing using JavaScript or other HTML techniques. The system also includes several tools to analyze the data [2].

Mayer (1997) described the development and utilization of the criteria based course and instructor evaluation system. He developed the system with various forms to use collect the data for lab based courses and theoretical courses [3].

Emery et al (2003) studied the effectiveness of student evaluations on faculty evaluation. They said that the administration uses this student evaluation instrument to measure the faculty member’s performances and classroom teaching effectiveness. They said that the students were empowered to yield much influences over the career of the lecturers [4].

Theall and Franklin (2001) suggested that the good evaluation practices and the attendant benefits must be based on a systematic and careful approach. The appropriate and accurate interpretation and use of data in statistical and analytical procedures is the most important [5].

Dommeyer et al (2004) found that the response rate to the online survey is generally lower than that of the inclass survey. Students need to be motivated by incentives to get more online responses. Also, they identified that both the online and in-class evaluation have same mean values irrespective of any incentives. [6]

• III.    RESEARCH METHODOLOGY

  The current manual system used in Sur University College has been considered for this system development. The requirements were studied, analyzed and identified based on the available manual system. Some sample systems in other college in Oman used by the students were analyzed for the features.The current system was observed with the help of Sur University College quality assurance staff members who are responsible for conducting and analyzing this assessment. Various software development resources, software engineering techniques and tools (Jawadekar2010 [11], Peters &Pedrycz 2010 [12],Sommerville 2009   [10])were studied for

  developing logical design for the system which will suit to any programming environment. Dix et al (2008) [9], Russell &Norvig (2009) [8] were studied for the

• 3.1.    Analysis on Student’s Responses

requirements and development of user interfaces and interaction methods.

A questionnaire was distributed to the students of Sur University College to identifysome of the user requirements. The questionnaire contained 5 demographic questions, 14 closed ended questions and 2 open ended questions. A small sample size (n = 50) of students were considered for this study. Observations and interviews were used as the major research methodology to collect identify the user requirements. An interview was also conducted with the quality assurance manager, student affairs department, dean, other administrative departments and academic staff members for understanding the current system and the administrative and academic perspectives of the system.

The questionnaire was distributed to 50 students (46% Diploma Level, 24% Bachelor Level, and 30% Foundation Level), to identify the various users requirements and their opinions. Table 1 show that students respondents levels.

TABLE 1: Students Levels

Student Level	No of Respondents	%
Foundation	15	30%
Diploma	23	46%
Bachelor	12	24%

60% of the respondents were female and 40% were male respondents. Fig1 shows the corresponding male and female respondent’snumbers.

No of Respondents

40%

60%

• ■ Male ■ Female

Figure 1: Gender wise Respondents

65% of the respondents either strongly agreed or agreed that the instructor’s evaluation should be converted to online rather than the manual. 25% did not agree on this issue. 10% had no idea of this conversion. More than 40% of the respondents strongly agreed or agreed on the fact that filling questionnaire manually takes long time, 40% said that both online and manual methods of filling the questionnaire will take same time and 10% do not have any idea. Fig2 shows the number of responses in this regard.

45% of the students strongly agreed or agreed that they study all the questions clearly and answer. 50% of the respondents do not give their opinion on this fact and 5% strongly disagreed on this fact and said that they do not read and understand the questionnaire well before answering.

25% of the students strongly agreed on the fact that they found changes in the education system due to their responses in the questionnaire. 25% agreed on the same fact. 30% of the respondents were neutral, 15% slightly disagreed and 5% strongly disagreed on this fact. Table 2 shows the number of responses in this regard.

TABLE 2: Responses to Reading the Questions Clearly

Comments	Responses
Strongly Agreed	13
Slightly Agreed	12
Neutral	15
Slightly Disagreed	8
Strongly Disagreed	2

55% of the respondents agreed that the instructor’s evaluation is important for the development. 30% do not have any opinion on this and 15% disagreed on this. 24% and 40% of the respondents either strongly agreed or agreed on the fact that the instructors should be evaluated every semester, 20% disagreed on this statement 16% had no idea. Fig3 shows the number of responses on the question that the instructor’s should be evaluated every semester.

60% of the respondents agreed that the online evaluation system will be flexible for them. 30% strongly agreed on this statement, 5% were neutral and 5% either disagreed or strongly disagreed on this statement. 25% of the respondents strongly agreed that the online evaluation will increase their confidence of giving the truth about the instructor, 40% of the respondents agreed on this fact, 10% had no idea, 20% disagreed on this statement and 5% strongly disagreed that they will give more clear ideas about the instructor. 45% of the students said that they have access to the computer at any time, 30% said that they have no idea and 25% said that they do not have access to computer all the time.

• 3.2.    Analysis on Staff Member Responses

20 academic staff members (35% Ph.D’s, 45%

Master’s and 20% Bachelor Degree holders) and 10

administrative staffs (40% quality assurance staff members, 40% programmers and technicians and 20% others) were interviewed to study the management and administrative requirements of the system. As the instructors are the second major stakeholders of the education system, they were also interviewed for the analysis purposes.

All the academic staff members strongly agreed or agreed that they should be evaluated for their performances with respect to their academic activities. They said that the evaluation results will enable them to make necessary changes in the teaching methods, providing learning environment and assessment of the students. 40% of the academic staff members agreed that they should be evaluated every semester, 40% disagreed on this statement and 20%do not give their opinion. 65% of the academic staff members said that they should be evaluated for all the courses they teach in the particular semester and the average should considered for further actions.25% disagreed on this fact and 10% did not give their opinion.

80% of the academic staff members agreed that the online evaluation will help them to get more precise feedback as the students can evaluate the instructors even during their free times. 15% of the instructors said that the students may not give their factual ideas through online as the students may feel the online evaluation will be laborious. 5% said that either online or manual the students’ responses will be the same.

• IV.    FINDINGS AND PROPOSED MODEL

The analysis shows that the students and quality assurance office (or) students affairs department who conducts the survey are the main users of the system. It has been observed that in some higher education institutions students affairs department conduct this survey and analyze the same and in some institutions the quality assurance office conducts the survey. As we have considered Sur University College as the system testing center, the quality assurance office is considered as the survey conductor.

It has been observed that the following are the user requirements of students.Each student (user) must be provided with user ID and password to logon to the system. As the students, irrespective of the specialization, have basic computer knowledge and operating skills to deal with the system.Student should have the any type of internet browser and internet connection to logon to the system. As the quality assurance office is the second main user of the system, the following the basic requirements at quality assurance office. The each staff in quality assurance office should be provided with admin ID and password to logon to the system.Each such user should have advanced computer knowledge and skills to deal with the system.Quality assurance office should have a database to collect and store the data.

Based on the above findings the following model has been proposed in Fig 4.

Figure 4: Context Diagram for the proposed model

The major users of this Online Instructor Evaluation System identified are: Students, Quality Assurance Office, Registration Department and Management. The students are the evaluators of the instructors. Quality assurance office collects and analyzes the data and prepares reports. The administration department provides the information about the students and instructors regarding the courses and corresponding sections. The management reviews the reports and takes further necessary actions. Fig5 shows the viewpoint of the proposed system.

Figure 5: Viewpoint Hierarchy of the proposed system

This Viewpoint hierarchy shows all services that Online Instructor Evaluation System (OIES) provide to main users of the system and sub users of the system. The students and quality assurance office are the main users of the system. The admin and management are the sub users of the system. Students access the system and input the data. Quality assurance office accesses the system database and analyzes and generates instructor evaluation report and update the database with reports. System administration provides the user id, password and maintain the system over the internet. The management will receive only instructor report. Then management sends the report to the corresponding instructors and takes actions.

• V.    LOGICAL DESIGN OF PROPOSED MODEL

Using above context diagram and viewpoint hierarchy the following logical design has been proposed. The students and the quality assurance office who are the major users of the system should have maximum access to the system. They need to follow the proper steps. Fig6 shows the flowchart diagram of the student’s process and interaction methods with the system.The figure shows that the students have to input the St_Id and St_Password. If the input values are correct then the students will be allowed to select their course and section. The system will show the corresponding instructor of the course. The system will automatically save the instructor values in the table. The students shall fill the questionnaire and submit the same to the system. The system will give the confirmation form for the students with the full details of filling including the instructor name, course, section, questionnaire filling date and time. On submit, the data entered will be converted into read only data. Further modification cannot be made there after by anyone interacting with the system.

Fig7 shows the activity and system interaction flowchart of the second major user quality assurance office. Quality Assurance staff members are the members to analyze and prepare the instructor evaluation. If their user name and password are correct, the staff members will be allowed to interact with the data. They can access only the read only data of the student evaluation. The system will allow them to analyze the evaluation and prepare the final instructor report.

Figure 6: Student Activity Flowchart

Figure 7: Quality Assurance Office Interaction Flowchart

Fig8 shows the user interface and process of the system.

Interface

Ms Access Forms *

VB

--------1-------

Processing

__________i_________

StudeotUser:

Get evaluation form

Enter St_ ID+ Password Check course_ ID Check Instructor-ID Fill the form

QAOUser:

Get evaluation form

Enter Admin_ ID+Password Check coiuse_ ID Check Instructor-ID Analyze result

Get Result UpdateRepon

Database

• *    Student Information (std ID, Password)

• •    Courseinfonnation (Course ID)

• *    Instructorlnformation (Instructor Name)

• •    Administration Information

Figure 8: User Interface and Process of the System

• VI.    CONCLUSION

The existing manual system of the college showed that there is a dire need for on line evaluation system with respect o quality purposes. The QAO found it is difficult to continue using the traditional system because the number of students is continuously increasing. It is feasible to implement and apply the OIES. It will save efforts, more efficient, fast, and more convenient. The Online Instructor Evaluation System (OIES) depends on deep research and good analysis of the need to digitize the evaluation in order to achieve the maximum benefits of the machine, minimize time and cost, and establish accuracy and security and to prevent missing and duplication evaluation papers. The system will be implemented using advanced technologies and it achieved its goals. The system is based on a strong database controlled by well secured distributed applications. The system designed to collect student’s filling evaluation formsto monitor instructor performance. The project is useful for all students. As a result of completing this system, the team learned how to solve the problems that we faced. They have learned how to exchange ideas and experience and how to manage times dividing the work at the time.

• VII.    LIMITATIONS AND FUTURE SCOPES

The system has been proposed with respect to the current requirement in the manual system. Some of the institutions are already using such system shall conduct feasibility study on their requirements. We have provided the logical design of the system. Using this logical design, the system design shall be developed using any preferred frontend and backend. This logical design shall be further studied with respect to the institution size, type and nature.

ACKNOWLEDGEMENT

First of all, we would like to thank Amna Said HamedAl.Satmi, Sheikha Abdullah HamedAl.Kasbi, AsmaSalehQasimAl.Balushi and Aala Said Ali Al.Balushi the final year bachelor students for their supports in collecting and developing this system. We would like to thank Sur University College Management for their financial and moral supports for completing this research. We would also like to thank the dean of Sur University College for his extensive supports in conducting the research activities in the institution.