Evaluation: The New Philosophical Roles & Psychological Means

Published Online September 2013 in MECS

Evaluation means many things to many people. According to Shadish W.R, Cook T.D, Leviton L.C [1] “Evaluation-More than any Science- is what people say it is, and people currently are saying it is many different things”. According to Torsten Husen and Neville Postlethwaite [2], “The process of defining the roles and the means of evaluation is almost as large as that of teaching.” To understand the concept of evaluation Nyberg.D [3] states, “the teacher is like a banker who should also control evaluation which is the currency of the classroom (and greater educational) economy. The evaluations are the loan notes that the student repays with interest and the bank may call at any time. In this system of exchange it is very likely that if the student shows great interest i. e. pays lot of attention, then the loan of positive evaluation is secure.”

• A.    Evaluation Concepts

It is continues process of improvements in concepts of evaluation over the years. From surveys and studies of concerned literature, it is found that these improvements have evolved primarily through interaction of evaluation practices with three other aspects. In one such interaction, evaluation practices have been affected by the acceptance of various learning theories and approaches. A second interaction has been that of evaluation practices with roles accepted for evaluation. The third interaction has been between evaluation practices and technical developments in measurement and evaluation itself.

• B.    Philosophy & Psychology of Evaluation

Philosophy based on truth and reality. Roles are assigned to researchers, evaluators, candidates, parents and decision making body like government with the use of philosophy. Basic logic of evaluation can be set after the roles are assigned. Psychology is based on reasoning. Psychological means can be determined by various phases of evaluation process. New Psychology roles and psychological means combined to gather, reconstruct the concept of evaluation.

This paper discussed the use of basic evaluation concepts applied for evaluation, basic logic of evaluation can be set philosophically and new psychological roles and means of candidate and evaluator to complete the evaluation process.

A balance must be struck between candidate and evaluation process. When candidate becomes an end in itself, the balance between candidate and process is upset, and process is a slighted partner. When a balance has been struck between candidate and process, evaluation become comprehensive and complete. The extent to which candidates know and comprehend things, and the extent to which they can do such things as think autonomously, use prior knowledge to solve new problems and to make decisions, are considered integral in this evaluative scheme.

• II.    THE RELATIONSHIP BETWEEN RESEARCH AND EVALUATION

The relationship between research and evaluation need to be made explicit. It is an evident that many of the activities associated with evaluation in education are also associated with educational research. In many instances, no distinctions are made between evaluation and research. Let us first look at the broad objectives of research and of evaluation as they can now be observed in the field of education. It appears to be well agreed that the objectives of educational research is to add to our knowledge of the practices and methods of education. Whether new knowledge created by educational research, should have some immediate usefulness or whether such research is sufficiently justified by the potential value of any new knowledge or by the satisfaction of any ideal curiosity. Thorstein [4] suggested distinctions between applied researches and Basic research are often based on consideration of immediate utility against the possible usefulness of specific new knowledge.

Evaluation studies are made to provide a basis for making decisions about alternatives and question of utility. Regardless of the lack of precision in thinking, providing information for choice among alternatives remains the basic and utilitarian purpose of evaluation studies. It is necessary to regard research, particularly basic research, having a distinctly different objective than evaluation. Often the information to be provided by an evaluation study is needed because decision must be made. This decision-making is not usually an integral part of the evaluation study itself, but subsequent activity. This fact might lead to the conclusion that an evaluation could avoid questions of value and utility, leaving them to the decision-maker and not need to be distinguished research, either basic or applied. Evaluation studies differ from research in the manner in which value questions are involved-especially value questions that under gird choices about what information or knowledge are sought. The implications of primacy of utility in evaluation studies and the relative low consideration in research are profound. Although there are differences in points of view among behavioral scientists and ideal research study in the following manner: Hemphill [5]

randomization.

Evaluation studies are often undertaken in response to a need to know the usefulness of invented alternatives to an existing mode of action that has resulted from some combination of old and new knowledge. An evaluation provides test of generalization and thus goes beyond the point at which most research stops- the verification of the hypotheses within only a very controlled and restricted situation. In this respect, evaluation contributes side by side, but with distinctly important way, to the development of an education science. Both research and evaluation studies share many characteristics of method and approach. Both contribute to expand the knowledge and stimulate evaluation process.

• III.    PHILOSOPHY OF EVALUATION

We all presumably subscribe to a democratic philosophy of living. One aspect of democracy is a concern for the worth and integrity of every individual. Our evaluation activity can be consistent with this democratic philosophy. In a class room there can be heterogeneous group and level of each student can be different.     Today, teacher constantly reminded inabilities of poor student and keeps up with rest of the class. It is a very slow process and students are not directly responsible for slowness. It is important that we evaluate student such away that the individuals feel about their constant failure. Current evaluation method is inconsistent with both of our democratic philosophy and everyday life.

Psychologists are generally argued that all individuals need successful experiences, in order to carry out the normal living. If individuals do not have experiences in one way, they will attempt to do in another way. Many students are not comfortable with current evaluation pattern so, they are turn to other types of undesirable behavior.

The solution of this problem would seem to be the teacher should be aware of individual differences and flexible enough to incorporate any level of student’s evaluation. The current student populations are so heterogeneous that the individual achievement in any given classroom is very comprehensively. Troyer [6] has summarized evaluation activities as follows:

• 1.    The major purpose of evaluation is to improve learning.

• 2.    Evaluation should be done with group rather than to an individual.

• 3.    Evaluation process identifies individual strengths and weaknesses of each student.

• 4.    Evaluation of progress is on the basis of an individual ability.

The well known philosopher Dressel. P. L. [7] discusses the basic philosophy of evaluation as under:

• 1.    In democracy each individual is evaluated in such an away that, they contribute to society and also receive personal satisfaction.

• 2.    The judgments required in assessing an individual’s potential are complex in their composition, difficult to make and filled with error. Such error can be reduced but never eliminated. Hence any evaluation can never be considered final.

• 3.    Composite assessment by a group of individuals is much less likely to be in error than assessment made by a single person.

• 4.    In a democracy every form of evaluation has critics, which is a spur to change and improved.

• IV.    PHYSIOLOGY OF EVALUATION

To make evaluation activities most effective, evaluator considers best possible techniques to be used with what we know to be the best and most effective psychological principles. For sound evaluation procedure, the following are the principles of psychology:

• 1.    In order to teach a child to read, he has to have reached a certain stage of mental and physical development. In the same manner, evaluation is most successful when the student is ready for it. A student is ready when he understands and accepts the values and objectives involved. Guilford [8]

• 2.    It has long been known that people tend to carry on those activities which have success associated with their results. This has been known as Thorndike’s Law of Effect. Thus if certain test demand rote memory, the student becomes memorizes. If a test on the other hand, requires students to apply principles, interpret data or solve problem then student performs very well in such type of evaluation.

• 3.    The third psychological aspect of evaluation to be discussed is motivation. The motivation of students is one of the most important and sometimes the most difficult to handle of all problems related to evaluation. A research study has shown that when a student is really motivated, performance is much closer to his real top performance than when motivation is lacking. Little [9]

• 4.    A forth-psychological factor or principle, which has implications in evaluation, is the learning is most efficient when there is activity on the part of the learner. When test results are not placed on answer sheets and run through a scoring machine, teacher nor student learns anything about how the student performed on the test. Important is the total score that the machine operator wrote or the machine printed on the answer sheet Downie [10].

• V.    EVALUATION IN APPLIED PSYCHOLOGY

Just as it was previously observed that a good grasp of probability and statistics had become an essential tool for a great deal of work in applied psychology, so we now find that knowledge of the logic of evaluation and of some of its specialized methodologies are increasingly crucial for much investigation in applied social sciences Scriven, Michael [11]. Funding agencies letting evaluation contracts or assigning staff to evaluation increasingly want to know not only what is happening when an intervention is supported, and exactly what causes the results - familiar questions for applied social science Downie [10].

• 1.   Whether the intervention is worth what it cost.

• 2.    Whether there are unintended bad results as well as planned good ones,

• 3.    Whether the methods used in the intervention were proper by current professional and ethical standards.

• 4.   Whether there are better ways to do the same thing.

This is the domain of evaluation and none of those questions can be answered reliably without use of its special logic and special methodologies.

• VI. THE BASIC LOGIC OF EVALUATION

In order to reach evaluative conclusions it is necessary to establish or to identify the following kinds of premises: Factual premises (e.g., about nature, performance        or        impact)        and

Value premises (e.g., about the relevant legal or scientific principles).

There are usually many of these premises in the evaluation of complex entities (or entities with complex functions). To obtain the required kind of overall evaluative conclusion, it is necessary to combine all of them by means of what is called 'the internal synthesis process.' Scriven Michael [11]. This synthesis step is one of the key logical processes in evaluation and is a long way from the simple deduction and statistical inference that are more common elements in scientific inference.

Factual premises in a field like program evaluation are commonly established using the standard procedures of social science with the assistance of other disciplines such as history and jurisprudence. Value premises usually come from one or more of eight sources:

• 1.    Legal principles

• 2.    Scientific and mathematical standards of truth (especially relevant when the program disseminates information or is based on scientific theories or common assumptions)

• 3.    Professional, cultural, or organizational standards of proper conduct (e.g., the APA testing standards)

• 4.   Needs assessments

• 5.   Definitions (which provide linguistic standards of

• 6.    Market research

• 7.   Logic

• 8.    Ethics

propriety)

Again, the social sciences are a common source for several of these types of values e.g., the scientific standards of truth used in judging the quality of the assertions--or assumptions--built into or propagated by the program. From psychology, we frequently encounter premises about maturational rates, cognitive processes, or leadership research. The logic of evaluation comes in with the frequent need to balance these value considerations when they conflict: that logic originates in jurisprudence and moral reasoning, but has been expanded to cover other fields of evaluation besides these, e.g., evaluation of alternatives in high-stakes decision-making Scriven Michael [11].

• VII. EVALUATION FIELDS

The better-known fields of applied evaluation vary considerably in quality as well as in their relevance are dependent on the social sciences, while personnel evaluation is heavily dependent on the social sciences and high degree of objectivity and utility. Remaining is independent from these dimensions and close to pseudo-evaluation (e.g., wine taste, art criticism). There are partially valid evaluations like architectural criticism, portfolio management and literary criticism. There are highly valid evaluations but are not dependent on the social sciences (e.g., the reviews done by appellate courts, the evaluations of claimed proofs of Fermat's Theorem in mathematical journals). All the eight applied fields equally important for practical and logical reason: they fall into two groups. The first Six are the fields of program, personnel, performance, policy, proposal, and product evaluation (the latter including technology assessment). The remaining two are:

• 1.    Interdisciplinary evaluation--the evaluation of the entities that are most important to faculty’s such as theories, hypotheses, classifications, data, research designs and results, practitioners, contributions, and journals--and of the faculty itself.

• 2.   Meta-evaluation, the evaluation of evaluations

themselves, a practice that demonstrates the impulsive nature   of evaluation and the philosophical reliability of its practitioners.

The first is the backbone of all disciplines--it is what makes them disciplines. The second is the backbone of evaluation--it is what makes it consistent by making it practice what it preaches.

The first Six fields--the conventional fields of evaluation--program evaluation is the one with the largest associated job market at the moment, with personnel evaluation (an Human Resources staple) and performance evaluation (especially in the educational area, where its academic fountain is often referred to as 'tests and measurement') coming next Robert [12].

• VIII. EVALUATION IN THE SYSTEM OF EDUCATION

Evaluation chart is given in Fig. 1. Evaluation methods must be provided to accommodate candidate with special needs and interests. Each candidate having unique identity. Special curriculum expectations and evaluation methods may have to be adapted to meet the requirements of candidate.

Figure 1. Evaluation Chart

VIIII. CHARACTERISTICS OF EVALUATION

The Education Commission (1964-66) has understood that, “Evaluation is continues process, forms an integral part of the total system of education and it is intimately related to educational objectives. The techniques of evaluation are means of collecting evidence about the student’s development in desirable directions. These techniques are valid, reliable, objective and practicable”. The following are the characteristics of Evaluation:

• C.    Evaluation includes Academic and Non-Academic Factors:

Evaluation is essential in the never ending cycle of formulating goals, measuring progress towards them and determining the new goals which emerge as a result of new warnings. Evaluation involves measurement that means objective qualitative evidence. But it is broader than measurement and implies that considerations have been given to certain values, standards and that interpretation of the evidence has been made in the light of the particular situation.

Evaluation in its broader con cept includes examination of academic and non-academic aspects of education. According to J. Wayne Wright-stone [13], “Evaluation is a technical term, introduced to design a more comprehensive concept of measurement than is implied in conventional tests and examinations.” Evaluation involves the identification and formulation of a comprehensive range of the major objectives of a curriculum. This definition is in terms of student behavior and the selection or construction of valid, reliable and practical instruments for appraising the specified phases of student behavior. Academic and Non-Academic factors are mentioned in the table –I

Academic Goals	Level of commitment to obtain a college degree
Academic Motivation	Level of motivation to achieve success
Academic Confidence	Level of academic self-confidence (of being successful in the academic environment)
Academic Skills	Time management skills, study skills, and study habits (taking notes, meeting deadlines, using information resources).
Relative Influence	The extent to which students receive financial aid, institution size and selectivity
General self Concept	Level of self-confidence and selfesteem
Social Support	Level of social support a student feels that the institution provides
Social Involvement	Extent to which a student feels connected to the college environment, peers, faculty, and others in college, and is involved in campus activities

• D.    Evaluation is a procedure for improving the product:

Wiles [14] define evaluation as, “Evaluation is a process of making judgments that are to be used as basis for planning.” It consists of establishing goals, collecting evidence concerning growth or lack of growth towards goals and making judgment about the evidence and revising procedures and goals in the light of judgments. It is a procedure for improving the product, the process and even the goals themselves.

• E.    Correlation between the Educational System and the System of Evaluation:

Evaluation cannot be done in vacuum. It is always with reference to the objectives of a particular system of education. Evaluation has to be very comprehensive in system of education which aims at the many sided development of the personality of student. The colleges concern them not only with intellectual pursuits but also with the emotional and social development of the student, his physical and mental health, his social adjustment and other equally important aspects of his life with all round development of his personality. In a light of these objectives of the system of education, a comprehensive program of evaluation includes:

• 1.   Evaluation of the academic subjects.

• 2.   Evaluation of the skills.

• 3.    Evaluation of Moral and Social Development

• F.    Evaluation is a complex process:

In modern times evaluation tends to become a complex process. It needs scientific techniques and tools. It needs the corporate efforts of teachers, students, parents and the administrative staff. It involves an element that brings cohesiveness or coordination in the whole activity Aggarwal. J.C [15].

• G.    The need of Cross-National Evaluations:

Evaluative judgments have always been passed upon the relative merits of educational systems in different countries. Apart from the results of the more formalized attempts of the educators to make comparisons, quite a lot of folklore has prevailed, particularly concerning the ‘productivity’ or ‘efficiency’ of the various systems. The need for making cross-national or cross-cultural evaluation in education has grown rapidly for various reasons during the last two decades. Some of the reasons for this need for more accurate evaluation of the outcomes of various educational systems are as follows:

International student mobility has increased tremendously during the last few decades. The number of students in Europe and America, particularly from Africa and Asia, has recently grown to tens of thousands. Furthermore, a growing number of young people go for graduation and post graduation in a country other than their native one because of the movement of their parents. The increase in the exchange of university students has led to a growing demand for international baccalaureates whereby university entrance requirements are evaluated cross-nationally.

The technical assistance in the educational field provided both multilaterally and bilaterally, has created a demand for techniques by which the ‘quality’ of the educational systems in emerging countries is assessed. There is indeed a strong need for both fruitful theoretical models and international standard by which educational systems are evaluated. The more interesting thing happed in education is an investment in human capital and bring the instrument for economic growth and social change, particularly in the developing countries, the stronger need to develop such models and measuring instruments for evaluation process. So far, many of the studies of the relationship between education and economic growth have been limited to the use of very crude ‘output’ variables – for instance, enrolment and graduation figures. No quality measures have been used until recently; because no internationally valid and applicable instruments for measuring outcomes of instruction are exist.

So far, many of the cross-national variables established are independent ones, many of them pertaining to input of money, physical plan and personnel. The situation has been much less satisfactory with regard to output variables. In most cases, enrolment and graduation figures have been used as evaluative measures. Harbison [16] stated that attempt should be made to devise internationally valid criterion measures, which would make it possible to evaluate uniformly the educational practices including the standards of the different countries. The standard procedure in any evaluative process is of course, to begin with an analysis of the objectives, which are to be achieved in the educational systems under consideration. But since the curriculums and syllabuses in most of the participating countries are drawn up at the national level, the objectives vary in educational traditions but also upon variations in social and economic structure and values, which are implicit in the educational policy as stated by central authorities or leading national bodies.

• X.    PHASES OF THE EVALUATION PROCESS

Although evaluation is not strictly sequential, it can be viewed as a cyclical process including four phases: preparation, assessment, evaluation, and reflection. The evaluation process involves the teacher as a decision maker throughout all four phases.

• A.    Preparation Phase:

In this phase, decisions are made by identifying what is to be evaluated, the type of evaluation (formative, summative, or diagnostic) to be used, the criteria against which student learning outcomes will be judged, and the most appropriate assessment strategies that provide student’s progress information collectively. The teacher's decisions in this phase form the basis for the remain ing phases.

• B.    Assessment Phase:

During this phase, the teacher identifies informationgathering strategies, collects student products, constructs or selects instruments, administers them to the student, and collects the information on student learning progress. The teacher continues to make decisions in this phase. The identification and elimination of bias (such as gender and culture bias) from the assessment strategies and instruments, and determining where, when, and how assessments will be conducted are examples of important considerations for the teacher in this phase of evaluation.

• C.    Evaluation Phase:

During this phase, the teacher interprets the assessment information and makes judgments about student progress. Based on the judgments or evaluations, teachers make decisions about student learning programs and report on progress to students, parents, and appropriate school personnel.

• D.    Reflection Phase:

It allows the teacher to consider the extent to which the previous phases in the evaluation process have been successful. Specifically, the teacher evaluates the utility and appropriateness of the assessment strategies used. Such reflection assists the teacher in making decisions concerning improvements or modifications to subsequent teaching and evaluation.

All four phases are included in formative, diagnostic, and summative evaluation processes Walter [17]. They are represented in Fig. 2.

Preparation Phase

Reflection Phase

Assessment Phase

Evaluation Phase

Figure 2. Process of Candidates Evaluation

• XI.    CONCLUSIONS

Internal synthesis process is one of the key logical processes in evaluation and is a long way from the simple deduction and statistical inference that are more common elements in scientific inference. Applied evaluation highly depends on social sciences while personnel evaluation heavily depends on high degree of objectivity and utility.