Enlightenment on Chinese Evaluation Standard of Educational Game Software from TEEM's

Автор: Liang Lu, Yang Huansong

Журнал: International Journal of Education and Management Engineering(IJEME) @ijeme

Статья в выпуске: 5 vol.1, 2011 года.

Бесплатный доступ

The author takes the evaluation standard of educational game software(EGS) made by TEEM as an example and analyzes its dimensionality, content and effectiveness. Then the author stresses the enlightenment on the evaluation standard of Chinese EGS about evaluation organization, evaluation scientificity and evaluation efficacy. The aim of this paper is to offer guidance to those enterprises producing EGS, schools and families purchasing EGS, teachers and students using EGS and promote Chinese evaluation standard to match international standard.

Educational game software, evaluation standard, enlightenment

Короткий адрес: https://sciup.org/15013618

IDR: 15013618

Текст научной статьи Enlightenment on Chinese Evaluation Standard of Educational Game Software from TEEM's

The evaluation standard of EGS has important and constructive significance for designing and developing EGS, improving the effect and efficiency of teaching. Because of the different cultural backgrounds, different countries have different standards. The famous international evaluation organization TEEM (Teacher’s Evaluating Educational Multimedia), IHMC (Institute for Human and Machine Cognition) and H2C (Happy Companies Healthy People) have issued different standards of appraisal. While Chinese evaluation criteria of EGS needs further study, there is no a canonical or systematic evaluation criteria or method which is comprehensive of EGS.

This paper selects the evaluation standard of EGS which made by TEEM as an example. This paper analyzes the dimensionality, content and effectiveness of the evaluation standard and gives us enlightenment on how to found organization, evaluate scientifically and have better efficacy, trying to help enterprises produce EGS, the school and the family purchase EGS and the teachers and students use EGS with guidance. This paper provides enlightenment on Chinese evaluation of EGS to come close to the international standard.

* Corresponding author.

  • 2.Brief introduction of teem and teem’s teacher evaluation framework

  • A.    Brief introduction of TEEM

    TEEM is the abbreviation of Teacher’s Evaluating Educational Multimedia. It is a service organization which engages in evaluating educational software, educational websites and educational tools. It is a non-profit group that is independent of government which established in 1997. It has a range of investors for free, such as: British Educational Communications Technology Agency, British Education Products Association, etc. These external assistance not only promote the smooth development of TEEM, but also make the evaluation standard issued by TEEM recognized by the society. The organization's main job is to provide reliable and practical suggestions or references for front-lines teachers by evaluating educational software. The objects of evaluation are educational software, educational websites and educational tools which applied in 1-4 teaching periods (in England and Wales teaching period).

  • B.  TEEM’s teacher evaluation framework

  • 3.Analysis of the evaluation standard of EGS made by TEEM

TEEM builds up assessment standards with many factors, and investigates with questionnaire undertakes. It evaluates from several dimensions, such as: overview of the use of this game in class curriculum relevance(content and skills) design and navigation ,etc. It analyzes the applicability of EGS from two aspects: the use in class and the technicality of EGS. Its evaluation covers almost all aspects of the procedure in detail with depth, and provides teaching practice. TEEM is well recognized by the society for its genuine evaluation of learning software.

  • C.    Evaluation dimensionality

As an independent appraisal agency, TEEM evaluates EGS from six dimensions of overview of the use of the game curriculum relevance(content and skills), design and navigation ease of use edutainment and installation in a fair and objective way. TEEM evaluates the use in class and the technicality of EGS. It evaluates whether the software create a reasonable situation or be as interactive and responsive as possible. It evaluates whether the software accord with the logic of learners’ perception or organize and regulate the learners and their learning process. Though TEEM’s evaluation is scientific and practical, it needs to be added game play of software as another evaluation dimension.

  • D.    Evaluation content

TEEM builds up evaluation standards with experts teachers students and so many other factors. It evaluates EGS on requirements of interface design difficulty degree of operation application in teaching and learning ,etc. In respect of design and operation, TEEM puts emphasis on assessing whether the design or navigation of the program is clear enough to support the use in class, whether the interface is simple and fascinating, whether the program is easy to operate, whether the software has examples or guidance for the students, whether the interaction is great. Regarding the application in teaching and learning, TEEM evaluates whether the game software support subject areas or teaching and learning objectives, which patterns for teaching and learning to choose, individual or group. It evaluates whether the users can record and evaluate the learning process exactly and reasonably in detail. It evaluates whether the game software help the students mine deep information and train educationally relevant skills like: communicating and working with others, improving own learning and performance, solving problem, managing financial, etc. It also evaluates whether the teachers meet a higher requirement in this process.

  • E.    Evaluation effectiveness

  • 4.Enlightenment on Chinese evaluation standard of EGS from TEEM’s

The outstanding feature of the utility of TEEM’s evaluation is directing at instruction in class. TEEM evaluates whether the EGS has enough curriculum relevance or edutainment. In respect of curriculum relevance, it evaluates from the two parts: content and skills. It evaluates whether EGS can optimize certain aspects of instruction in class to help students learn the relevant content sufficiently and pertinently, whether the EGS has substantive effect on developing educational skills, whether it is easy to use, whether its design is distinctive and so on. Its evaluation emphasizes the technicality of EGS, but my point of view is that it neglects the evaluation of difficulty of developing EGS,the maintenance cost and life cycle of EGS. In order to update the EGS, we should fully consider these aspects.

  • F.    About the evaluation organizations

The research of Chinese evaluation standard of EGS has still paused at a preliminary stage, there is no a canonical or systematic evaluation criteria or method which is thorough and comprehensive. From the analysis we know only a neutral and professional organization can issue excellent evaluation standards, can promote the development of EGS, and can make users get the information of EGS accurately. Such as: applicable environment, the advantages and disadvantages of the design of EGS, applicability for class and so on. While China lacks professional evaluation organizations which do not pursue commercial interests or do not represent any organizations or collective interests like TEEM. Therefore, China should learn from the foreign distinct experience to establish evaluation organizations which is independent of government and the publishers. Such organizations can create scientific appraisal from an objective perspective, can evaluate in detail with depth, and can be well recognized by the society for its genuine evaluation.

From the analysis above we need stable and sufficient funds to establish such evaluation organizations, like :TEEM has a range of investors for free, such as British Educational Communications Technology Agency British Education Products Association ,etc. So the support from our government and some private capital is the precondition of establishing and developing excellent evaluation organizations to issue first-class evaluation standards of EGS.

  • G.    About the evaluation scientificity

  • 1)    About the scientific methods of evaluation

Currently, due to the cost of evaluation the limit of time and bound, etc, our evaluation standard of EGS is not scientific enough. The division of labor is also not very reasonable.

We can learn the scientific methods of making scientific evaluation standards from foreign evaluation organizations, like: TEEM builds up assessment standards with many factors. It contains the ideals of experts of Education Science Curriculum and Technology. And it investigates with questionnaire undertakes to get teachers' and students' feedback of using the EGS. This method can help us to know the actual use of the software deeply. Besides the questionnaire, we can have reports by communicating with the teachers and students who have used the software. And the teachers and students who participate in evaluating should have been trained to reduce human error to ensure the evaluation can be fair and objective. In my opinion, we can let the Professional of Educational Technology to participate in evaluation. They can evaluate on the point, because EGS is the combination of education and technology.

  • 2)    About the scientific content of evaluation

The evaluation standard of EGS of China has the following shortcomings. Such as: the educational index is generalized, and the objective index that can be measured is over-pursuit. These lead to the low level and repeatability of the software investment which not only brings the waste of resources, but also hinders the communication with the international evaluation standards.

From the analysis, we find that scientific evaluation of EGS should have reasonable and scientific content in the technical aspect; it can evaluate whether the EGS has reliability, clarity and so on. In the learning aspect, it can evaluate whether they focus on the interaction, feedback, whether the difficulty is moderate to be consistent with the users’ logic, and whether the software meet the curriculum needs and course standards. In the artistic aspect, we should evaluate the language, screen art, sound effects, etc. Besides, the standard of the evaluation should be classified and refined reasonably, updated continually, and different systems of evaluation standards should be developed according to the different disciplines and users’ characteristics.

  • H.    About the evaluation efficacy

  • 5.Conclusion

With the continuous development of our time and society, the evaluation standard of the EGS should be upgraded with the latest educational ideas and technologies from time to time, and be made to get the vigorous characteristics of our time. The reasonable evaluation standards play a key role in the promotion of Chinese teaching outcomes, which can be seen as follows: Firstly, feedback functions. They can provide the feedback of the teaching activities, which is vital to adjust the teaching and learning and make the process go well. Secondly, diagnostic functions. Not only can they be used to analyze the outcomes and constitutions of the EGS, but also identify the problems. Thirdly, incentive functions. The scientific evaluation can arouse the enthusiasm of teaching and the students' interests in learning. Fourthly, goal oriented functions. They can help the teachers identify the knowledge which is necessary for teaching. And above all, teachers can assist students in learning to achieve the goals and improve their reflective abilities by themselves. Good evaluation standards pay more attention to the process of students’ learning. They facilitate the transformation of traditional teaching to let each individual be the learner and contributor of knowledge and information.

Outstanding evaluation standards of EGS are necessary, which can improve the quality of EGS and offer reference to those units or individuals purchasing the software. Nowadays, the development of EGS has progressed each passing day and many new types of EGS emerge in a highly intelligent and humane trend, so the evaluation methods should be improved accordingly.

According to the researches and analysis above, a first-class evaluation standard of EGS includes: It offers guidance for the users to see if the Software Architecture meets the pattern of learning cognitive and the content of curriculum. It reflects the support of different teaching strategies, the increase in working efficiency of software operators’ and the higher availability of the software itself. It reflects the improvement of the rate of return and the scientific methods of purchasing for the deliverers of the software. It reflects educational software engineering provides guidance for the producers to ensure high quality and low-cost inputs of developing EGS. If the evaluation work can attach importance to the points above, it will promote the development of EGS in the practical and healthy direction.

Список литературы Enlightenment on Chinese Evaluation Standard of Educational Game Software from TEEM's

  • Han Ying and Lin Peiying, Analyzing the evaluation organization form of educational software of ICT in British _Taking the website of TEEM for example[J]. Journal of Distance Education.2004.(in Chinese)
  • Report on the educational use of games. In http://www.teem.org.uk
  • Alvaro Galvis and Babette Moeller, Rubric for assessing or designing playful learning spaces[J]. in press.
  • Educational Software Evaluation Form. In http://h2cleadership.com
  • Claudio Dondi and Michela Moretti, A methodological proposal for learning games selection and quality assessment[J]. British Journal of Educational Technology,2007.
  • Functional Requirements Learning Platforms.In http://www.becta.org.uk
  • Yuan Lixin,Thought on evaluation of educational software[J]. Education Exploration.2007.(in Chinese)
  • Fang Haiguang, Researh on value evaluation of educational software[J]. 2006.(in Chinese)
Статья научная