Testing design methodology based on two-level multifactor experiment planning and data processing

The decomposition of knowledge into an array of “micro-knowledge” permitting a single-valued binary test score is studied. In this case, the test task is formulated as a set of micro-knowledge responses jointly revealing the knowledge level of the specific subsection. It allows presenting the development of a set of responses to a test question as the objective of investigating the multifactor dependence. Thus, the development process of a set of responses can be considered from the perspective of planning a multifactor two-level experiment. This approach allows a good formalization of the test construction, and its result estimation which becomes available to the teachers who do not specialize in the test task development. The method is illustrated by an example of the test construction, and application to assess the knowledge level on the automatic control theory. The data statistics has shown the respondents’ knowledge level, as well as the degree of a tester’s belief to the response awareness.