Developing Criteria and Indicators for Assessing Students’ Knowledge in a Digital Educational Environment

Бесплатный доступ

The digital transformation of higher education requires the development of assessment systems capable of measuring not only the amount of knowledge acquired by students but also their ability to understand, apply, analyze, evaluate, and create information in technology-mediated contexts. This article examines the theoretical and methodological foundations for developing criteria and indicators for assessing students’ knowledge in a digital educational environment. The study employs systematic analysis, comparison, classification, synthesis, and pedagogical modelling. The proposed framework includes six assessment criteria: conceptual knowledge, comprehension and interpretation, practical application, analytical and problem-solving ability, knowledge integration and creation, and reflective and self-regulated learning. Each criterion is operationalised through measurable qualitative and quantitative indicators. A four-level assessment scale — advanced, proficient, developing, and insufficient — is introduced to improve transparency and consistency. The article also proposes a weighted formula for calculating an integrated student knowledge score. Particular attention is given to validity, reliability, fairness, transparency, accessibility, academic integrity, and the pedagogically responsible use of learning analytics and artificial intelligence. The findings indicate that digital assessment becomes educationally meaningful when assessment tasks are aligned with learning outcomes, indicators are observable and measurable, students receive timely feedback, and automated data are combined with professional teacher judgement. The proposed framework may be adapted to different university disciplines, learning management systems, and blended or fully online educational programmes.

digital educational environment \ knowledge assessment \ assessment criteria \ performance indicators \ digital assessment \ learning outcomes \ higher education \ formative assessment \ learning analytics \ assessment validity \ academic integrity \ online learning

Короткий адрес: https://sciup.org/14139302

IDS: 14139302   |   УДК: 378(575.2(045)   |   DOI: 10.33619/2414-2948/131/71

Разработка критериев и показателей для оценки знаний учащихся в цифровой образовательной среде

Цифровая трансформация высшего образования требует разработки систем оценки, способных измерять не только объем знаний, полученных студентами, но и их способность понимать, применять, анализировать, оценивать и создавать информацию в технологически опосредованном контексте. В данной статье рассматриваются теоретические и методологические основы разработки критериев и показателей оценки знаний учащихся в цифровой образовательной среде. В исследовании использованы методы системного анализа, сравнения, классификации, синтеза и педагогического моделирования. Предлагаемая система включает шесть критериев оценки: концептуальные знания, понимание и интерпретация, практическое применение, способность к анализу и решению проблем, интеграция и создание знаний, а также рефлексивное и саморегулируемое обучение. Каждый критерий реализуется с помощью поддающихся измерению качественных и количественных показателей. Для повышения прозрачности и последовательности введена четырехуровневая шкала оценки: продвинутый, профессиональный, развивающий и недостаточный уровни. В статье также предлагается взвешенная формула для расчета интегрального балла знаний учащихся. Особое внимание уделяется обоснованности, надежности, справедливости, прозрачности, доступности, академической честности и педагогически ответственному использованию учебной аналитики и искусственного интеллекта. Полученные результаты показывают, что цифровое оценивание становится значимым с образовательной точки зрения, когда задачи оценивания согласованы с результатами обучения, показатели доступны для наблюдения и измерения, учащиеся получают своевременную обратную связь, а автоматизированные данные объединяются с профессиональной оценкой преподавателя. Предлагаемая система может быть адаптирована к различным университетским дисциплинам, системам управления обучением и смешанным или полностью онлайн-образовательным программам.

Текст научной статьи Developing Criteria and Indicators for Assessing Students’ Knowledge in a Digital Educational Environment

UDC 378(575.2(045)                              

The digital transformation of higher education has changed the ways in which educational content is delivered, learning activities are organised, and students’ achievements are evaluated. Learning management systems, online tests, interactive assignments, electronic portfolios, simulations, videoconferencing platforms, and learning analytics provide educators with considerable amounts of data about student performance. Nevertheless, the availability of digital data does not automatically guarantee accurate or pedagogically meaningful assessment. Traditional assessment has frequently focused on the reproduction of factual information. Such an approach is insufficient in a digital educational environment, where students are expected to search for information, evaluate digital sources, solve problems, participate in collaborative projects, and create new educational products. Consequently, assessment must address different levels and manifestations of knowledge rather than measure only the number of correct answers. Online assessment can be understood as the systematic collection and interpretation of data about students and learning processes for the purpose of making evidence-based conclusions about learning outcomes. A systematic review of online assessment in higher education demonstrates that digital assessment can support feedback, student engagement, and interactive learning, but its effectiveness depends on assessment design and pedagogical integration [1].

The principal difficulty lies in transforming broadly formulated learning outcomes into observable and measurable indicators. For instance, the outcome “the student understands the theoretical concept” is too general for consistent assessment. It must be operationalized through specific actions: defining the concept, explaining its essential characteristics, comparing it with related concepts, providing examples, and using it to interpret a professional problem.

The distinction between assessment criteria and indicators is therefore fundamental. A criterion represents a significant dimension or quality against which performance is judged. An indicator is an observable or measurable manifestation showing the extent to which that criterion has been achieved.

The purpose of this study is to develop and theoretically substantiate a system of criteria and indicators for assessing students’ knowledge in a digital educational environment.

The study addresses the following research questions: What principles should guide the development of digital knowledge assessment? Which criteria adequately represent different levels of student knowledge? Through which indicators can these criteria be measured? How can qualitative and quantitative evidence be integrated into an overall assessment? What conditions ensure the validity, reliability, fairness, and transparency of digital assessment?

A digital educational environment is an integrated system of technological, informational, pedagogical, and organizational resources supporting teaching, learning, communication, assessment, and educational management. It includes not only technological platforms but also learning materials, assessment practices, interaction patterns, feedback mechanisms, and institutional policies.

Digital assessment may be formative or summative, synchronous or asynchronous, automated or teacher-mediated. Its instruments include: online quizzes and examinations, case-based assignments, discussion forums, digital portfolios, simulations and virtual laboratories, project presentations, peer- and self-assessment, oral examinations conducted through videoconferencing, and analytics generated by learning management systems [2, 5].

Technology can improve assessment efficiency, accessibility, feedback speed, and variety. At the same time, it can introduce problems related to digital inequality, academic integrity, privacy, technical failure, and excessive dependence on automatically generated scores. Technology should therefore enhance assessment principles rather than determine them.

Bennett et al. argue that technology shapes assessment design by influencing what evidence can be collected and how students interact with assessment tasks [2].

The implication is that an assessment method should be chosen in relation to the intended learning outcome, not merely because a digital tool is available. Assessment validity depends on the relationship among learning outcomes, assessment tasks, criteria, and evidence. If a course outcome requires students to apply knowledge, a multiple-choice test measuring factual recall cannot serve as the only assessment instrument. Similarly, collaborative competence cannot be assessed solely through an individual final examination (Figure 1).

Learning outcomes

Assessment criteria

Observable indicators

Digital assessment tasks

Evidence of learning

Judgement and feedback

Figure 1. Constructive alignment in digital assessment

The framework is cyclical because assessment results should provide information not only about students but also about the appropriateness of learning outcomes, teaching activities, and assessment tasks. Validity, reliability, and fairness are generally recognised as central assessment quality requirements. Validity concerns whether the assessment supports the intended interpretation of students’ performance. Reliability refers to the consistency of results across tasks, assessors, and conditions. Fairness requires that students have equitable opportunities to demonstrate their knowledge without irrelevant technological, linguistic, social, or physical barriers .

Digital assessment additionally requires: technological accessibility; transparency of algorithms and scoring rules; protection of personal data; compatibility with assistive technologies; mechanisms for handling technical disruption; authentication proportionate to assessment risks; meaningful human oversight. UNESCO’s principles of good practice emphasise technical rigour, ethical responsibility, fairness, and appropriate use of assessment data .

These principles remain applicable when assessment is conducted through digital platforms. The study follows a theoretical and methodological research design. It does not report experimental scores from a particular student sample. Instead, it develops a conceptual framework that can subsequently be tested and validated in different educational contexts. The following methods were employed:

— Theoretical analysis to identify major concepts and quality requirements in educational assessment.

— Comparative analysis to distinguish criteria, indicators, descriptors, and assessment instruments.

— Classification to organise indicators according to levels of cognitive and self-regulated activity.

— Synthesis to combine pedagogical and technological requirements.

— Pedagogical modelling to develop an integrated assessment framework.

— Expert-oriented validation design to specify how the framework should be reviewed before institutional implementation.

Four principles guided the selection of criteria: alignment with expected learning outcomes; observability of student performance; measurability through more than one source of evidence; adaptability to different disciplines and digital platforms. The proposed system is based on eight principles [4, 6-9].

Table 1 PRINCIPLES OF CRITERIA-BASED DIGITAL ASSESSMENT

Principle

Meaning

Practical implication

Outcome alignment

Assessment reflects intended learning outcomes

Every criterion is linked to a course or programme outcome

Validity

Tasks measure the intended knowledge or ability

Recall tests are not used as the sole measure of problem-solving ability

Reliability

Judgements remain sufficiently consistent

Standardised rubrics and assessor moderation are applied

Transparency

Requirements are understandable to students

Criteria and performance descriptors are published in advance

Measurability

Indicators can be observed and documented

Ambiguous indicators such as “knows well” are avoided

Multidimensionality

Different manifestations of knowledge are assessed

Tests are combined with cases, projects, and reflective tasks

Fairness and accessibility

Irrelevant barriers are minimised

Alternative formats and reasonable adjustments are available

Developmental orientation

Assessment supports further learning

Students receive feedback and opportunities for improvement

The principles must operate together. Highly reliable automated testing, for example, may still lack validity if it measures only recognition of information while the learning outcome requires independent analysis. Six criteria are proposed for assessing students’ knowledge in a digital educational environment (Table 2).

Table 2

CRITERIA AND INDICATORS FOR ASSESSING STUDENTS’ KNOWLEDGE

Criterion

Core question

Main indicators

Sources of digital evidence

Conceptual knowledge

What does the student know?

Accuracy of definitions; command of terminology; recognition of facts, principles, and relationships

Online tests, concept maps, short-answer tasks

Comprehension and interpretation

Does the student understand the content?

Ability to explain ideas; classify information; compare concepts; provide examples; summarise arguments

Written responses, recorded explanations, discussion forums

Practical application

Can the student use knowledge?

Appropriate selection of methods; correct execution of procedures; transfer to new situations

Cases, simulations, virtual laboratories, applied assignments

Analysis and problemsolving

Can the student investigate and solve a problem?

Identification of relevant data; evaluation of sources; logical reasoning; justification of decisions

Problem scenarios, research tasks, data analysis

Integration and knowledge creation

Can the student construct a new product or solution?

Synthesis of ideas; originality; interdisciplinary connections; quality of the final product

Projects, portfolios, digital presentations, prototypes

Reflection and self-regulation

Can the student evaluate and improve learning?

Goal setting; self-assessment accuracy; use of feedback; revision of work; learning strategy selection

Reflective journals, portfolio revisions, selfassessment forms

The six criteria form a progression from knowledge reproduction to the independent construction and regulation of knowledge.

Figure 2. Progression of knowledge assessment criteria

The progression does not imply that lower-level knowledge is unimportant. Conceptual accuracy remains necessary for analysis and creation. However, assessment should not stop at factual reproduction. An indicator must describe observable student behaviour. The statement “the student has good analytical knowledge” cannot be scored consistently. It can be operationalised through the following indicators: identifies the central problem; distinguishes relevant from irrelevant information; verifies the credibility of sources; establishes causal relationships; compares alternative explanations; justifies a conclusion with evidence; acknowledges limitations or uncertainty. Indicators may be quantitative, qualitative, or combined [10].

EXAMPLES OF QUANTITATIVE AND QUALITATIVE INDICATORS

Table 3

Assessment dimension

Quantitative indicator

Qualitative indicator

Conceptual accuracy

Percentage of correct answers

Accuracy and appropriateness of terminology

Source use

Number or proportion of relevant sources

Credibility, diversity, and integration of sources

Problem-solving

Number of correctly completed stages

Coherence and justification of the selected strategy

Participation

Frequency of meaningful contributions

Relevance and intellectual quality of contributions

Revision

Number of corrected errors

Extent to which feedback improves the work

Project performance

Proportion of requirements completed

Originality, applicability, and coherence of the product

Activity data require cautious interpretation. A high number of platform logins does not necessarily demonstrate strong knowledge. Time spent online can also be affected by technical conditions, learning preferences, or accessibility needs. Behavioural analytics should therefore support, but not replace, direct evidence of student performance. A four-level scale is proposed to describe the quality of student knowledge.

GENERAL PERFORMANCE DESCRIPTORS

Table 4

Level

General descriptor

Typical evidence

4: Advanced

Demonstrates accurate, integrated, transferable, and independently applied knowledge

Provides well-supported solutions, critically evaluates information, and creates original outcomes

3: Proficient

Demonstrates generally accurate knowledge and applies it in familiar and partly unfamiliar situations

Explains concepts correctly and completes tasks with minor limitations

2: Developing

Demonstrates partial or fragmented knowledge and requires guidance

Reproduces basic information but has difficulty applying or justifying it

1: Insufficient

Demonstrates major conceptual gaps and cannot apply knowledge appropriately

Provides inaccurate, incomplete, or unsupported responses

Descriptors should be adapted to the subject area. In a language course, application may concern communicative performance; in engineering, it may involve technical calculations or design; in medicine, it may relate to clinical reasoning; and in law, it may involve interpretation of norms and construction of legal arguments. The following rubric illustrates how the criteria can be applied to an online case-based assignment.

SAMPLE ANALYTICAL RUBRIC FOR A DIGITAL ASSESSMENT TASK

Table 5

Criterion      Advanced — 4

Proficient — 3      Developing — 2      Insufficient — 1

Conceptual  Uses all relevant

knowledge   concepts accurately

and establishes

Uses most          Demonstrates partial   Demonstrates

concepts accurately  knowledge and some  major

with minor          terminological errors   misconceptions

omissions

Бюллетень науки и практики / Bulletin of Science and Practice Т. 12. №10 2026

Criterion

Advanced — 4

Proficient — 3

Developing — 2

Insufficient — 1

relationships among them

Application

Selects and applies an appropriate method independently in a new context

Applies an appropriate method with minor errors

Applies the method only with guidance or incompletely

Cannot select or apply an appropriate method

Analysis

Critically evaluates evidence and alternative explanations

Analyses relevant information and supports the main conclusion

Analysis is descriptive or partially supported

Conclusions are unsupported or inconsistent

Digital source use

Selects, verifies, and integrates credible sources ethically

Uses mainly credible sources with generally correct attribution

Uses a limited range of sources or inconsistent citation

Uses unreliable or unattributed material

Solution quality

Produces a coherent, feasible, and original solution

Produces a coherent and feasible solution

Produces a partially feasible solution

Does not provide a workable solution

Reflection

Critically evaluates the process and uses feedback to improve the result

Identifies strengths and limitations and makes revisions

Provides general reflection with limited revision

Provides no meaningful reflection or revision

An analytical rubric is preferable when the assessment is intended to provide detailed feedback. It shows where the student performs well and where improvement is needed.

The overall score may be calculated using a weighted model:

n

S = ^ Wi Pi, i=i where: Sis the integrated assessment score; Wiis the weight assigned to criterion i; Pi is the normalised student score for criterion i; ^wi = 1. A possible distribution is presented below.

Table 6 SUGGESTED WEIGHTING OF ASSESSMENT CRITERIA

Criterion

Weight

Conceptual knowledge

0.20

Comprehension and interpretation

0.15

Practical application

0.20

Analysis and problem-solving

0.20

Integration and knowledge creation

0.15

Reflection and self-regulation

0.10

Total

1.00

Suppose a student receives normalised scores of 85, 80, 75, 90, 78, and 88 respectively. The integrated score is: S = (0.20 x 85) + (0.15 x 80) + (0.20 x 75) + (0.20 x 90) + (0.15 x 78) + (0.10 x 88) = 82.5.

The weighting must reflect course priorities. A theoretically oriented course may assign greater weight to conceptual understanding, whereas a practice-based programme may prioritise application and problem-solving. Weights should be approved before assessment and communicated to students. A single assessment instrument cannot capture all dimensions of knowledge. A balanced digital assessment system should therefore triangulate evidence [11-13].

Figure 3. Triangulation of digital assessment evidence

Automated evidence includes quiz results and task-completion data. Performance evidence includes projects, cases, simulations, and portfolios. Dialogic evidence comes from oral examinations, forums, consultations, and presentations. Reflective evidence includes self-assessment, revision histories, and reflective journals. Triangulation reduces the risk of judging knowledge on the basis of a single test score or a potentially misleading platform metric. The proposed framework demonstrates that knowledge assessment in a digital educational environment should be multidimensional. Digital technologies can make assessment more frequent, interactive, and data-informed, but they may also encourage the collection of easily measurable information instead of educationally significant evidence. A major concern is the relationship between measurement and learning. Indicators should not be selected merely because the platform can generate them. Login frequency, number of clicks, or time spent viewing a resource may describe participation, but these metrics do not directly demonstrate understanding. Claims about student knowledge should be primarily based on the quality of completed tasks, explanations, decisions, and created products.

The findings correspond with research showing that online assessment supports learning when it includes authentic tasks, interaction, timely feedback, and opportunities for self-regulation. Gikandi, Morrow, and Davis identified learner engagement, authentic assessment, and continuous feedback as important characteristics of online formative assessment [3].

Digital assessment also requires attention to academic integrity. Traditional high-stakes examinations frequently depend on surveillance and remote proctoring. However, excessive surveillance may create privacy, equity, and trust concerns. An alternative is to use authentic, personalised, and process-oriented tasks that require students to apply knowledge, explain decisions, provide intermediate products, and reflect on revisions. The development of generative artificial intelligence has made assessment redesign even more important. If students can generate standardised answers using AI tools, assessment should place greater emphasis on: application to specific contexts; explanation of reasoning; evaluation of AI-generated content; oral defence of submitted work; documentation of the work process; ethical disclosure of permitted AI use; comparison and revision of different solutions. AI-assisted scoring can increase speed and consistency, particularly for structured tasks. Nevertheless, automated systems may reproduce bias, misinterpret creative responses, or provide insufficiently transparent scores. Human academic judgement must therefore remain part of consequential assessment decisions. Before institutional use, the criteria and indicators should undergo a validation process. Reliability should not be reduced to a single coefficient. For complex projects, inter-rater agreement and moderation may be more informative than internal consistency. For tests, item difficulty, discrimination, and score stability may be appropriate. The proposed framework offers several benefits: alignment between outcomes and assessment; transparent expectations; consistent teacher judgement; integration of formative and summative assessment; support for meaningful feedback; adaptability to different disciplines; reduced dependence on a single assessment instrument [12].

Table 7

RECOMMENDED VALIDATION STAGES

Stage

Procedure

Purpose

Content validation

Review by subject and assessment experts

Determine whether criteria adequately represent learning outcomes

Cognitive review

Interviews or think-aloud sessions with students

Establish whether instructions and descriptors are understood

Pilot implementation

Application to a small student group

Identify technical and pedagogical problems

Inter-rater analysis

Independent scoring by two or more assessors

Examine scoring consistency

Statistical analysis

Item analysis and reliability estimation where appropriate

Evaluate measurement properties

Fairness review

Comparison across relevant student groups

Detect irrelevant barriers or systematic disadvantage

Revision

Modification of tasks, indicators, and descriptors

Improve the framework before wider implementation

However, several limitations should be recognized. First, the framework is theoretical and requires empirical validation. Second, the weighting of criteria cannot be universal because disciplines have different epistemological and professional requirements. Third, digital access and teacher competence may influence implementation. Finally, the use of detailed rubrics may become overly mechanical if assessors treat descriptors as substitutes for professional judgment.

Practical Recommendations: Higher education institutions and teachers should take the following steps when developing digital assessment criteria: begin with clearly formulated and measurable learning outcomes; distinguish knowledge reproduction from comprehension, application, analysis, and creation; use observable indicators rather than general expressions such as “good knowledge”; select assessment instruments after defining criteria, not before; combine tests with authentic tasks, projects, oral explanations, and reflection; publish criteria, weights, and descriptors before students begin the task; use several sources of evidence when making high-stakes decisions; provide timely feedback connected to particular criteria; allow students to revise their work when the assessment is formative; review indicators for accessibility and cultural or technological bias; establish procedures for academic integrity and acceptable AI use; and regularly evaluate the validity and reliability of the assessment system.

Conclusion

The development of criteria and indicators is a central requirement for objective and pedagogically meaningful assessment in a digital educational environment. Digital technologies expand the range of available evidence but do not independently ensure assessment quality. The proposed framework includes six interrelated criteria: conceptual knowledge, comprehension and interpretation, practical application, analysis and problem-solving, knowledge integration and creation, and reflection and self-regulation. Each criterion is represented by observable indicators and can be assessed through multiple forms of digital evidence. A four-level performance scale improves transparency, while a weighted scoring model supports the calculation of an integrated result. Nevertheless, numerical aggregation should be accompanied by qualitative feedback and professional interpretation. Effective digital assessment must be valid, reliable, fair, transparent, accessible, and aligned with learning outcomes. Automated tools and learning analytics should support teacher judgement rather than replace it. Future studies should empirically test the proposed model, examine its psychometric characteristics, and adapt the indicators to specific academic disciplines and cultural contexts.