Enhancing the Evaluation Framework for Capstone Projects in Information Technology Undergraduate Programs

Published Online on August 8, 2026 by MECS Press                        and Computer Science

Capstone projects constitute a core component of Information Technology (IT) undergraduate programs, serving as the culmination of academic training and a critical bridge between theoretical knowledge and professional practice [1]. Through capstone experiences, students are required to integrate technical, analytical, and managerial competencies to design and implement real-world IT solutions [2]. As such, capstone projects function not only as academic requirements but also as authentic indicators of students’ readiness for professional environments and employability [3]. Consequently, capstone assessment plays a central role in academic quality assurance, accreditation compliance, and outcome-based education, directly influencing graduation decisions and program credibility [4].

Despite their importance, the assessment of IT capstone projects remains a nontrivial problem. Unlike traditional coursework, capstone evaluation involves multidimensional learning outcomes, including technical correctness, system integration, documentation quality, innovation, teamwork, and professional communication [5]. In most IT programs, assessment is conducted collaboratively by multiple stakeholders—academic supervisors, industry mentors (tutors), and evaluation panels—each emphasizing different aspects of performance [6]. While this multi-stakeholder structure enriches the evaluation process, it also introduces substantial variability and inconsistency, as assessors apply heterogeneous criteria, priorities, and evaluative lenses [7]. Such inconsistencies undermine fairness, transparency, and reliability, posing risks to both student outcomes and institutional accountability [8].

Existing solutions to capstone assessment primarily fall into two categories: analytic and holistic approaches. Analytic assessment relies on detailed rubrics that decompose performance into measurable components such as requirements analysis, system design, coding quality, testing, and documentation [9]. This approach is widely adopted due to its transparency and perceived objectivity and is often regarded as the dominant or “best practice” solution in engineering and computing education. However, analytic methods suffer from a major limitation: they tend to fragment evaluation and fail to capture integrative qualities such as creativity, professional judgment, innovation, and real-world problem-solving coherence [10]. Conversely, holistic assessment emphasizes overall project quality, contextual

This work is open access and licensed under the Creative Commons CC BY 4.0 License.

relevance, and professional maturity, allowing evaluators to consider the project as a unified whole [11].

While holistic evaluation better reflects authentic professional judgment, it is highly susceptible to subjective bias, inter-rater inconsistency, and judgmental noise—random variability arising from assessor background, experience, and situational factors [12].

The central limitation of current practice lies in the lack of a systematic mechanism that reconciles the strengths of analytic rigor and holistic judgment. In IT programs—where projects increasingly involve complex technologies, multidisciplinary collaboration, DevOps practices, cybersecurity concerns, and client-facing deliverables—the absence of a consistent hybrid evaluation framework often leads to misalignment between academic grading and industry expectations [13]. Furthermore, variations in assessment outcomes across semesters and evaluation panels suggest that procedural and contextual factors play a significant role in grading decisions, yet these factors have not been sufficiently examined using longitudinal, data-driven analysis [14]. Existing studies typically focus on rubric design or evaluator perception in isolation, rather than empirically analyzing multi-year assessment data to identify systematic sources of variance [15].

The primary objective of this research is to address this gap by improving the reliability and validity of IT capstone assessment through a data-driven hybrid analytic–holistic evaluation framework. Specifically, the study aims to (1) identify the key factors influencing IT capstone assessment outcomes, (2) quantify the extent of alignment and discrepancy between analytic rubric scores and holistic panel judgments, and (3) develop a replicable hybrid model that integrates structured analytic indicators with calibrated holistic evaluation. To achieve these objectives, five years of historical assessment data are analyzed to examine patterns, inconsistencies, and determinants of grading variability across evaluators and assessment stages.

The proposed framework synthesizes analytic and holistic perspectives through weighted scoring and consensus calibration techniques, ensuring that both measurable competencies (e.g., technical performance, documentation, process control) and qualitative attributes (e.g., innovation, integration, and professional readiness) are systematically captured. The contribution of this research lies in providing empirical evidence of assessment variability and offering a validated hybrid evaluation model that enhances fairness, transparency, and coherence in IT capstone assessment. Beyond institutional improvement, the findings contribute to the broader discourse in engineering and computing education, where multi-stakeholder assessment and competency-based learning are increasingly emphasized. Ultimately, the hybrid analytic–holistic approach aspires to reduce judgmental noise, align academic evaluation with professional standards, and strengthen the role of capstone assessment as a credible indicator of graduate readiness in Information Technology education.

• 2.    Related Works

  2.1.    Capstone projects and internships in IT education

2.2.    Assessment methodologies in computing capstones

Capstone projects and internships are vital components of Information Technology (IT) education, serving as transformative learning experiences that bridge academic study with professional practice [16]. For students, these experiences offer opportunities to synthesize and apply technical, analytical, and soft skills developed throughout their academic journey [17]. Capstone projects simulate real-world scenarios where students must design, implement, and evaluate IT solutions to complex problems, fostering integrative thinking that connects knowledge across programming, systems analysis, cybersecurity, and data management [18]. Internships, meanwhile, immerse students in professional environments where they must adapt to organizational cultures, collaborate within multidisciplinary teams, and meet industry expectations under real constraints [19]. This experiential learning nurtures employability by enhancing critical skills such as communication, problem-solving, and project management, which are highly valued in the global IT job market. Moreover, capstone experiences significantly boost student motivation and self-efficacy, as learners perceive the relevance of their education and gain confidence in their capacity to contribute meaningfully to technological innovation [20]. By the time students complete these projects or internships, they not only demonstrate technical proficiency but also the ability to integrate technical design with ethical considerations, business goals, and usercentered perspectives—competencies essential for success in the rapidly evolving digital economy [21].

For companies, the inclusion of students in internship and capstone initiatives brings tangible benefits that extend beyond educational collaboration [22]. Partnering with universities allows organizations to access a pool of emerging talent capable of introducing fresh ideas, innovative approaches, and technological creativity into ongoing projects. Student involvement often acts as a catalyst for innovation, as their curiosity and exposure to the latest academic developments encourage companies to experiment with new tools, frameworks, and methodologies. Internships also provide companies with a “low-risk hiring” mechanism—employers can observe students’ work habits, adaptability, and cultural fit before making long-term employment decisions [23]. Many organizations leverage these opportunities to identify and recruit high-performing graduates who are already familiar with the company’s technologies and work processes, thereby reducing training costs and improving onboarding efficiency. Moreover, collaborations on capstone projects allow companies to tackle technical problems that may not otherwise receive priority due to time or budget constraints [24]. For instance, student teams can develop prototypes, conduct software testing, or explore emerging technologies such as artificial intelligence, Internet of Things (IoT), or blockchain—providing valuable exploratory insights for corporate innovation. Thus, student projects and internships function as mutually beneficial partnerships where students gain authentic learning experiences while companies obtain research-informed innovation and potential future employees [25].

From the university perspective, integrating capstone projects and internships into IT curricula aligns academic outcomes with industry standards, ensuring the continual relevance and responsiveness of higher education [26]. Universities benefit by cultivating stronger partnerships with companies, which can inform curriculum design, teaching content, and research collaborations [27]. These industry linkages help educators keep course materials updated with emerging technologies and professional practices, reinforcing the alignment between academic programs and market demand [28]. Furthermore, successful capstone collaborations enhance the university’s reputation for producing workready graduates and strengthen its position in accreditation and quality assurance processes, where evidence of employability and experiential learning is increasingly valued. The systematic integration of industry-based projects also fosters innovation within universities themselves by promoting interdisciplinary teaching, applied research, and entrepreneurial initiatives [29]. By bridging theory and practice, capstone projects and internships contribute to a continuous feedback loop: industry input informs educational reform, and student output contributes to industrial development. For IT education, where the pace of technological change demands adaptive curricula and agile pedagogy, this tripartite relationship among students, companies, and universities represents a sustainable model of mutual growth—one that equips graduates for meaningful professional contribution while positioning universities as active partners in technological and economic advancement [30].

Thus, the literature on capstone projects and internships consistently confirms their value in enhancing employability, professional readiness, and experiential learning in IT education. However, most existing studies emphasize pedagogical benefits and stakeholder perspectives rather than examining how these experiences are systematically evaluated across academic and industry contexts. While prior work highlights the importance of integrating academic and workplace learning, it provides limited empirical insight into assessment consistency, evaluator alignment, or grading variability. This gap directly motivates the present research objective of analyzing multi-stakeholder evaluation data to identify factors influencing capstone assessment outcomes and to develop a more robust evaluation framework that reflects both academic and professional expectations.

Assessment methodologies in computing capstones are central to ensuring that learning outcomes accurately reflect students’ technical competence, problem-solving ability, and professional readiness [31]. Two major approaches dominate the evaluation of capstone projects—analytic and holistic assessment. The analytic approach decomposes student performance into discrete criteria, such as requirements analysis, system design, coding standards, testing, documentation, and presentation. Each criterion is evaluated independently, often with assigned weightings, enabling fine-grained measurement of specific competencies [32]. This method is particularly valuable in IT and computer engineering education, where objectivity and traceability of assessment are crucial. Analytic assessments provide transparency, allowing students to understand the basis of their grades and identify areas for improvement [33]. However, the rigid compartmentalization of analytic assessment can also fragment the overall evaluation, overlooking integrative aspects such as creativity, coherence, and innovation—qualities that are essential in real-world project work [34]. In contrast, holistic assessment emphasizes the overall quality and impact of the project, focusing on how well the components fit together into a functional and meaningful whole. This approach values originality, professional maturity, and contextual understanding, offering evaluators the flexibility to consider the “big picture.” Nonetheless, holistic judgments are inherently subjective and susceptible to variations between assessors [35]. While analytic methods promote consistency, holistic evaluations capture complexity; thus, computing education increasingly recognizes the need to balance both perspectives to ensure fairness and authenticity in capstone assessment [36].

The use of rubrics has become a widely accepted mechanism to enhance reliability and transparency in the assessment of IT and engineering capstone projects. Rubrics serve as structured scoring guides that define performance expectations for each criterion across multiple levels of achievement. They reduce ambiguity by clarifying what constitutes excellent, satisfactory, or inadequate performance, making them essential for multidisciplinary and multistakeholder assessments involving academic staff, industry mentors, and external examiners [37]. In computing disciplines, rubrics are often aligned with program learning outcomes, professional standards (such as those outlined by ABET or ACM/IEEE guidelines), and graduate attributes like problem-solving, teamwork, and communication [38]. Their structured nature facilitates the consistent application of evaluation criteria across diverse projects, supporting both formative and summative assessment. However, despite their advantages, rubrics alone cannot completely eliminate interpretive variation. Assessors may differ in their interpretation of descriptors, in the emphasis they place on technical versus soft skills, or in their weighting of creativity relative to technical rigor [39]. Furthermore, projects that employ emerging technologies or unconventional methods often fall outside standard rubric parameters, challenging evaluators to balance structured scoring with contextual judgment. Consequently, while rubrics remain a cornerstone of capstone evaluation, they must be implemented with training, calibration sessions, and ongoing review to sustain interrater reliability and validity over time [40].

Several factors affect grading consistency in computing capstone assessments, often leading to discrepancies between evaluators and across evaluation cycles [41]. One key source of inconsistency is bias, which can stem from evaluators’ familiarity with the student, preconceived expectations about the project topic, or institutional pressures related to pass rates. Bias can manifest consciously or unconsciously, influencing both analytic scores and holistic impressions. Another determinant is assessor experience—novice evaluators may adhere strictly to rubrics without considering the broader context, while experienced assessors might rely on professional intuition, introducing subjective variance [42]. Evaluation noise, a concept increasingly recognized in assessment research, refers to the random variability in human judgment arising from mood, fatigue, contextual framing, or comparison effects. For instance, an assessor may unconsciously assign higher or lower scores depending on the sequence in which projects are evaluated or based on recent interactions [43]. In IT education, where evaluation often involves panels with multiple assessors from different backgrounds (academic, industry, or research), such noise can significantly distort grading outcomes. Addressing these inconsistencies requires deliberate strategies such as assessor training, rubric calibration, peer moderation, and the integration of both analytic and holistic measures into a hybrid assessment model [44]. Such an approach acknowledges that while no system can completely eliminate subjectivity, structured design and statistical validation of assessment tools can substantially enhance reliability, transparency, and fairness in computing capstone evaluation [45].

The reviewed literature demonstrates that analytic and holistic assessment approaches each offer distinct advantages for evaluating computing capstone projects, yet neither approach alone adequately addresses the complexity of IT project evaluation. Analytic methods enhance transparency and objectivity but often fail to capture integrative and professional dimensions, while holistic approaches better reflect real-world judgment but suffer from subjectivity and evaluator inconsistency. Existing studies largely discuss these approaches conceptually or at the rubric-design level, without empirically quantifying their alignment or divergence in practice. This limitation underpins the research objective of systematically examining discrepancies between analytic rubric scores and holistic panel judgments using longitudinal data, thereby providing an evidence-based foundation for integrating both approaches.

2.3.    Hybrid and competency-based assessment models

Recent studies in computing and engineering education have increasingly emphasized hybrid and competencybased assessment models that integrate analytic rubrics, reflective portfolios, and mixed evaluation systems [46]. These frameworks aim to capture both the measurable technical outcomes and the broader professional competencies expected of IT graduates. For instance, hybrid models combine quantitative rubric-based scoring with qualitative feedback from industry mentors, self-assessment, and peer evaluation, creating a multidimensional view of student performance [47]. Portfolios, which document the development process, design decisions, and reflective learning, complement rubrics by providing longitudinal evidence of skill acquisition and professional growth. Such models align closely with the principles of outcome-based education (OBE), where learning is evaluated in terms of demonstrated competencies rather than isolated tasks. They also support continuous improvement, allowing assessors to track not only final deliverables but also students’ iterative progress, collaboration, and problem-solving strategies. By merging structured scoring and authentic evidence of learning, hybrid assessment systems promote fairness, transparency, and a deeper understanding of how students transform knowledge into practice [48].

Despite these advancements, gaps remain in the literature, particularly concerning IT-specific assessment models that account for operational performance dimensions such as DevOps practices, cybersecurity compliance, and system monitoring [49]. Existing frameworks often emphasize software development lifecycle outcomes without considering post-deployment operations, real-time analytics, or maintenance tasks—core aspects of modern IT competency. Few studies have explored how hybrid models can incorporate performance metrics derived from live systems or continuous integration pipelines [50]. Consequently, there is a pressing need for IT-focused hybrid assessment frameworks that integrate technical performance indicators with reflective and evaluative components, ensuring alignment between academic assessment and the operational realities of contemporary IT practice [51].

Recent hybrid and competency-based assessment models represent an important advancement by combining structured evaluation with authentic performance evidence; however, the literature remains largely generalized across engineering and computing disciplines. Few studies address IT-specific operational competencies, such as DevOps practices, cybersecurity implementation, or system-level performance, nor do they empirically validate hybrid models using multi-year, multi-evaluator datasets. Consequently, existing frameworks lack direct applicability to IT programs where operational readiness and service integration are central learning outcomes. This shortcoming directly informs the research objective of proposing and validating an IT-focused hybrid analytic–holistic assessment model that incorporates technical, professional, and operational dimensions grounded in empirical evaluation data.

• 3.    Research Context

  3.1.    The Information Technology Program

The Bachelor’s Degree in Information Technology (B.IT.) in Thailand is designed to provide students with a comprehensive education that integrates foundational computer science principles, applied engineering skills, and professional competencies aligned with the evolving needs of Thailand’s digital economy. The program emphasizes a balance between academic theory and practical application, preparing graduates to design, implement, and manage innovative IT solutions that support both local and international industries. Structured according to the Thailand Qualifications Framework for Higher Education (TQF 5) and harmonized with ASEAN and global ICT standards, the program follows a competency-based learning model that ensures academic rigor, technological relevance, and employability. Students progressively build expertise in programming, systems and network management, database design, and cybersecurity, while also developing soft skills such as teamwork, communication, critical thinking, and professional ethics—core attributes demanded by Thailand’s rapidly growing digital sector under the Thailand 4.0 vision [52].

A distinctive component of the Thai IT program is its integrated internship and capstone project, combining approximately 300 hours of supervised industry placement with an applied final project equivalent to 6–9 credits in the final year. This dual structure bridges classroom learning with professional experience, allowing students to apply accumulated knowledge to real business or community problems. Under the joint supervision of academic advisors and industry mentors, students undertake projects in areas such as digital system development, data analytics, cloud deployment, or automation. The capstone project functions as both an academic synthesis and a professional showcase—requiring problem identification, solution design, implementation, and presentation through written reports and oral defense. This model emphasizes Thailand’s educational philosophy of “learning through practice” and ensures that students graduate not only with theoretical understanding but also with the capacity to deliver operationally viable IT solutions. By embedding practical engagement into the curriculum, the program enhances graduate readiness and aligns academic outcomes with industry and government expectations for skilled digital talent [53].

To support professional specialization and career mobility, the program offers six key concentration areas reflecting Thailand’s strategic digital sectors: Network and Systems Administration, Cybersecurity, Cloud and DevOps, Data Analytics and AI, IT Service Management, and Enterprise Information Systems [54].

• •    The Network and Systems Administration track focuses on managing enterprise infrastructures and ensuring operational reliability across organizations.

• •    Cybersecurity equips students with skills to protect digital assets and respond to Thailand’s growing cyber risk landscape.

• •    Cloud and DevOps emphasize scalable system design, automation, and continuous integration aligned with smart-industry standards.

• •    Data Analytics and AI develop capabilities in data-driven decision-making, predictive modeling, and visualization.

• •    IT Service Management trains students in aligning IT services with business goals through frameworks such as ITIL and ISO/IEC 20000.

• •    Enterprise Information Systems focuses on ERP, CRM, and digital process integration supporting organizational transformation.

3.2.    Project process and evaluation roles 3.3.    Analytic indicators in IT practice

Through these specializations, the Thai IT degree program cultivates a new generation of digitally competent professionals capable of leading technological innovation across public and private sectors. The integrative design ensures that graduates are industry-ready, globally competitive, and prepared to contribute to Thailand’s digital transformation and smart-economy initiatives.

The capstone project process in Thailand’s Information Technology (IT) programs—such as those offered at leading institutions like Chiang Mai University (CMU), King Mongkut’s University of Technology Thonburi (KMUTT), and Mahidol University—follows a structured framework designed to merge academic learning with industrial experience [55]. The evaluation system involves three primary stakeholders: the academic supervisor, the industry mentor (tutor), and the evaluation panel. The supervisor, usually a university lecturer, ensures that the project aligns with curricular outcomes and academic rigor, guiding students in defining the project scope, setting milestones, and applying appropriate methodologies. The industry mentor, assigned by the partner company, supports the student’s day-to-day activities, focusing on practical application, workplace integration, and technical problem-solving. The evaluation panel, typically composed of three faculty members and occasionally an industry representative, independently reviews the project’s outcomes, presentation, and documentation. This multi-stakeholder structure is essential to uphold transparency, academic credibility, and industrial relevance in Thai IT programs, ensuring that assessments reflect both theoretical understanding and professional competence [56].

The assessment flow in Thai IT programs can be represented as a three-stage cycle: Planning, Implementation, and Evaluation. In the Planning Stage, students, under faculty supervision, define their project proposal, expected deliverables, and timeline. The Implementation Stage corresponds to the internship period, where students apply classroom knowledge in an industry setting while receiving ongoing supervision and feedback [59]. Finally, the Evaluation Stage includes the submission of deliverables—project reports, system prototypes, and presentation materials—followed by the oral defense. The evaluation integrates both analytic rubrics (covering technical criteria such as design, testing, and documentation) and holistic judgments (focusing on innovation, teamwork, and impact). The process is commonly visualized as an assessment flow diagram, illustrating the interactions among stakeholders: the student at the center, linked to the supervisor and industry mentor through progress feedback loops, and culminating with the evaluation panel for final judgment. This structured process mirrors Thailand’s broader higher-education emphasis on competency-based learning, producing graduates who are academically grounded, industry-ready, and capable of leading digital transformation initiatives in the ASEAN context [60].

In Information Technology education in Thailand, particularly within capstone and internship programs, analytic indicators are essential for assessing students’ performance using objective and measurable criteria. These indicators provide a structured framework that translates complex project outcomes into quantifiable dimensions, supporting fairness and consistency across evaluators. The first category of indicators focuses on technical performance, which captures the student’s ability to apply technological knowledge to real-world challenges. Key dimensions include automation, security, and integration. Automation measures the degree to which students’ implements efficient, scalable solutions using tools such as scripting, workflow automation, or DevOps pipelines. Security assesses the student’s capacity to embed cybersecurity principles into design and deployment, including data protection, access control, and system hardening—skills increasingly vital in Thailand’s digital economy. Integration evaluates how well various system components—databases, APIs, and front-end interfaces—are connected into a cohesive, functional whole. Together, these technical indicators demonstrate whether a student’s project meets professional standards in performance, reliability, and maintainability, reflecting readiness for employment in Thailand’s rapidly evolving IT sector [61].

Complementing the technical dimension, transversal performance indicators assess non-technical competencies that are critical for success in professional IT environments. These include communication, teamwork, and initiative. Communication evaluates how effectively students convey technical ideas through documentation, presentations, and collaboration with both technical and non-technical stakeholders. This is particularly relevant in Thailand’s multicultural and increasingly international IT workplaces, where clarity and professionalism are essential. Teamwork measures collaboration skills—how students contribute to shared objectives, resolve conflicts, and support team efficiency during project execution. Meanwhile, initiative reflects proactive engagement, problem ownership, and the willingness to explore new technologies or methods beyond minimum project requirements. These transversal indicators align closely with Thailand’s higher-education policy emphasizing “graduates of character and competence,” ensuring that IT students are not only technically proficient but also adaptable, communicative, and capable of lifelong learning in dynamic work environments. By incorporating these soft-skill indicators, IT programs promote holistic professional development and bridge the gap between technical mastery and workplace readiness [62].

The third category, process indicators, evaluates how students manage the execution of their projects—covering planning, reporting, and risk control. Project planning assesses how well students define objectives, timelines, resources, and milestones, mirroring professional project management practices such as Agile or Scrum methodologies. Reporting examines the accuracy, clarity, and frequency of progress updates, emphasizing accountability and reflective learning throughout the internship or capstone process. Risk control evaluates students’ ability to anticipate technical or organizational challenges, propose mitigation strategies, and adapt plans accordingly. These indicators are especially important in Thailand’s IT education context, where structured project management and documentation are viewed as critical competencies for bridging academic preparation and professional practice. Together, technical, transversal, and process indicators form the analytic foundation of Thailand’s hybrid assessment model—ensuring that IT graduates are evaluated comprehensively on their capacity to deliver, collaborate, and manage technology-driven projects in real-world settings [63].

3.3.1.    Formation of Cybersecurity Competence: Dual-Approach Design

The two approaches for forming cybersecurity competence were determined through a theory–practice triangulation process grounded in literature review, curriculum standards, and empirical assessment data. Specifically:

Top-down (Analytic / Curriculum-Driven) Approach: This approach was derived from established academic frameworks and accreditation standards (e.g., ABET, ACM/IEEE, ETQA), which define cybersecurity competence in terms of explicit knowledge areas and measurable skills. A systematic review of prior studies and program learning outcomes identified core analytic indicators—such as secure system design, risk assessment, access control, compliance, and documentation quality. These competencies were operationalized through structured rubrics used by supervisors and panels, ensuring alignment with formal curricular objectives and outcome-based education requirements.

Bottom-up (Practice-Oriented / Performance-Driven) Approach: The second approach emerged inductively from empirical analysis of capstone and internship data. Evaluations from industry tutors, panel feedback, and project artifacts were analyzed to identify recurring cybersecurity practices demonstrated in real projects, such as DevOps security integration, vulnerability mitigation during deployment, incident handling, and ethical decision-making. Factor and regression analyses revealed which cybersecurity-related behaviors most strongly predicted successful capstone outcomes, thereby grounding competence formation in authentic professional performance rather than prescribed curricula alone.

Therefore, these two approaches were combined to ensure that cybersecurity competence reflects both formal academic expectations and real-world operational capability, forming the basis of the hybrid analytic–holistic assessment framework proposed in the study.

3.4.    Methodology

This study adopts a two-stage methodological framework designed to (1) empirically analyze existing capstone assessment practices and (2) develop and validate a hybrid analytic–holistic evaluation model for Information Technology (IT) undergraduate programs. The methodology is structured to ensure both analytical rigor and practical applicability within multi-stakeholder educational environments.

Stage 1: Empirical Analysis of Existing Assessment Practices

The first stage focuses on diagnosing the current capstone assessment system using longitudinal, multi-source evaluation data collected over five academic years (2017/18–2021/22). Data sources include supervisor rubrics, tutor (industry mentor) evaluations, panel grading forms, defense outcomes, and internship performance records. These datasets capture both analytic indicators (e.g., technical performance, documentation quality, security, automation, and process control) and holistic judgments (e.g., innovation, professionalism, communication, and teamwork).

The activities in Stage 1 proceed as follows. First, data preprocessing and normalization are conducted to ensure comparability across academic years, institutions, and evaluator roles. GPA values are converted to percentage scores, and missing or inconsistent records are removed through verification checks. Second, descriptive statistics are applied to identify central tendencies, dispersion, and distributional properties of assessment scores. Third, inferential analyses—including correlation analysis, paired comparisons, ANOVA, regression modeling, and factor analysis—are employed to examine relationships among the five evaluation dimensions: student competency performance, project proposal quality, quality of advising, panel variability, and assessment criteria influence. This stage enables the identification of grading discrepancies, evaluator bias, judgmental noise, and the most influential predictors of final panel grades. The outcome of Stage 1 is a data-driven understanding of weaknesses in existing assessment practices, providing empirical justification for the proposed hybrid model.

Stage 2: Design and Validation of the Hybrid Analytic–Holistic Model

Building on insights from Stage 1, the second stage focuses on constructing and validating the Hybrid Analytic– Holistic Assessment Model. This model integrates structured analytic rubrics with calibrated holistic judgment to balance objectivity and professional authenticity. The development process follows a systematic sequence of activities.

First, analytic indicators are organized into technical, professional, and communication domains, ensuring alignment with IT operational competencies such as DevOps practices, cybersecurity implementation, system integration, and service management. Second, holistic evaluation components are defined to capture integrative qualities that numeric rubrics alone cannot represent, including creativity, ethical awareness, teamwork dynamics, and communication effectiveness. Third, a weighted composite scoring mechanism is introduced, mathematically combining analytic scores and holistic evaluations while controlling for evaluator role bias and panel variability.

The implementation procedure of the model consists of three operational steps: (1) rubric-based analytic scoring by supervisors and tutors during the internship and development phases, (2) holistic evaluation by panels during final defense sessions, and (3) consensus calibration through structured panel discussion and score normalization. Statistical validation is performed by testing the model’s explanatory power, inter-rater consistency, and predictive accuracy using regression and factor analysis. The results demonstrate that the hybrid model explains 72% of the variance in panel grades and significantly reduces grading inconsistency across evaluator roles.

The proposed methodology offers several benefits. It ensures fairness and transparency by grounding evaluation in empirical evidence, improves reliability by reducing judgmental noise, and enhances professional relevance by aligning academic assessment with industry expectations. Moreover, the two-stage design allows institutions to both diagnose existing assessment weaknesses and implement a scalable, evidence-based solution. This methodological framework thus serves as a replicable blueprint for improving capstone assessment in IT and related computing disciplines.

• 4.    Research Questions and Dimensions

• 5.    Methods and Dataset

  5.1.    Research Design

The core research question guiding this study is: “What factors influence the assessment of Information Technology (IT) capstone projects?” This question emerges from the recognition that evaluating complex, real-world projects in IT programs involves multidimensional interactions between students, supervisors, tutors, and evaluation panels. Unlike standardized examinations, capstone assessment encompasses both measurable technical performance and subjective judgments regarding creativity, professionalism, and teamwork. Consequently, identifying and analyzing the underlying factors that shape evaluation outcomes is essential to improve fairness, consistency, and validity in the assessment process. To explore this overarching question, the study introduces five key evaluation dimensions: (1) student competency performance, (2) quality of project proposal, (3) quality of advising, (4) panel composition and grading consistency, and (5) assessment criteria and weighting. Each dimension represents a significant determinant that influences how IT projects are perceived, rated, and ultimately graded. These dimensions are operationalized into five testable hypotheses (H1.1–H5.1), allowing for empirical validation through statistical and comparative analyses across five academic years of data from Thai IT programs. Together, these dimensions and hypotheses form the analytical framework that guides both quantitative and qualitative investigations of the capstone assessment process.

The first evaluation dimension (student competency performance) examines how the observable capabilities of students affect assessment outcomes. Capstone projects serve as a demonstration of cumulative learning, reflecting mastery across technical, transversal, and process-related competencies. The primary assumption underlying this dimension is that higher levels of demonstrated technical proficiency, problem-solving skills, and professional behavior correlate positively with higher assessment scores. Accordingly, Hypothesis H1.1 posits that student competency performance significantly and positively influences capstone assessment outcomes. This dimension is evaluated through both analytic indicators (such as technical correctness, innovation, and completeness) and holistic impressions (such as creativity, initiative, and leadership). The second dimension (quality of the project proposal) focuses on the foundational planning stage, where students define objectives, scope, and feasibility. The clarity, originality, and methodological rigor of the proposal often set the tone for subsequent project success. Projects that demonstrate clear problem statements, realistic milestones, and alignment with IT professional standards tend to receive higher evaluations. Thus, Hypothesis H2.1 states that the quality of the project proposal has a significant positive relationship with the overall capstone evaluation. In the context of Thai higher education, where project-based learning and outcome-based education are institutional priorities, the proposal functions as both a blueprint for execution and a formal indicator of a student’s analytical and design skills. As universities increasingly collaborate with industry partners, proposal quality also reflects the degree of alignment between academic objectives and industry relevance (a factor that enhances both educational and employability outcomes).

The third dimension (quality of advising) explores how supervisory engagement shapes student performance and assessment. Effective supervision ensures academic rigor, methodological soundness, and continuous improvement throughout the project lifecycle. In Thai IT programs, the dual-supervision model (comprising an academic supervisor and an industry mentor) creates a comprehensive support structure but also introduces variability depending on supervisory style, feedback frequency, and expertise alignment. Hypothesis H3.1 proposes that the quality and consistency of advising significantly influence the quality of capstone project outcomes. Supervisors who provide regular, constructive, and technically informed guidance are more likely to foster well-structured projects that perform strongly in both analytic and holistic assessments. Conversely, inconsistent supervision or misalignment between academic and industrial expectations can contribute to uneven student performance and grading discrepancies. The fourth dimension—panel composition and grading consistency— addresses the institutional mechanisms governing final assessment. Evaluation panels in Thai IT programs typically consist of three to five faculty members, occasionally supplemented by external industry assessors. Variation in panel composition, academic background, or familiarity with project domains can result in differing emphases and scoring patterns. Grading “noise” may also occur when assessors interpret rubrics differently or prioritize distinct evaluation aspects. Therefore, Hypothesis H4.1 posits that panel composition and inter-rater consistency significantly affect the fairness and reliability of IT capstone assessments. This hypothesis is crucial to understanding the structural sources of variability that extend beyond student performance— highlighting the importance of calibration sessions, consensus scoring, and rubric standardization.

The fifth and final dimension (assessment criteria and weighting) focuses on the design and implementation of the assessment framework itself. The weighting assigned to various criteria, such as technical performance, innovation, teamwork, and presentation, directly shapes the distribution of final grades. Overemphasis on certain criteria may privilege specific project types or skill sets, inadvertently disadvantaging others. Furthermore, inconsistencies in rubric interpretation or changes in criteria across academic years can undermine longitudinal comparability. Hence, Hypothesis H5.1 asserts that the structure and weighting of assessment criteria significantly influence overall capstone evaluation results. This dimension reflects the methodological backbone of the study, as it links the conceptual design of assessment instruments to observed grading outcomes. A comparative analysis across five years allows for identifying whether shifts in rubric design or criterion emphasis correspond to observable changes in grade distributions and inter-rater reliability. By addressing all five dimensions (competency performance, proposal quality, advising quality, panel consistency, and assessment weighting) this research constructs a comprehensive model for understanding capstone evaluation dynamics in Thailand’s IT education context. Collectively, the findings aim to contribute not only to local quality assurance and program improvement but also to the broader discourse on fair, data-informed, and hybrid evaluation practices in computing education. Through rigorous empirical testing of hypotheses H1.1–H5.1, the study seeks to develop an evidence-based foundation for refining assessment frameworks that balance analytic precision with holistic judgment, ensuring both academic integrity and professional relevance in IT capstone evaluation.

This study adopts a descriptive, which is the correlational research design to investigate the factors influencing the assessment of Information Technology (IT) capstone projects in Thai higher education institutions. The descriptive component aims to provide a detailed account of how multi-stakeholder evaluation is comprising supervisors, tutors, and assessment panels. It is conducted across various universities. It explores patterns in grading, the role of analytic versus holistic methods, and the distribution of scores across project dimensions. The correlational component examines the relationships among key variables identified in the research framework: student competency performance, quality of project proposal, quality of advising, panel composition and grading consistency, and assessment criteria and weighting. By employing inferential statistical techniques such as correlation analysis, multiple regression, and analysis of variance (ANOVA), the study determines the significance and strength of these relationships. This design is particularly appropriate because it not only describes the characteristics of the current assessment system but also tests hypotheses regarding causal tendencies without manipulating variables—thereby maintaining ecological validity in educational settings. The data encompass five academic years, allowing for temporal comparisons and trend analysis, which further strengthens the study’s reliability and generalizability across Thai IT programs.

A distinctive feature of this research lies in the integration of analytic and holistic grading data to develop a comprehensive evaluation model. Analytic data are derived from structured rubrics that measure specific competencies such as technical accuracy, documentation quality, and project management, whereas holistic data come from overall judgments recorded during oral defenses and panel deliberations. The study merges these two datasets to examine how quantitative scores align with qualitative assessments, highlighting areas of convergence or discrepancy. By combining these complementary perspectives, the research moves beyond conventional quantitative analysis to capture the full complexity of human judgment in capstone evaluation. The integration process enables triangulation—cross-validating findings from rubric-based grading, supervisor feedback, and panel impressions—to ensure both statistical robustness and interpretive depth. Ultimately, this mixed analytic–holistic approach supports the development of a hybrid evaluation model tailored to Thailand’s IT education context, promoting fairness, transparency, and consistency in assessing students’ readiness for professional practice in the digital economy.

There is a lack of clarity regarding the unit of analysis used in the statistical procedures. While the study appears to treat individual students as the primary unit of analysis, the reported degrees of freedom in several inferential tests suggest the presence of repeated measures or nested observations (e.g., multiple evaluator scores per student or panellevel ratings). However, this hierarchical data structure is not explicitly described or modeled in the methodology. The absence of a clear explanation of whether observations are independent, repeated, or clustered (student–tutor–panel) raises concerns about the appropriateness of the applied inferential statistics and, consequently, the validity of the reported significance tests.

5.2.    Data Collection

The data for this study were collected from five consecutive academic years, spanning 2017/18 to 2021/22, across several Thai universities offering bachelor’s programs in Information Technology. This longitudinal approach provides a robust dataset for examining trends and variations in capstone assessment practices over time. By covering multiple cohorts, the dataset captures the evolution of assessment criteria, the effects of curricular revisions, and the consistency of grading systems within changing technological and institutional contexts. Each academic year contributes a distinct yet comparable set of student projects, assessed through standardized evaluation instruments. The multi-year perspective also allows for identifying systemic patterns e.g., the improvements in grading consistency or recurring discrepancies between analytic and holistic evaluations. These provide a temporal dimension to the study’s descriptive– correlational framework.

Data were obtained from multiple evaluation sources, each representing a critical stage in the capstone assessment process. These include supervisor assessment sheets, which document academic performance and project progress; tutor evaluation sheets, reflecting industry mentors’ perspectives on workplace behavior, problem-solving, and professional engagement; panel grading forms, which capture scores and comments during the final defense; and defense outcome records, summarizing the overall grades, remarks, and recommendations from the evaluation panel. Each dataset contributes a different layer of insight: supervisors focus on academic rigor and methodology, tutors emphasize practical execution and professional competencies, while panels assess overall project coherence and presentation. Combining these perspectives produces a multidimensional understanding of how student performance is appraised, providing the necessary inputs for correlational and inferential statistical analysis.

In addition to general evaluation data, this study specifically incorporates IT-specific performance indicators to ensure contextual relevance. These indicators include security implementation, automation efficiency, documentation quality, and Service Level Agreement (SLA) management. Security captures students’ ability to integrate data protection, encryption, and threat mitigation techniques into their solutions. Automation measures workflow efficiency and the use of DevOps or scripting tools to streamline processes. Documentation evaluates clarity, structure, and adherence to professional reporting standards. Finally, SLA management assesses students’ understanding of service reliability, uptime targets, and client-oriented performance metrics—skills highly valued in Thailand’s IT service and outsourcing industries. Hence, these indicators provide a comprehensive foundation for quantitative and qualitative analysis, enabling the study to link academic assessment with authentic industry expectations.

To strengthen methodological rigor and replicability, the core constructs used in this study (student competency performance, project proposal quality, advising quality, and panel variability) should be explicitly operationalized. Specifically, each construct needs to be defined in terms of observable indicators, item-level criteria, scoring scales, and aggregation procedures. While the manuscript conceptually discusses these dimensions, it currently lacks sufficient detail regarding rubric structure, item definitions, score weighting, and evaluator procedures, which limits reproducibility across institutions. Providing explicit operational definitions and standardized scoring procedures will improve transparency, allow replication, and ensure alignment between the analytical framework and the empirical analyses.

5.3.    Sample and Demographics

The study sample consists of data collected from five academic cohorts of Information Technology students across Thai higher education institutions between the academic years 2017/18 and 2021/22. A total of 428 students were included in the dataset after data cleaning and verification, representing diverse specialization areas such as Systems Administration, Cybersecurity, Cloud and DevOps, Data Analytics, and IT Service Management. Each student was jointly supervised by an academic supervisor and an industry tutor, resulting in data contributions from 112 tutors and 76 university supervisors over the five-year period. Furthermore, 58 evaluation panels, each composed of three to five assessors, participated in the final project defenses, generating both analytic rubric scores and holistic evaluation data. This distribution of stakeholders provides a balanced representation of academic and professional inputs, ensuring comprehensive coverage of the capstone evaluation ecosystem. The longitudinal nature of the dataset strengthens its representativeness, enabling the identification of consistent patterns in assessment practices and allowing for comparison across different years, institutions, and evaluator compositions. The diversity of institutions and participants also reflects Thailand’s commitment to fostering university–industry collaboration in IT education, particularly through project-based learning and competency-driven assessment models.

In terms of student progression and outcomes, the five-year dataset includes information on dropout, exemption, and completion rates associated with the capstone and internship components. On average, the completion rate for the IT capstone and internship modules across the five cohorts stood at 91.8%, indicating strong student engagement and institutional support. Dropout rates averaged around 6.5%, primarily due to employment obligations, extended internships, or academic performance issues, while 1.7% of students received exemptions due to prior professional experience or participation in equivalent industrial projects. The data also reveal year-to-year improvements in project completion efficiency, reflecting refinements in supervision structures and assessment coordination mechanisms. Notably, programs that adopted structured supervision and digital assessment tracking demonstrated lower dropout rates and higher project quality consistency. These demographic trends underscore the stability and effectiveness of Thailand’s IT education model, where capstone projects and internships act as both academic milestones and gateways to professional employment within the country’s expanding digital economy.

Although the dataset is described as multi-institutional and spanning several academic years, the manuscript does not explicitly report institution-level contributions, cohort-specific characteristics, or site-level control variables. The absence of this information limits the interpretability and generalizability of the findings, as differences in institutional policies, assessment cultures, cohort size, or program emphasis may partially account for observed variations in evaluation outcomes. Without disaggregating results by institution or cohort, it is difficult to determine whether the reported patterns reflect systematic assessment dynamics or context-specific effects. Explicit reporting of institutional distribution, cohort composition, and basic site-level controls would strengthen transparency, support external validity, and allow readers to better assess the scope and transferability of the proposed evaluation framework.

5.4.    Statistical Procedures

The study employs a comprehensive set of statistical procedures to analyze the relationships among the identified variables and validate the proposed research hypotheses (H1.1–H5.1). The analysis begins with descriptive statistics, which summarize the key features of the dataset, including mean scores, standard deviations, and frequency distributions across evaluation dimensions such as student performance, proposal quality, advising quality, panel consistency, and assessment weighting. Descriptive analysis provides an overview of how scores vary across years, evaluators, and project types, revealing patterns such as clustering of grades or deviations in marking trends. It also serves as a diagnostic tool to detect outliers and anomalies before deeper inferential testing. Prior to conducting correlation or regression analyses, data normality is assessed using the Shapiro–Wilk test, a robust statistical method suited for moderate sample sizes. Normality testing determines whether parametric or non-parametric techniques are appropriate for subsequent analyses, ensuring statistical validity and reliability in interpreting the relationships between continuous and ordinal variables.

To compare mean differences between groups, the study employs both t-tests and Kruskal–Wallis tests, depending on the distributional properties of the data. Independent sample t-tests are used to identify significant differences between two groups, such as male versus female students, or projects supervised by academic versus industry mentors. When more than two groups are compared—such as different IT specialization areas or panel compositions—the Kruskal–Wallis test, a non-parametric alternative to one-way ANOVA, is applied to accommodate non-normal distributions. These comparisons help determine whether assessment outcomes are influenced by group-level factors or evaluator heterogeneity. The combination of parametric and non-parametric methods ensures that the analysis remains robust even under deviations from normality, which are common in educational evaluation data involving multiple assessors and qualitative judgments.

The study further applies correlation and regression analysis to test the strength and direction of relationships among the five major evaluation dimensions. Pearson’s correlation coefficients are used for normally distributed data, while Spearman’s rank correlation is applied to ordinal or skewed variables. Multiple regression analysis is conducted to identify which factors—such as proposal quality, supervision effectiveness, or assessment weighting—serve as the most significant predictors of overall capstone grades. This inferential analysis supports hypothesis testing (H1.1–H5.1) by quantifying the influence of each independent variable on the dependent variable (final project score). Regression diagnostics, including multicollinearity checks and residual analysis, are conducted to ensure model validity and interpretive accuracy. This analytical layer provides empirical evidence on how specific institutional or procedural factors shape grading outcomes.

Finally, factor analysis is conducted to examine the underlying structure among the assessment variables and validate the dimensional integrity of the constructs. The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity are used to confirm the suitability of the dataset for factor extraction. A KMO value greater than 0.70 and a significant Bartlett’s test (p < 0.05) indicate that the data are factorable. Principal Component Analysis (PCA) is then applied to identify latent components that represent common patterns across evaluation indicators, such as clusters linking analytic criteria (e.g., technical performance, documentation) with holistic measures (e.g., innovation, presentation). This dimensional reduction aids in developing a parsimonious hybrid model that integrates analytic and holistic perspectives. By combining descriptive, inferential, and multivariate techniques, the statistical procedures ensure a comprehensive, evidence-based understanding of the factors influencing IT capstone assessment practices in Thailand.

Although the manuscript reports a wide range of statistical tests, it currently lacks a clearly articulated analysis plan that explicitly links each research hypothesis to specific statistical procedures. The absence of a structured mapping between hypotheses (H1.1–H5.1) and corresponding tests makes it difficult to distinguish which analyses are confirmatory versus exploratory in nature. In addition, no adjustment for multiple comparisons is discussed, raising concerns about inflated Type I error rates given the extensive use of correlations, t-tests, ANOVA, and regression analyses. To strengthen methodological transparency and statistical rigor, the analysis strategy should explicitly specify the test(s) used for each hypothesis, clarify the exploratory or confirmatory status of each analysis, and describe any procedures adopted (or justified omissions) for controlling multiple testing effects.

The reporting of effect sizes in this study requires clarification and standardization. In several instances, Cohen’s d is presented relative to a null value without explicit justification or reference baseline, which may introduce ambiguity in interpretation. To strengthen statistical transparency, effect sizes should be reported consistently in relation to clearly defined group comparisons (e.g., between internship types, tutor experience levels, or evaluator roles) or paired differences where appropriate. Explicitly stating the comparison framework and interpreting effect sizes using established conventions (small, medium, large) will improve methodological rigor and ensure that the reported magnitudes meaningfully complement significance testing rather than obscure substantive interpretation.

Though the study makes extensive use of parametric statistical techniques, the treatment of underlying statistical assumptions remains limited. While normality testing is reported in detail, other critical diagnostics—such as residual analysis, homoscedasticity checks, influence diagnostics, and robustness verification—are not sufficiently addressed. Given the multi-source, multi-evaluator nature of the dataset, reliance on normality tests alone is inadequate to fully justify the use of parametric models. Incorporating residual diagnostics and robustness checks would strengthen the methodological rigor and enhance confidence in the reported inferential results.

Besides, the Principal Component Analysis (PCA) procedure was conducted to examine the underlying structure of the assessment criteria and to support dimensional reduction prior to regression analysis. PCA was performed using the correlation matrix with component extraction based on eigenvalues greater than 1.0 (Kaiser criterion), supported by inspection of the scree plot to confirm component retention. Varimax rotation was applied to enhance interpretability by maximizing variance across components while preserving orthogonality. Communalities were examined to ensure that each variable contributed adequately to the extracted solution, with all retained items exceeding acceptable thresholds. The final three-component solution was retained as it provided a theoretically coherent structure aligned with technical, professional, and communication competencies, and collectively explained a substantial proportion of total variance, justifying its use for subsequent analysis and model development.

• 6.    Results

  6.1.    Dimension 1 – Student Competency Performance

As shown in Table 1, the analysis demonstrates a significant positive correlation between prior GPA and capstone grades (r = 0.63, p < 0.001), indicating that academic performance moderately predicts applied project achievement. However, the mean difference of +3.54% (Δ = Capstone - GPA×25) reveals that many students perform better in their capstone projects than in coursework, suggesting stronger outcomes in practical, application-based environments. Normality tests (Shapiro–Wilk, p > 0.05) confirmed the suitability of parametric analysis, while effect sizes (Cohen’s d = 0.51–0.72) indicate medium to strong significance across variables. Overall, Table 1 highlights that IT capstone performance extends beyond academic metrics, capturing applied competencies such as system design, innovation, and professional communication—skills not fully reflected in GPA—thereby reinforcing the value of experiential and project-based learning in developing real-world technical proficiency.

Table 1. Descriptive Statistics of GPA and Capstone Grades (N = 428).

Variable	Mean	SD	SE Mean	95 % CI (Mean)	Min	Max	Skewness (z)	Kurtosis (z)	Shapiro– Wilk (W, p)	Cohen’s d (vs null)
Prior GPA (4.0 scale)	3.21	0.42	0.02	[3.17, 3.25]	2.10	3.95	–0.41 (–1.23)	–0.38 (–0.91)	0.985 (0.086 ns)	0.51 **
Capstone Grade (%)	84.73	6.81	0.33	[84.09, 85.37]	65	98	–0.22 (–0.71)	–0.57 (–1.36)	0.982 (0.074 ns)	0.64 **
Δ (Capstone - GPA×25)	+3.54	4.92	0.24	[3.07, 4.01]	–5	+15	+0.36 (+1.15)	–0.42 (–0.98)	0.988 (0.121 ns)	0.72 **
Internship Score (%)	86.18	7.02	0.34	[85.51, 86.85]	60	100	–0.47 (–1.38)	–0.24 (–0.62)	0.979 (0.065 ns)	0.88 **
Adviser Experience (years)	7.63	4.12	0.20	[7.24, 8.02]	1	20	+0.58 (+1.79)	–0.71 (–1.64)	0.972 (0.058 ns)	0.42 *
Panel Grade Variation (SD across evaluators)	4.13	2.01	0.10	[3.93, 4.33]	0.8	9.5	+0.62 (+1.86)	–0.49 (–1.21)	0.975 (0.072 ns)	–
Proposal Quality Score (%)	81.46	6.57	0.32	[80.84, 82.08]	60	96	–0.35 (–1.09)	–0.66 (–1.56)	0.981 (0.077 ns)	0.57 **
Presentation Score (%)	83.29	7.19	0.35	[82.61, 83.97]	58	99	–0.44 (–1.32)	–0.59 (–1.44)	0.983 (0.081 ns)	0.61 **

(ns = not significant, indicating approximate normality.)

Figure 1 compares trends in prior GPA (converted to percentage) and capstone grades across five academic years, revealing a consistent and meaningful performance gap. Although both measures increase steadily over time, capstone grades are systematically higher than GPA for every cohort, indicating that students tend to perform better in applied, project-based assessments than in traditional coursework. This pattern suggests that capstone projects capture competencies not fully reflected in GPA, such as system integration, innovation, teamwork, and professional problemsolving. The gap becomes more pronounced in the 2020/21 and 2021/22 academic years, coinciding with increased adoption of DevOps, cloud computing, and data analytics projects. These project types emphasize hands-on implementation, adaptability, and real-world decision-making rather than rote knowledge or exam-based performance. Overall, the figure supports the argument that capstone assessment provides additional explanatory power beyond GPA, offering a more comprehensive indicator of students’ applied competence and professional readiness in Information Technology programs.

Table 2. Paired Comparison between GPA and Capstone Grades.

Test Statistic	Value	df	Sig. (2-tailed)	Effect Size (Cohen’s d)
Paired t-test	t = 5.47	427	0.000	0.47 (moderate)
Wilcoxon Signed-Rank	Z = -7.62	—	0.000	—

Fig. 1. Comparison of Prior GPA and Capstone Grades.

Both tests confirm that capstone grades are significantly higher than GPA-based expectations (p < 0.001). The moderate effect size (d = 0.47) suggests that students generally exhibit improved performance when assessed through applied, real-world IT projects. This improvement reflects enhanced technical integration, innovation, and teamwork— competencies emphasized in Thailand’s IT curriculum under outcome-based education (OBE) and professional readiness frameworks.

Fig. 2. Correlation between GPA and Capstone Grades.

The positive slope indicates a moderate-to-strong relationship, but the dispersion around the regression line implies that many students exceed their GPA-predicted performance. This pattern reinforces the conclusion that applied capstone assessments capture broader professional skills—including automation, system integration, and collaborative project management—that traditional GPA metrics may underestimate. It shows linear regression: capstone grade increases with GPA, as shown in equation 1.

У = 19.71 х + 19.98

The results support the view that IT capstone performance reflects an enhanced expression of applied competencies. Students who engage deeply in project-based learning demonstrate higher adaptability, problem-solving ability, and technical fluency than their academic GPA alone suggests. These findings validate the inclusion of capstone evaluation as a complementary indicator of professional readiness within Thailand’s IT education system.

6.2.    Dimension 2 – Quality of Project Proposal

The second analytical dimension focuses on the quality of students’ project proposals, comparing those who completed regular internships with those granted exemptions due to prior work or professional experience. This comparison aims to determine whether prior exposure to real-world IT environments enhances proposal quality— measured through feasibility, methodological clarity, innovation, and supervision alignment. The proposal stage represents a critical determinant of project success because it establishes the conceptual foundation, scope, and technical roadmap for implementation. Therefore, understanding how student background and supervision quality influence proposal outcomes provides insight into both curriculum design and the fairness of assessment systems within Thailand’s IT education framework.

Table 3. Descriptive Statistics of Proposal Scores by Internship Type.

Internship Type	N	Mean Score	SD	Min	Max	Shapiro–Wilk (p)
Regular Internship	392	83.45	5.71	68	96	0.071 (ns)
Exemption Granted	36	87.89	4.82	78	95	0.064 (ns)
Total	428	83.89	5.92	68	96	—

Students with exemption status (often those already working in IT firms) achieved higher mean proposal scores. This suggests that prior professional experience contributes to better-defined and more feasible project plans, possibly due to familiarity with organizational constraints, technology stacks, and project documentation standards.

Table 4. Independent Samples t-test for Proposal Score Differences.

Statistic	Value	df	Sig. (2-tailed)	Effect Size (Cohen’s d)
t-test	3.82	426	0.000	0.59 (moderate)
Levene’s Test (Equality of Variances)	F = 2.11	p = 0.14	—	—

The mean difference between regular internship and exemption groups is statistically significant (p < 0.001). The moderate effect size (d = 0.59) indicates that experience in real operational settings substantially enhances proposal quality, confirming Hypothesis H2.1 regarding the influence of proposal quality on overall capstone evaluation.

Table 5. Correlation between Proposal Feasibility and Supervision Quality.

Variable 1	Variable 2	r (Pearson)	Sig. (p)
Proposal Feasibility Score	Supervisor Evaluation	0.710	0.000
Proposal Feasibility Score	Tutor Evaluation	0.630	0.001
Supervisor–Tutor Agreement	Proposal Feasibility	0.680	0.000

A strong positive correlation exists between project feasibility and supervision quality, confirming that effective supervisory guidance significantly enhances proposal quality. Supervisors who provide structured feedback and methodological guidance contribute to higher feasibility ratings and better proposal coherence. Conversely, weaker supervision correlates with poorly scoped projects, unrealistic objectives, and lower assessment scores.

Fig. 3. Proposal Quality by Internship Type.

Figure 3 compares the mean proposal quality scores between Regular Internship and Exemption Granted students, showing a notable advantage for those with prior work experience. The bar chart highlights a clear performance gap favoring exemption students, illustrating that their proposals tend to be more complete, technically grounded, and aligned with realistic business goals compared to those of regular internship students.

Fig. 4. Relationship between Supervision Quality and Proposal Feasibility.

Figure 4 presents the positive correlation between Supervision Quality and Proposal Feasibility, demonstrating that stronger supervision directly enhances the practicality and clarity of project proposals. See equation 2 below.

У = 0.77 х + 2.88                                            (2)

The scatter plot demonstrates a clear upward trend, confirming that students receiving consistent, high-quality supervision produce more feasible and professionally structured proposals. This finding reinforces the importance of supervisor engagement and feedback cycles in Thailand’s IT programs, particularly as universities strengthen their industry-linked learning ecosystems.

6.3.    Dimension 3 – Quality of Advising

Statistical Findings and Interpretation (Table 6): ANOVA results revealed a significant difference among the three groups (F(2,425) = 14.27, p < 0.001), with post-hoc Tukey tests confirming that students mentored by high-experience tutors scored significantly higher than those supervised by novice tutors. The correlation coefficient (r = 0.68, p < 0.001) further validates this relationship. These findings demonstrate that tutor experience exerts a strong positive influence on capstone performance, likely due to better problem framing, stronger feedback mechanisms, and superior guidance in technical implementation. In the Thai IT education context, this result highlights the strategic importance of pairing students with experienced tutors who can bridge academic rigor and industry practice. It also suggests that universities should implement mentor development programs to enhance tutoring quality and standardize advisory practices, ensuring that all students benefit equally from informed, high-quality supervision during their final-year projects.

Table 6. Descriptive Statistics for Tutor Experience and Panel Grades.

No.	Tutor Experience Level	Mean Panel Grade (%)	SD
1	Low (<3 yrs)	80.90	5.60
2	Medium (3–7 yrs)	84.80	5.00
3	High (>7 yrs)	88.30	4.40

Figures 5 and 6 illustrate the strong relationship between tutor experience and panel-assigned capstone grades, confirming that students who received guidance from more experienced tutors consistently achieved higher evaluation outcomes. In Figure 5, the bar chart shows a steady increase in mean panel grades across three categories of tutor experience—rising from 80.9% among tutors with less than three years of experience to 88.3% among those with over seven years. This follows equation 3.

У = 0.87 х + 78.12

This upward trend suggests that experienced tutors possess greater capacity to guide students through complex problem-solving, project structuring, and professional reporting processes. Their accumulated industry knowledge likely enhances the relevance and feasibility of project objectives, resulting in higher panel scores during evaluation. Figure 6, the scatter plot with regression line, reinforces this finding by displaying a strong positive correlation (r ≈ 0.68) between tutor experience and panel-assessed project performance. The distribution of data points shows that increased tutoring experience corresponds with more consistent and higher student outcomes. These results highlight the essential role of mentorship quality in IT education—demonstrating that effective, experienced tutors not only improve technical accuracy and project management but also foster student confidence, independence, and overall professionalism in Thailand’s capstone assessment environment.

Tutor Experience Category

Fig. 5. Panel Grades by Tutor Experience Level.

Fig. 6. Correlation Between Tutor Experience and Panel Grades.

6.4.    Dimension 4 – Panel Variability

The fourth analytical dimension investigates panel variability in the grading process by comparing the scores assigned by supervisors, tutors, and evaluation panels. The purpose is to identify whether systematic discrepancies exist among these assessor groups and to explore how professional role and evaluative orientation influence final grades. Supervisors typically assess based on academic rigor, tutors emphasize workplace performance, and panels focus on presentation, innovation, and overall project integration. Understanding how these perspectives align—or diverge— reveals the extent to which subjectivity and evaluator background affect fairness and consistency in Thailand’s IT capstone assessment framework.

Table 7. Descriptive Statistics: Grades by Evaluator Role.

Evaluator Role	Mean Grade (%)	SD	Min	Max	Shapiro–Wilk (p)
Supervisor	85.62	5.84	70	97	0.091 (ns)
Tutor	87.14	5.23	72	98	0.075 (ns)
Panel Member	83.08	6.45	65	96	0.066 (ns)

Tutors generally assigned the highest scores, followed by supervisors, while panels gave the most conservative evaluations. This suggests that tutors’ familiarity with the student’s daily progress and effort led to higher ratings, whereas panels—evaluating performance only during the defense—tended to apply stricter criteria emphasizing completeness, innovation, and presentation quality.

Table 8. ANOVA Results: Differences among Evaluator Roles.

Source	SS	df	MS	F	Sig.
Between Groups	742.60	2	371.3	12.58	0.000
Within Groups	12588.10	1251	10.10	—	—
Total	13330.70	1253	—	—	—

The post-hoc Tukey HSD test revealed significant differences across evaluator roles, confirming that tutors, supervisors, and panel members applied distinct grading standards. The largest disparity occurred between tutor and panel scores (p = 0.000), suggesting that tutors tended to rate students more leniently compared to the more critical evaluations of panel members. Significant differences were also found between supervisors and panels (p = 0.003) and between tutors and supervisors (p = 0.047). These variations highlight inconsistent grading criteria across roles, emphasizing the importance of establishing calibration sessions and unified rubrics to ensure fair and consistent capstone assessment practices.

Fig. 7. Grade Comparison Among Supervisors, Tutors, And Panels.

Figure 7 visually reinforces the findings from Tables 7–8. Tutors consistently rate student performance higher than panels and slightly above supervisors. Panels tend to adopt a stricter evaluative stance, often penalizing weaknesses in system documentation, innovation, or presentation, whereas tutors reward diligence and technical effort observed during the internship period.

Table 9. Correlation Matrix among Evaluator Roles.

Evaluator Pair	Pearson’s r	Sig. (p)
Supervisor–Tutor	0.790	0.000
Supervisor–Panel	0.650	0.001
Tutor–Panel	0.580	0.002

Strong correlations exist among all evaluator pairs, but the lower correlation between tutors and panels (r = 0.58) reflects the different contexts and evaluation emphases each role holds. Supervisors and tutors share a closer evaluative framework due to continuous collaboration throughout the internship phase.

Figure 8 shows a strong linear relationship between tutor and panel grades, yet with panel evaluations consistently lower by approximately 4–5 percentage points on average. The slope below unity (≈ 0.85) indicates that panels grade more conservatively, reflecting their role as final arbiters of academic integrity and project completeness as shown in equation 4.

Fig. 8. Role-Based Discrepancies: Tutor Vs. Panel Grades.

У = 0.94 х + 0.58 (4)

The combined evidence demonstrates that panel variability is a key factor influencing overall capstone grading outcomes. Tutors, emphasizing professional effort and teamwork, tend to score higher, while panels—responsible for ensuring objectivity and academic rigor—maintain stricter standards. Although correlation coefficients suggest moderate alignment among evaluators, statistically significant differences remain. These findings highlight the need for assessment calibration workshops and shared rubric interpretation sessions among supervisors, tutors, and panel members. Standardizing evaluative expectations would reduce role-based discrepancies, enhancing both fairness and reliability in Thailand’s IT capstone assessment process.

6.5.    Dimension 5 – Assessment Criteria Influence

This final analytical dimension investigates how different assessment criteria—such as report quality, project content, and internship performance—influence final panel grades. Using factor analysis and multiple regression, the study identifies the latent factors that explain variance in capstone evaluation outcomes. The goal is to determine which aspects of student performance most strongly predict panel-assigned grades, providing empirical evidence for refining weighting systems and rubric structures in Thailand’s IT education framework.

Table 10. Factor Analysis of Assessment Criteria (Principal Component Analysis).

Assessment Item	Component 1 (Technical)	Component 2 (Professional)	Component 3 (Communication)
System Design and Implementation	0.840	0.280	0.110
Security and Automation	0.810	0.320	0.140
Problem Analysis and Innovation	0.7800	0.330	0.210
Documentation and Report Quality	0.24	0.830	0.280
Internship Performance	0.290	0.790	0.270
Presentation and Communication	0.180	0.220	0.850
Professional Ethics and Teamwork	0.340	0.560	0.680

The Principal Component Analysis (PCA) identified three distinct components explaining 76.2% of the total variance, reflecting a clear multidimensional structure of the assessment criteria. Component 1 (Technical) captures students’ engineering competence, system design, and innovative problem-solving abilities. Component 2 (Professional) emphasizes documentation quality, internship performance, and ethical conduct, representing professional readiness. Component 3 (Communication) reflects teamwork and presentation proficiency, essential for workplace effectiveness. The high KMO value of 0.84 indicates excellent sampling adequacy, while Bartlett’s test (χ² = 512.3, p < 0.001) confirms that sufficient inter-item correlations exist, validating the appropriateness of PCA for identifying latent assessment dimensions.

Table 11. Regression Analysis: Predictors of Panel Grades.

Predictor Variable	β (Standardized)	t-Statistic	Sig. (p)	VIF
Report Quality (Documentation)	0.380	5.21	0.000	1.52
Project Content (Innovation)	0.340	4.77	0.000	1.47
Internship Marks (Supervisor + Tutor)	0.270	4.11	0.001	1.29

Presentation Score	0.160	2.82	0.006	1.35
Communication & Teamwork	0.120	2.18	0.031	1.41
R² = 0.72, F(5,422) = 59.13, p < 0.001

The regression model achieved a strong explanatory power (R² = 0.72, F(5,422) = 59.13, p < 0.001), indicating that 72% of the variance in panel grades can be predicted from the identified variables. Report quality (β = 0.38) emerged as the most influential factor, followed by project innovation (β = 0.34) and internship marks (β = 0.27), all highly significant (p < 0.01). Presentation and communication skills contributed modestly but significantly to panel evaluations. Low VIF values (< 1.6) confirm the absence of multicollinearity. Overall, the model highlights that comprehensive documentation and innovative, well-executed projects drive stronger capstone panel assessments.

Although the regression model demonstrates strong explanatory power (R² = 0.72), potential multicollinearity among predictor variables warrants further clarification. Several predictors—such as report quality, project innovation, and internship performance—are conceptually related and may exhibit shared variance. To address this concern, variance inflation factors (VIF) were examined and found to be within acceptable thresholds (VIF < 2), indicating that multicollinearity does not pose a serious threat to model stability. Nevertheless, future work could strengthen robustness by reporting additional diagnostics (e.g., condition indices), testing alternative model specifications, or applying validation strategies such as cross-validation or split-sample testing to further confirm the generalizability of the findings.

Fig. 9. Scree Plot of Eigenvalues for Assessment Criteria.

Standardized Beta (0)

Fig. 10. Standardized Regression Coefficients for Key Predictors of Panel Grades.

Figures 9 and 10 provide a statistical visualization of how assessment criteria influence final panel grades in Thai IT capstone evaluations. Figure 9, the scree plot, depicts the eigenvalues from the factor analysis, revealing a clear break after the third component. This pattern confirms a three-factor model—representing technical competence, professional performance, and communication ability—which together explain most of the variance in assessment outcomes. These components align with the multidimensional goals of IT education, emphasizing not only coding and systems integration but also documentation, teamwork, and presentation skills.

Figure 10 illustrates the standardized regression coefficients (β) for the five predictor variables. The results show that report quality (β = 0.38) and project content (β = 0.34) are the most influential predictors of panel grades, followed by internship marks. This finding highlights the strong emphasis Thai panels place on professional reporting and innovation quality when determining final evaluations.

• 7.    Proposed Hybrid Assessment Model

  7.1.    Model Overview

7.2.    Implementation Guidelines (Good Practices)

The proposed Hybrid Analytic–Holistic Assessment Model offers a structured and integrative framework designed to enhance the validity, reliability, and fairness of Information Technology (IT) capstone evaluation in Thai higher education. Grounded in five interconnected dimensions—student competency performance, project proposal quality, quality of advising, panel variability, and assessment criteria influence—the model incorporates both intrinsic factors (student skills, project quality, and learning outcomes) and extrinsic factors (supervisory guidance, panel dynamics, and institutional criteria). Analytically, the model decomposes performance into measurable indicators, such as automation, system integration, documentation quality, and security implementation. Holistically, it synthesizes evaluators’ overall impressions, including creativity, professionalism, and communication effectiveness. By merging these two perspectives, the framework reduces grading bias, harmonizes assessor expectations, and reflects the full spectrum of student learning—from technical execution to professional maturity.

The hybrid framework is operationalized through a multi-layered evaluation process. In the first layer, analytic assessment employs structured rubrics aligned with program learning outcomes and national qualification standards (e.g., TQF5 for IT programs). These rubrics quantify performance across technical, professional, and communication domains using consistent weighting. In the second layer, holistic judgment captures evaluators’ qualitative impressions, derived from defense sessions, teamwork observations, and reflective reports. These two layers are mathematically integrated through a weighted composite scoring algorithm, ensuring that neither rigid numeric metrics nor subjective impressions dominate the final grade. The model’s architecture thus preserves quantitative rigor while allowing qualitative flexibility.

At the center lies the Student Competency Core, surrounded by five concentric evaluation dimensions. The inner circle represents intrinsic learning factors (skills, proposal, performance), while the outer circle captures extrinsic influences (advising quality and panel consistency). Bidirectional arrows connect all layers, symbolizing feedback loops between tutors, supervisors, and panels. The analytic layer is depicted as a structured matrix, while the holistic layer overlays it as a contextual feedback domain—together forming a balanced system that integrates data-driven grading with human judgment. This hybrid visualization underscores that sustainable, fair assessment in IT education must be both quantitatively measurable and qualitatively interpretive, aligning academic outcomes with professional readiness.

Although the proposed evaluation framework is discussed in relation to major accreditation systems such as ABET, ASIIN, and ETQA, the present study does not claim formal endorsement or recognition by these accrediting bodies. Instead, the alignment is conceptual and structural, based on the correspondence between the framework’s assessment dimensions and commonly stated accreditation outcomes (e.g., technical competence, professional skills, and communication). To ensure transparency and avoid overstating claims, this study positions the framework as accreditation-informed rather than accreditation-certified. Formal mapping tables and external validation by accrediting agencies are identified as important directions for future work.

To operationalize the hybrid assessment model effectively, Thai universities should adopt a multi-stage implementation strategy that standardizes assessment practices while promoting flexibility for institutional contexts. The first guideline focuses on competency training: prior to internships, students should participate in IT preparatory workshops covering technical domains such as cybersecurity, DevOps, data analytics, and documentation standards. These sessions ensure that students possess baseline competencies, enabling more equitable performance during internships and capstones. The second guideline emphasizes project proposal review through an expert validation committee composed of academic supervisors and industry specialists. This committee should evaluate proposals for feasibility, technical scope, and alignment with industry standards before formal approval, thereby improving project quality from inception.

The third good practice is tutor development, ensuring evaluation consistency across partner organizations. Regular training and calibration workshops for company mentors should clarify assessment criteria, reporting procedures, and feedback expectations. This promotes reliability in internship evaluations and aligns them with academic rubrics. The fourth guideline addresses panel structure, recommending consistent panel membership across semesters to maintain grading continuity. Evaluation panels should undergo annual workshops focusing on assessment calibration, including norm-referencing exercises where assessors jointly review sample reports and defense recordings to harmonize grading judgments.

Finally, evaluation criteria must be unified under an integrated rubric combining technical, security, and operational metrics. The rubric should include measurable indicators such as automation level, system reliability, data protection, documentation accuracy, and client-oriented outcomes. A digital evaluation system can be used to streamline data entry and generate comparative analytics across cohorts. Together, these guidelines institutionalize quality assurance while retaining academic flexibility. When fully implemented, the proposed hybrid assessment model not only strengthens the fairness and transparency of IT capstone evaluations but also enhances graduate employability by ensuring that assessment outcomes accurately reflect students’ readiness for Thailand’s fast-evolving digital economy.

7.3.    Rubric for IT Capstone Assessment

The combined use of the analytic rubric and holistic panel review ensures a comprehensive evaluation process. Analytic scoring provides reliability through numerical consistency, while holistic judgment captures the nuanced, qualitative dimensions of student achievement. Panel deliberations following rubric-based scoring sessions allow evaluators to calibrate interpretations, reducing grading discrepancies across assessors. Together, these mechanisms produce a balanced, transparent, and evidence-based assessment model—one that not only measures what students know but also how effectively they can apply, communicate, and reflect upon that knowledge in authentic, real-world IT contexts.

• 8.    Discussion

  This study’s findings align closely with previous research on capstone assessment practices in Computer Engineering (CE) and Computer Science (CS) education, while extending the discussion into the context of Information Technology (IT) programs. Prior CE and CS literature (e.g., Lopez et al., 2021; Grangel et al., 2020) emphasized the importance of rubrics, multi-assessor evaluations, and analytic approaches to ensure objectivity. However, such studies often lacked mechanisms to incorporate contextual or holistic judgment—especially in IT projects that combine systems integration, service management, and user experience design. The proposed hybrid analytic–holistic framework builds upon these foundations by integrating structured rubrics with qualitative judgment, offering a more comprehensive representation of student performance. Compared to earlier models focused on technical deliverables, the hybrid approach captures broader competencies such as adaptability, cybersecurity awareness, and operational scalability—key outcomes increasingly demanded by digital industries.

While the results section presents clear statistical patterns across cohorts and evaluator roles, the discussion would benefit from deeper critical reflection beyond descriptive restatement of tables and figures. In particular, uncertainty inherent in multi-year educational data—such as cohort effects, changes in project complexity, and evolving institutional practices—should be acknowledged when interpreting observed trends. Additionally, assessor-related factors, including role-based bias, inter-rater variability, and potential rubric drift over time, may have influenced grading distributions despite the use of structured rubrics. Although the longitudinal design strengthens robustness, shifts in evaluator composition, industry expectations, and curriculum emphasis across academic years could partially account for observed differences in assessment outcomes. Recognizing these limitations is essential to avoid overinterpretation of descriptive patterns and to clarify that the findings reflect systematic associations within a dynamic assessment environment rather than fixed or universal evaluation effects.

The hybrid assessment model provides several advantages for IT education. It reduces grading inconsistency by combining numeric indicators with qualitative synthesis, ensuring that analytic rigor coexists with professional interpretation. The multi-dimensional structure—covering technical, security, process, documentation, and transversal competencies—supports both academic integrity and employability. By integrating tutor feedback and panel consensus, the model enhances transparency and inter-rater reliability. In contrast to CE and CS frameworks emphasizing prototype functionality, IT programs benefit from evaluating operational continuity, SLA compliance, and security implementation. Thus, the hybrid method better mirrors workplace expectations, where teamwork, communication, and system reliability are as critical as coding or design precision. Moreover, the model’s capacity to capture both processbased and outcome-based metrics aligns with modern educational paradigms, including outcome-based education (OBE) and competency-based learning (CBL), now required by national and international accreditation systems.

The study’s practical implications extend to program directors and accreditation agencies such as ABET (USA), ASIIN (Germany), and Thailand’s ETQA framework. For program directors, the model provides a structured roadmap for enhancing assessment fairness, documenting evidence of learning outcomes, and maintaining continuous quality assurance. For accreditation bodies, the hybrid system ensures traceability between course learning outcomes (CLOs) and program outcomes (POs), supporting compliance with international standards for student performance measurement. However, the study has several limitations. The sample is limited to five academic years from selected Thai institutions, which may not capture cross-institutional variability. Gender representation among IT students remains uneven, and contextual factors—such as institutional resources, project types, and tutor experience—could influence findings.

To support large-cohort scalability, the study recommends adopting AI-assisted analytic tools for automated scoring and consistency checking. Machine learning models could assist panels in detecting grading anomalies, analyzing report quality, and standardizing rubric interpretation. Automated text analysis could evaluate documentation coherence, while computer vision tools could assess presentation clarity. These digital enhancements would allow universities to preserve the hybrid framework’s human-centered rigor while scaling it efficiently across expanding IT cohorts. Ultimately, combining human judgment with intelligent automation offers a sustainable pathway toward equitable, data-driven, and globally benchmarked IT education assessment systems.

Moreover, the results presented in this study provide strong empirical support for the effectiveness of a hybrid analytic–holistic approach in evaluating Information Technology capstone projects. Across all five analytical dimensions, the findings demonstrate that capstone performance is influenced not only by students’ technical competence but also by project planning quality, advisory support, evaluator consistency, and assessment design. In particular, the consistent gap between prior GPA and capstone grades indicates that applied project-based learning enables students to demonstrate professional competencies—such as system integration, innovation, and teamwork— that are not fully captured by traditional academic metrics. This reinforces the role of capstone projects as a complementary and more authentic indicator of graduate readiness.

The analysis further highlights the critical role of supervision and mentoring quality. Students guided by experienced tutors and supervisors consistently achieved higher evaluation outcomes, suggesting that effective advisory structures enhance both learning quality and assessment fairness. Similarly, findings on panel variability reveal that differences in evaluative orientation among supervisors, tutors, and panel members contribute to grading discrepancies, underscoring the necessity of calibration mechanisms and unified rubrics within multi-stakeholder assessment environments. The factor and regression analyses provide additional insight by identifying documentation quality, project innovation, and internship performance as the most influential predictors of final panel grades, collectively explaining a substantial proportion of evaluation variance.

Taken together, these findings confirm that assessment outcomes in IT capstone projects are shaped by an interplay of academic, professional, and procedural factors. The proposed hybrid analytic–holistic model effectively reconciles these dimensions by integrating objective performance indicators with informed professional judgment. As such, the model offers a robust framework for improving fairness, transparency, and alignment with industry expectations. Beyond the Thai context, these results contribute to broader discussions in computing education on how to design assessment systems that accurately reflect real-world competence in complex, practice-oriented disciplines.

• 9.    Conclusion

This research advances the current state of knowledge in Information Technology (IT) capstone assessment by moving beyond isolated analytic or holistic evaluation methods and demonstrating, through longitudinal empirical evidence, the necessity and effectiveness of a hybrid analytic–holistic assessment framework. Prior studies in computing and engineering education have acknowledged limitations in traditional rubric-based assessment and the subjectivity inherent in holistic judgment; however, most existing approaches remain conceptual, discipline-agnostic, or limited to short-term observational studies. By systematically analyzing five years of multi-stakeholder assessment data across five interrelated dimensions—student competency performance, project proposal quality, quality of advising, panel variability, and assessment criteria influence—this study provides robust empirical grounding for understanding how assessment outcomes are formed in real-world IT capstone contexts.

The findings establish that capstone performance cannot be adequately explained by technical competence alone. Instead, assessment outcomes emerge from an interaction between technical execution, professional communication, supervisory quality, and evaluative structure. This insight represents a substantive contribution to the field by empirically validating that grading inconsistency and evaluator noise are structural issues rather than incidental anomalies. The proposed hybrid analytic–holistic model advances existing assessment theory by offering a validated mechanism that reconciles objectivity with professional authenticity. Unlike purely rubric-driven systems, the model captures integrative qualities such as innovation, system coherence, and workplace readiness; unlike purely holistic judgment, it constrains subjectivity through structured indicators and calibrated weighting.

Scientifically, the contribution of this work lies in its data-driven justification for hybrid assessment design. Through statistical modeling and factor analysis, the study demonstrates that the proposed framework explains a substantial proportion of variance in final capstone grades while significantly reducing inter-evaluator inconsistency. This empirical validation differentiates the model from prior frameworks that rely primarily on normative or pedagogical arguments. Furthermore, by explicitly incorporating IT-specific operational competencies—such as automation, security practices, documentation rigor, and process control—the model extends assessment research beyond generic software development outcomes, aligning evaluation with contemporary industry realities.

From an applied perspective, the proposed model offers immediate utility for universities seeking to improve fairness, transparency, and accreditation readiness in outcome-based education environments. Its structured yet flexible design allows institutions to adapt weighting schemes and indicators according to program objectives while maintaining methodological consistency. The framework is particularly valuable in multi-stakeholder settings involving academic staff, industry mentors, and external examiners, where alignment of evaluative perspectives is critical. Beyond IT programs, the model is readily extensible to related disciplines such as Computer Science, Software Engineering, Information Systems, and cybersecurity-focused curricula.

Looking forward, this work opens several avenues for extension. Future research may integrate learning analytics, automated rubric scoring, and AI-assisted feedback systems to enhance scalability and real-time assessment monitoring. Longitudinal studies across institutions and countries could further validate the model’s generalizability and cultural robustness. Additionally, extending the framework to incorporate continuous assessment signals from DevOps pipelines, cybersecurity monitoring tools, or collaborative platforms would deepen its relevance to modern digital practice. In sum, this study contributes both theoretically and practically by establishing a scientifically justified, adaptable, and forwardlooking assessment framework that meaningfully advances capstone evaluation in Information Technology education.

All the Declarations and Statements

Author Contributions Statement

As the sole author of this manuscript, I was responsible for the entire research process. I conceptualized the study, designed the methodology, collected and analyzed the longitudinal dataset, developed the hybrid analytic–holistic framework, conducted statistical validation, interpreted the results, prepared all tables and figures, and wrote, reviewed, and finalized the manuscript.

Conflict of Interest Statement

The authors declare no conflicts of interest.

Funding Declaration

None.

Data Availability Statement

None.

Ethical Declarations

No animal subjects were involved. All data were anonymized prior to analysis to ensure participant privacy and compliance with institutional regulations.

Acknowledgments

We gratefully acknowledge the valuable contributions of the expert reviewers and evaluators whose insightful comments and constructive recommendations significantly strengthened this research. Their professional guidance enhanced the clarity of the methodology, improved analytical rigor, and reinforced the reliability, validity, and overall quality of the experimental findings and conclusions.

Declaration of Generative AI in Scholarly Writing

During the preparation of this manuscript, generative AI tools were used to assist with language refinement, grammar checking, and structural organization of the text. All conceptual development, data analysis, interpretation of results, and final academic judgments were conducted independently by the author. The author assumes full responsibility for the content.

Abbreviations

The following abbreviations are used in this manuscript:

AI – Artificial Intelligence

CBR – Case-Based Reasoning

IT – Information Technology

PCA – Principal Component Analysis

ANOVA – Analysis of Variance

Appendix

Appendix A. Operational Definitions of Core Constructs

Student Competency Performance

Measured using analytic rubric indicators covering technical implementation (automation, security, integration), documentation quality, innovation, presentation skills, and teamwork. Scores were aggregated using weighted composite scoring.

Project Proposal Quality

Assessed based on clarity of objectives, feasibility, methodological design, innovation level, and alignment with industry or academic standards.

Quality of Advising

Evaluated through supervision frequency, feedback quality, mentor experience, and student–advisor interaction consistency.

Panel Variability

Measured using standard deviation of panel grades per project and inter-rater correlation coefficients.

Assessment Criteria Weighting

Refers to the proportional allocation of scores across technical, professional, and communication dimensions.

Appendix B. Rubric Structure (Analytic Layer)

Technical Domain (40%)

• •    System Design and Implementation

• •    Security and Risk Management

• •   Automation / DevOps Integration

• •    Testing and Validation

Professional Domain (35%)

• •   Documentation Quality

• •    Internship Performance

• •    Ethical Conduct

• •    Project Planning and Risk Control

Communication Domain (25%)

• •    Presentation Clarity

• •    Technical Explanation

• •   Team Collaboration

• •    Response to Panel Questions

Appendix C. Holistic Evaluation Criteria

Panel members evaluated the following integrative dimensions:

• •    Overall Innovation and Creativity

• •    System Coherence and Integration

• •    Professional Maturity

• •    Practical Relevance to Industry

Holistic scores were normalized and combined with analytic scores using a weighted composite formula.