Factors Considered in the Assessment of Computer Science Engineering Capstone Projects and Their Influence on Discrepancies Between Assessors

Abstract

A capstone project is an extensive learning experience traditionally developed during a student's final academic year. Assessing such a complex assignment involves several challenges and is usually based upon the evaluations of at least two different people: the capstone project advisor, and one or more other assessors. Quantitative studies comparing only different assessors' grades and qualitative studies investigating the origin of possible discrepancies have been conducted. In both cases, contradictory conclusions were reached. The objective of this study is to analyze the factors that are given consideration by assessors of engineering capstone projects and the influence of these factors on the discrepancies between different assessors’ opinions of the same project. This study quantitatively examined 162 computer science engineering capstone projects developed by one student and supervised by one advisor. Each project was assessed by the project advisor and a committee. For each project, the advisor and the committee were asked to complete an additional questionnaire on product characteristics, student competences, and project supervision. Competences demonstrated by the student were found to be the most relevant element when a capstone project was evaluated by the advisor and the committee; product characteristics were found to be second in influence. Furthermore, advisors grant minor significance to the advisor-involvement component. Discrepancies between grades seem to be associated with those aspects to which one assessor has access, while the other does not, such as student skills demonstrated during project development or their performance in the oral defense. Both the advisor's and the committee's perspectives are important in the assessment of this complex task and they complement one another.