Development of an online evaluation awareness scale

Abstract

The COVID-19 epidemic has precipitated a rapid and widespread adoption of online education, leading to its normalization in contemporary society. Online education is evident across several educational levels. However, assessing the efficacy and effectiveness of these training programs can only be achieved by implementing a suitable evaluation methodology. One of the primary challenges associated with online education is the difficulty in assessing its quality and effectiveness. One of the contributing factors to this issue is the instructor's lack of technological skills and knowledge relevant to online teaching. This research aims to develop a scale (Online Evaluation Awareness-OnEvA) to determine the instructors’ awareness of online evaluation. 165 participants’ data from 63 universities was used for exploratory factor analysis. The items of the scale are designed to measure awareness in both pedagogical and technological dimensions. The items in the pedagogical dimension include the essential competencies that instructors should possess, and alternative evaluation methods offered online. In the technological dimension, the items are created to determine instructors’ awareness of the use of technological platforms and tools. For the next step, a confirmatory factor analysis was performed to establish the scale’s construct validity with 161 instructors. Additionally, the scale’s internal consistency was determined to be high, with a Cronbach's alpha of .964. It is affirmed that this awareness scale developed within the scope of this research will contribute to the development of future models or frameworks related to the dimensions of evaluation knowledge in online learning.