Psychometric analysis of a questionnaire with BARS. An opportunity to improve teaching effectiveness measurement programs and decision making in accreditation processes

Abstract

In the field of teaching effectiveness measurement programs, studies on the validation of Behavioral Anchored Rating Scales (BARS) are minimal when compared with Likert instruments. The reason for this situation is a consequence of the limited number of universities opting for this type of questionnaire in their teaching effectiveness measurement programs. This situation is due to the thoroughness, time investment and strong involvement of human resources required in the design of these scales. The aim of this investigation is twofold. On the one hand, to analyze the validity of a questionnaire for measuring teaching effectiveness that uses BARS. On the other, to check whether this instrument, designed in a given university with the participation of the professors and students of this institution, can be valid for other universities. The study is carried out in three Spanish universities. The validation process considers: comprehension validity, EFA, CFA with structural equation modeling, and reliability analysis. The results show that BARS under examination are valid for measuring teaching effectiveness; not only in the institution where they are designed, but also in other universities different from the one in which the questionnaire is constructed. The findings of this research open new alternatives not only to improve teaching effectiveness measurement programs but also to enhance decision making in accreditation processes.