Developing and Validating the Evaluator Competencies Assessment Tool (ECAT) Cultural Competencies Subscale

Abstract

This study aimed to develop and validate the Evaluator Competencies Assessment Tool (ECAT) Cultural Competencies Subscale, which measures cultural competence among evaluators. By addressing the scarcity of validated tools in this area, the study offers professionals a valuable resource to assess their strengths and areas for improvement. The subscale comprises 11 items rated on a seven-point Likert scale, derived from the American Evaluation Association (AEA) Evaluator Competencies. Validation involved a survey of 116 AEA members, employing multiple validity procedures. The subscale demonstrated excellent internal consistency (α=.96) and significant correlation with the Cultural Competence of Program Evaluators scale, confirming its convergent validity. However, the subscale's structural validity yielded mixed results, indicating the need for further exploration. Moreover, the demographic analysis highlighted underrepresentation of Asian and younger members. Overall, the ECAT Cultural Competencies Subscale shows promise for assessing cultural competency, but refinement of its factor structure and additional research are needed.