Evaluation Quality Assessment Frameworks: A Comparative Assessment of Their Strengths and Weaknesses

Abstract

Quality Assessment Frameworks (QAFs) are standardized templates that include pre-defined criteria, open questions, or a combination to assess the quality of evaluations. They support organizations to limit risks associated with poor-quality evaluation. Qualitative research focused on four QAFs shows that five dimensions of quality are assessed within the frameworks that include pre-defined criteria. These are aligned with the criteria that audiences of evaluation identify as important to assessing quality: substantive findings, robust and appropriate methodology, accessibility, inclusive processes and analysis of wider systems. Consequently, these QAFs are likely to support critical assessment of the main dimensions of quality that are relevant to audiences of evaluation. Audiences also note synergies between these five dimensions of quality and appropriate contextualization as crucial to assessing quality. Consequently, the structure and categorization of quality within QAFs which only include pre-defined criteria risks limiting reviewers’ ability to reflect on these synergies and contextualize assessments.