Clinical Scholars: Using Program Evaluation to Inform Leadership Development

Abstract

Leadership development programs are notoriously difficult to evaluate, and when evaluations are attempted, they often do not go beyond measuring low-level, short-term outcomes of the impacts experienced by participants. Many leadership development programs do not systematically assess changes that are catalyzed within the organizations, communities and systems in which participants lead. To address these challenges, evaluators of the Clinical Scholars National Leadership Institute (CNLI) have designed a comprehensive, mixed-methods evaluation approach to determine the effectiveness of the training and explore the impacts of participants in the spheres in which they lead. Guided by Michael Patton’s Developmental Evaluation approach and framed by Kirkpatrick’s Training Evaluation Model, the CSNLI evaluation collects data on multiple levels to provide a robust picture of the multiple outcomes of the program. The approach focuses on individual participant outcomes, by measuring competency changes over time and exploring how participants use the competencies gained through the training in their work. Social network analysis is utilized to measure the development and expansion of participants’ networks and collaboration within the teams, cohorts, and across sectors and disciplines throughout their time in the CSNLI. The Most Significant Change methodology and semi-structured alumni interviews are used to measure impacts participants identify as occurring as a result of their participation. Finally, Concept Mapping is implemented to explore how Fellows make meaning of the foundational concepts and values of the CSNLI. The outcome and impact evaluation activities employed by the CSNLI, in combination with quality improvement-focused process evaluation, support innovation and excellence in the provision of a health equity-grounded leadership development program.