Programmatic assessment prototype design using integrated system thinking and design thinking framework

Abstract

Objective: To design a prototype programmatic assessment model using integration of system thinking and design thinking frameworks, and to explore barriers in the way of its implementation in medical education. Method: The qualitative case-study was conducted at Bahria University Medical and Dental College, Karachi from September to December 2022. Methodological approaches used were systems thinking and design thinking. Philosophical paradigm used was critical realism. Maximum variation purposive sampling technique was used for selecting faculty members and medical students. Data was collected in two sets of semi-structured interviews. The initial interviews were at the levels of empathise, define and ideate. After that low-fidelity prototype programmatic assessment design was created and presented to the participants. The second set of interviews were focussed on feedback on prototype of programmatic assessment design. Pattern matching method was used for data analysis. Results: Of the 65 subjects, 20(30.7%) were faculty members; 5(25%) lecturers and 5(25%) professors each from basic and clinical sciences. The remaining 45(69.2%) were students; 5(11.1%)from each year of medical and dental streams. Initial interviews revealed that after failure, students struggle to understand the cause of failure as assessment system lacked narrative feedback mechanism and numbers or pass/fail decisions were not self-explanatory. Students lacked attitude towards continued improvement, formative assessment were not taken seriously as they carried only 10% weightage. After the programmatic assessment design was presented to the participants, faculty members voiced concerns regarding faculty training, shortage of resources and legal issues related to its implementation. Students supported the design, especially continued narrative feedback for their academic improvement. Portfolio development was considered a time-consuming task and they had reservations regarding their ability to make quality assessment portfolio. Conclusion: Implementation of programmatic assessment required faculty development and was resource-intensive, but stakeholders, especially students, were found interested in its implementation in medical education. Key Words: Programmatic assessment, Medical education, Medical students.