Tıp Eğitiminde Klinik Akıl Yürütme Becerisinin Ölçümü için Türkçede İlk Otomatik Soru Üretimi

Abstract

Aim: Writing high-quality items (questions) is a resource-intensive task. Particularly, the development of one context-rich multiple-choice question (MCQ) for assessing higher-order cognitive skills may cost hours of medical teachers. The aim of this study was to find out whether it is possible the use of Automatic Item Generation (AIG) in Turkish to generate case-based MCQs that assess clinical reasoning skills. Methods: By following the template-based AIG method developed by Gierl et al., MCQs on hypertension were generated with the help of software after the development of a cognitive model and an item model. The cognitive model and the item model was developed by a medical doctor and a cardiologist by considering Turkish Hypertension Consensus Report. The software was built as a Python-based code intended for single use without a user interface. The items were recorded in a MySQL database. Of these questions, 10 questions were randomly chosen to be reviewed by three subject matter experts (cardiologists). The evaluation was based on the quality of the questions and whether the questions assess higher-order skills such as clinical reasoning rather than factual recall. Results: In 1.73 seconds, 1600 MCQs on hypertension were generated. Although there were some minor revision suggestions in a few questions, each question was stated by all cardiologists as an acceptable item. The cardiologists also stated that the questions assess clinical reasoning skills rather than factual recall. Conclusions: This study demonstrated for the first time that AIG for assessing clinical reasoning skills in the context of medical education in Turkish is possible. This method of augmented intelligence to generate items can be used in Turkish as it has been used in other five languages. The use of this method could bring about more questions to assess clinical reasoning skills. It may also lead medical teachers to spend less amount of time and effort compared to traditional item writing.