Validity and reliability of the Spanish version of the ACE tool for assessing competencies in evidence-based medicine in medical students

Abstract

ObjectiveTo translate and culturally adapt the tool ‘Assessing Competency in evidence-based medicine (EBM)’ (ACE) to Spanish and to implement it in a cohort of medical students for the evaluation of the instrument’s psychometric properties.DesignBilingual translators produced a translation and backtranslation of the original instrument, with interim consensus in each stage with oversight and input by a group of experts. We then performed cognitive interviews to adapt the wording of the tool culturally. Finally, we implemented the final version in a cohort of medical students on a virtual general practice course with EBM modules.SettingMedical School in Buenos Aires, Argentina. Due to restrictions to in-person teaching during the COVID-19 pandemic, we conducted this study in the context of virtual learning.ParticipantsWe included 125 fourth and fifth-year medical students.Main outcome measuresWe measured internal consistency with the Kuder-Richardson coefficient (>0.6 as a threshold for reliability) and construct validity through a Pearson’s correlation between the examinations carried out with the translated instrument and the results of the regular examinations of EBM in the same students (expected values of at least 0.3 to 0.7). We also compared the total score of the instrument of fifth-year students to fourth-year students.ResultsAs for measurements for internal consistency, the coefficient Kuder-Richardson resulted in a value of 0.268, below our prespecified threshold. For construct validity, the Pearson correlation between the sum of the items and regular examinations was 0.139, also below our prespecified threshold. However, fifth-year students averaged 0.94 points more than fourth-year students (95% CI 0.24 more to 1.65 more).ConclusionThe translated and cross-culturally adapted version of the ACE tool into Spanish had low reliability and validity in an MBE course taught and evaluated in a virtual environment.Trial registrationNot applicable.