Development and preliminary validation of an Objective Structured Assessment of Technical Skills (OSATS) for a partial pulpotomy procedure

Abstract

AbstractAimTo develop an Objective Structured Assessment of Technical Skills (OSATS) tool for partial pulpotomy and to preliminary assess the validity of this tool. Secondary objectives were to check internal consistency as well as the interrater reliability of this specific checklist and to compare it with the previously validated global rating score (GRS).MethodologyThe study assessed three groups of dental participants (mix of practitioners or students) possessing differing levels of clinical experience (7 novices, 7 intermediates, 7 experts), during a partial pulpotomy procedure on a simulated tooth with deep caries, using a bespoke procedure‐specific checklist (PSC) and the GRS. Two independent examiners received the study participants in groups of two, in order to be able to directly observe their actions and grade the different steps of the procedure from 1 to 5 on the PSC and GRS. The internal validity of the specific checklist was assessed using a Cronbach's alpha test. As the construct validity of such a tool can be determined when the tool differentiates performance based on the level of experience, the total score of the PSC of the three groups (e.g., novice group, intermediate group, experienced group) were compared using the Kruskal–Wallis and post‐hoc Mann–Whitney pairwise U test. The total scores of the GRS were also analysed in the same manner. A correlation test (Correlation Matrices Test) was carried out for the entire sample between the specific checklist and the GRS (on total score). The intraclass correlation coefficient was used to measure the degree of agreement between the two evaluators. The statistical analysis was performed with XLSTAT® and Statistica® (significance p = .05).ResultsThe partial pulpotomy specific checklist graded from one to five demonstrated good internal consistency (0.86) and a good interrater reliability (0.91), a correct construct validity (p generally < .05) and a good positive correlation with the validated GRS (r = .92).ConclusionsTo address the need to develop education in vital pulp treatment, and more specifically the technical skills assessment of partial pulpotomy, an OSATS‐specific checklist was developed and preliminary validated. The checklist will need to be validated in larger cohorts.