Assessing Physician’s Motivational Communication Skills: The development of the Motivational Communication Competency Assessment Test (MC-CAT) (Preprint)

Abstract

Training physicians to provide effective behaviour change counselling (BCC) using approaches like motivational communication (MC) is an important aspect of non-communicable chronic disease (NCD) prevention and management. However, existing evaluation tools for MC skills are complex, invasive, time consuming, and impractical for use within the medical context.

The objective of this study was to develop and validate a short, web-based tool evaluating healthcare providers (HCPs) skills in MC, the Motivational Communication Competency Assessment Test (MC-CAT).

Between 2016 and 2021, starting with a set of core 11 MC competencies previously identified and using a 5-step mixed-methods integrated knowledge translation (iKT) approach, the MC-CAT was created by: 1) developing a series of four base cases and a scoring scheme; 2) validating the base cases and scoring scheme with international experts; 3) creating three alternative versions of the four base cases (to create a bank of 16 cases, four of each type of ‘base’ case) and translating the cases into French; 4) integrating the cases into the online MC-CAT platform; and 5) conducting initial internal validity assessments with university health students.

The MC-CAT assesses MC competency in 20 minutes by presenting HCPs with 4 out of a possible 16 cases (randomly selected and ordered) addressing various behavioural targets (e.g., smoking, physical activity, diet, medication adherence). Individual and global competency scores are calculated automatically for the 11 competency items across the 4 cases providing automatic scores out of 100. From the factorial analysis of variance for the difference in competency and ranking scores, no significant differences were identified between the different case versions across individual and global competency and ranking scores (p’s > 0.05). Initial tests of internal consistency for rank order among 24 student participants were in the ‘acceptable’ range (alpha = 0.78).

Results suggest the MC-CAT is an internally valid tool to facilitate the evaluation of MC competencies among HCPs, and is ready to undergo comprehensive psychometric property analyses with a national sample of healthcare providers. Once psychometric property assessments have been completed, this tool is expected to facilitate the assessment of MC skills among HCPs, skills that will better support patients to adopt healthier lifestyles, which will significantly reduce the personal, social, and economic burden of NCDs.