Item-Level Analysis of a Newly Developed Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) Using the Rasch Measurement Model

Abstract

The Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) is a tool designed specifically to observe and measure registered dietitian nutritionists’ (RDNs) nutrition-focused physical exam (NFPE) competence in authentic acute care settings. The initial INSPECT items were generated and tested for content and face validity using expert RDNs’ input. The INSPECT was further examined for inter-rater, intra-rater, and internal consistency using clinical supervisor observations of RDNs performing NFPE on patients in real-life acute care settings. These previous studies showed the INSPECT to have excellent content validity, acceptable face validity, good inter-rater reliability, moderate to strong intra-rater reliability, and excellent internal consistency. In the current study, the Rasch measurement model was applied to examine the item-level properties of the INSPECT. Results confirm that the INSPECT measured a single construct. All items fit the established criteria for clinical observations of >0.5 and <1.7, had positive point measure correlations, met the Wright Unidimensionality Index criteria of ≥0.9, exhibited one latent construct with >40% variance explained by the Rasch dimension as well as a sub-dimension based on item difficulty from the principal component analysis of the first contrast Rasch residuals. Rasch rating scale analysis revealed that the rating scale and majority of the items (39/41) fit the Rasch model. Rasch item hierarchy analysis matched the a priori hypothesized hierarchy for the top-most and bottom-most items. Ceiling effects were seen for three items (hand hygiene, personal protective equipment, and patient position) and one item (handgrip using hand dynamometer) reached the floor effect. Rasch reliability assessment demonstrated high person reliability (0.86), high item reliability (0.96), and person separation of 3.56 ability levels. The principal component analysis of residuals revealed two factors based on item difficulty, one for micronutrient exam and another for macronutrient exam, initial steps, and bedside manner. The resulting two factors may likely be due to a sub-dimension of the latent NFPE trait. Overall, the INSPECT items were found to have good item-level psychometrics. Continued testing of the INSPECT with RDNs at different ability levels will help to determine cut-off scores ranging from novice to expert. Establishing cut-off scores for the INSPECT will further enhance the utility of the tool.