Procedures for Improving Self-report Measurements to Capture Behavior Change: An Illustration

Abstract

Programs utilizing research-tested evaluation tools can help identify effective educational strategies and document program effectiveness and impact.  Using the case of the UF/IFAS Extension Family Nutrition Program (FNP), this article illustrates steps for conducting a rigorous assessment of the measurement properties of evaluation instruments.  The Youth Behavior Survey (YBS) was originally developed to measure students’ nutrition and physical activity behaviors before and after an educational intervention.  In order to report FNP behavior change data under indicators for the national evaluation framework, the evaluation instrument was revised.  The revision included modifying item wording to reflect national indicators and changing response options.  The psychometric characteristics of the revised instrument were assessed in comparison to those of the original instrument.  The main objective was to examine aspects of content and construct validity for the scores produced by the instruments.  The assessment included content validity of the instrument, item discrimination, consistency of relationships in item response patterns, and change between pre-test and post-test scores.  We concluded that the scores produced by the revised instrument were modestly more accurate than the original.  This research suggests procedures that can be applied widely to evaluating instruments for other educational interventions.