Is there a difference in floor effects and reliability between intermixed and fixed-order items in a questionnaire?

Abstract

Objective We tested whether intermixing mental health items with items addressing comfort and capability could limit the floor effects noted when mental health is measured in musculoskeletal specialty care. Methods One hundred and 31 people seeking care for upper and lower extremity musculoskeletal conditions were randomized to complete randomly ordered, unlabeled mental health items intermixed with comfort and capability items, or intact and labelled questionnaires. For the two approaches, we compared: (1) flooring and ceiling effects; (2) mean and median questionnaire scores; (3) internal consistency (Cronbach alpha); and (4) exploratory factor analysis. We sought correlations between mental health and levels of pain intensity and capability. Results We found slightly more flooring in the intermixed group for symptoms of depression (66% [41 of 62] vs 46% [32 of 69], p-value = .034), no differences in the mean and median scores for each questionnaire, lower internal consistency measured by Cronbach alpha, and lower factor loading coefficients in exploratory factor analysis for symptoms of depression and anxiety in the intermixed group. The mean level of symptoms of anxiety was significantly different between two groups (intermixed: 0.87 [95% CI 0.82 to 0.92], fixed: 0.96 [95% CI 0.93 to 0.98]). There were no differences in the association of the mental health measures gathered via the two different strategies with measures of pain intensity and magnitude of capability. Conclusion The finding that intermixing mental health questions with questions about comfort and capability did not diminish floor effects suggests no advantage to intermixing mental health items in questionnaires used in musculoskeletal care and research.