The Ultrashort Mental Health Screening Tool Is a Valid and Reliable Measure With Added Value to Support Decision-making

Abstract

Abstract Background Mental health influences symptoms, outcomes, and decision-making in musculoskeletal healthcare. Implementing measures of mental health in clinical practice can be challenging. An ultrashort screening tool for mental health with a low burden is currently unavailable but could be used as a conversation starter, expectation management tool, or decision support tool. Questions/purposes (1) Which items of the Pain Catastrophizing Scale (PCS), Patient Health Questionnaire (PHQ-4), and Brief Illness Perception Questionnaire (B-IPQ) are the most discriminative and yield a high correlation with the total scores of these questionnaires? (2) What is the construct validity and added clinical value (explained variance for pain and hand function) of an ultrashort four-item mental health screening tool? (3) What is the test-retest reliability of the screening tool? (4) What is the response time for the ultrashort screening tool? Methods This was a prospective cohort study. Data collection was part of usual care at Xpert Clinics, the Netherlands, but prospective measurements were added to this study. Between September 2017 and January 2022, we included 19,156 patients with hand and wrist conditions. We subdivided these into four samples: a test set to select the screener items (n = 18,034), a validation set to determine whether the selected items were solid (n = 1017), a sample to determine the added clinical value (explained variance for pain and hand function, n = 13,061), and a sample to assess the test-retest reliability (n = 105). Patients were eligible for either sample if they completed all relevant measurements of interest for that particular sample. To create an ultrashort screening tool that is valid, reliable, and has added value, we began by picking the most discriminatory items (that is, the items that were most influential for determining the total score) from the PCS, PHQ-4, and B-IPQ using chi-square automated interaction detection (a machine-learning algorithm). To assess construct validity (how well our screening tool assesses the constructs of interest), we correlated these items with the associated sum score of the full questionnaire in the test and validation sets. We compared the explained variance of linear models for pain and function using the screening tool items or the original sum scores of the PCS, PHQ-4, and B-IPQ to further assess the screening tool’s construct validity and added value. We evaluated test-retest reliability by calculating weighted kappas, ICCs, and the standard error of measurement. Results We identified four items and used these in the screening tool. The screening tool items were highly correlated with the PCS (Pearson coefficient = 0.82; p < 0.001), PHQ-4 (0.87; p < 0.001), and B-IPQ (0.85; p < 0.001) sum scores, indicating high construct validity. The full questionnaires explained only slightly more variance in pain and function (10% to 22%) than the screening tool did (9% to 17%), again indicating high construct validity and much added clinical value of the screening tool. Test-retest reliability was high for the PCS (ICC 0.75, weighted kappa 0.75) and B-IPQ (ICC 0.70 to 0.75, standard error of measurement 1.3 to 1.4) items and moderate for the PHQ-4 item (ICC 0.54, weighted kappa 0.54). The median response time was 43 seconds, against more than 4 minutes for the full questionnaires. Conclusion Our ultrashort, valid, and reliable screening tool for pain catastrophizing, psychologic distress, and illness perception can be used before clinician consultation and may serve as a conversation starter, an expectation management tool, or a decision support tool. The clinical utility of the screening tool is that it can indicate that further testing is warranted, guide a clinician when considering a consultation with a mental health specialist, or support a clinician in choosing between more invasive and less invasive treatments. Future studies could investigate how the tool can be used optimally and whether using the screening tool affects daily clinic decisions. Level of Evidence Level II, diagnostic study.