Reliability testing of the Health of the Nation Outcome Scales 2018

Abstract

Accessible SummaryWhat is known on the subject The Health of the Nation Outcome Scales (HoNOS) is a widely used clinical measure designed to rate and monitor the outcomes of service users accessing specialist mental healthcare. Since its development (in 1996), numerous research studies have confirmed the HoNOS captures the aspects of care that it purports to (validity), and that clinicians' ratings are consistent both over time, and between different raters (reliability). What the paper adds to existing knowledge In 2018, the HoNOS was reviewed with updates made to some terminology and other revisions intended to remove ambiguity in the guidance for raters. However, although the new version (HoNOS 2018) was accompanied by a recommendation that its validity and reliability be re‐tested this was not undertaken. To our knowledge, this is the first study to re‐assess the updated tool's reliability by measuring the level of agreement between different raters. Our findings confirm that there is an acceptable level of consistency between student mental health nurses that have been trained to use the (new) HoNOS 2018. What are the implications for practice The HoNOS is nationally mandated for use by all specialist mental healthcare providers in the UK. Our findings provide some assurance that, with appropriate update training and monitoring of organisational‐level data sets, the original HoNOS glossary can safely be replaced with the HoNOS 2018 to ensure more contemporary routine outcome measurement can occur. AbstractIntroductionThe Health of the Nation Outcome Scales (HoNOS) is a well‐established clinician rated outcome measure for use in mental health services. Following an international review, an updated version (HoNOS 2018) was published with a recommendation that its psychometric properties be re‐tested prior to widespread implementation. To date, only one such study has been published.AimsTo test the inter‐rater agreement levels for HoNOS 2018.MethodThird‐year student mental health nurses received training to complete the HoNOS 2018. Following this timetabled session, they were each invited to independently rate two, randomly selected, videos of (simulated) patient interviews. The resulting data were then analysed to calculate the tool's internal consistency and inter‐rater agreement levels.ResultsThe 55 participants provided 106 ratings from four vignettes. Cronbach's alphas and McDonalds omegas confirmed the revised tool's internal consistency was acceptable. Average measure intraclass correlation coefficients for the four patient vignettes indicated excellent reliability.Implications for practiceThis study provides initial assurance that the HoNOS 2018 is a reliable clinician rated outcome measure suitable for use in routine clinical practice by relatively inexperienced mental health practitioners with limited training.