Interrater agreement of multi-professional case review as reference standard for specialist palliative care need: a mixed-methods study

Abstract

Abstract Background A wide variety of screening tools for the need for specialist palliative care (SPC) have been proposed for the use in oncology. However, as there is no established reference standard for SPC need to compare their results with, their sensitivity and specificity have not yet been determined. The aim of the study was to explore whether SPC need assessment by means of multi-professional case review has sufficient interrater agreement to be employed as a reference standard. Methods Comprehensive case descriptions were prepared for 20 inpatients with advanced oncologic disease at the University Hospital Freiburg (Germany). All cases were presented to the palliative care teams of three different hospitals in independent, multi-professional case review sessions. The teams assessed whether patients had support needs in nine categories and subsequently concluded SPC need (yes / no). Interrater agreement regarding SPC need was determined by calculating Fleiss’ Kappa. Results In 17 out of 20 cases the three teams agreed regarding their appraisal of SPC need (substantial interrater agreement: Fleiss’ Kappa κ = 0.80 (95% CI: 0.55–1.0; p < 0.001)). The number of support needs was significantly lower for patients who all teams agreed had no SPC need than for those with agreed SPC need. Conclusions The proposed expert case review process shows sufficient reliability to be used as a reference standard. Key elements of the case review process (e.g. clear definition of SPC need, standardized review of the patients’ support needs) and possible modifications to simplify the process are discussed. Trial registration German Clinical Trials Register, DRKS00021686, registered 17.12.2020.