Usability testing of the Smart Work Injury Management (SWIM) system to assist case managers in predicting the work disability of injured workers

Abstract

Abstract Purpose Many countries have developed clinical decision-making support (CDMS) tools, such as the Smart Work Injury Management (SWIM) system in Hong Kong, to predict rehabilitation paths and address global issues related to work injury disability. This study aims to evaluate the accuracy of SWIM by comparing its predictions on real work injury cases to those made by human case managers, specifically with regard to the duration of sick leave and the percentage of permanent disability. Methods The study analysed a total of 442 work injury cases covering the period from 2012 to 2020, dividing them into non-litigated and litigated cases. The Krustal-Wallis post hoc test with Bonferroni adjustment was used to evaluate the differences between the actual data, the SWIM predictions, and the estimations made by three case managers. The intra-class correlation coefficient (ICC) was used to assess the inter-rater reliability of the raters. Results The Krustal-Wallis test revealed a statistically significant similarity between the predictions of SWIM and a case manager with about four years of case management experience on non-litigated cases. The predictions made by case managers with 9 and 20 years of experience were also significantly similar for all cases. However, SWIM’s prediction of the permanent disability percentage differed significantly from the predictions of the other groups. Conclusions The study confirmed that SWIM could effectively simulate the predictions of a case manager with roughly four years of case management experience regarding sick leave duration. It is recommended that future research considers the influence of psycho-social factors on predictions as these are often factored in by experienced case managers. Implications SWIM can aid case managers in estimating non-litigated cases, thereby addressing the shortage of human resources in Hong Kong and reducing inconsistencies in case estimations.