Assessment of Smartphone-Based Spiral Tracing in Multiple Sclerosis Reveals Intra-Individual Reproducibility as a Major Determinant of the Clinical Utility of the Digital Test

Abstract

Technological advances, lack of medical professionals, high cost of face-to-face encounters, and disasters such as the COVID-19 pandemic fuel the telemedicine revolution. Numerous smartphone apps have been developed to measure neurological functions. However, their psychometric properties are seldom determined. It is unclear which designs underlie the eventual clinical utility of the smartphone tests. We have developed the smartphone Neurological Function Tests Suite (NeuFun-TS) and are systematically evaluating their psychometric properties against the gold standard of complete neurological examination digitalized into the NeurExTM app. This article examines the fifth and the most complex NeuFun-TS test, the “Spiral tracing.” We generated 40 features in the training cohort (22 healthy donors [HD] and 89 patients with multiple sclerosis [MS]) and compared their intraclass correlation coefficient, fold change between HD and MS, and correlations with relevant clinical and imaging outcomes. We assembled the best features into machine-learning models and examined their performance in the independent validation cohort (45 patients with MS). We show that by involving multiple neurological functions, complex tests such as spiral tracing are susceptible to intra-individual variations, decreasing their reproducibility and clinical utility. Simple tests, reproducibly measuring single function(s) that can be aggregated to increase sensitivity, are preferable in app design.