Error Propagation from Sleep Stage Classification to Derived Sleep Parameters in Machine Learning on Data from Wearables

Abstract

Abstract Purpose of Review: Automatic analysis of sleep is an important and active area of research. Machine learning models are commonly developed to classify time segments into sleep stages. The sleep stages can be used to calculate various sleep parameters, such as sleep efficiency and total sleep time. The machine learning models are typically trained to minimize the sleep stage classification error, but little is known about how error propagates from sleep stages to derived sleep parameters. Recent findings: We review recently published studies where machine learning was used to classify sleep stages using data from wearable devices. Using classification error statistics from these studies, we perform a Monte Carlo simulation to estimate sleep parameter error in a dataset of 197 hypnograms. This is, to our knowledge, the first attempt at evaluating how robust sleep parameter estimation is to misclassification of sleep stages. Summary: Our analysis suggests that a machine learning model capable of 90% accurate sleep stage classification (surpassing current state-of-art in wearable sleep tracking) may perform worse than a random guess in estimating some sleep parameters. Our analysis also indicates that sleep stage classification may not be a relevant target variable for machine learning on wearable sleep data and that regression models may be better suited to estimating sleep parameters. Finally, we propose a baseline model to use as a reference for sleep stage estimation accuracy. When applied to a test set, the baseline model predicts 2-, 3-, 4- and 5-class sleep stages with an accuracy of 74%, 54%, 46% and 35%, respectively