Convolutional neural networks on eye tracking trajectories classify patients with spatial neglect

Abstract

AbstractBackground and ObjectiveEye-movement trajectories are rich behavioral data, providing a window on how the brain processes information. We address the challenge of characterizing signs of visuo-spatial neglect from saccadic eye trajectories recorded in brain-damaged patients with spatial neglect as well as in healthy controls during a visual search task.MethodsWe establish a standardized preprocessing pipeline adaptable to other task-based eye-tracker measurements. We use a deep convolutional network, a very successful type of neural network architecture in many computer vision applications, including medical diagnosis systems, to automatically analyze eye trajectories.ResultsOur algorithm can classify brain-damaged patients vs. healthy individuals with an accuracy of 86±5%. Moreover, the algorithm scores correlate with the degree of severity of neglect signs estimated with standardized paper-and-pencil test and with white matter tracts impairment via Diffusion Tensor Imaging (DTI). Interestingly, the latter showed a clear correlation with the third branch of the superior longitudinal fasciculus (SLF), especially damaged in neglect.ConclusionsThe study introduces a new classification method to analyze eyes trajectories in patients with neglect syndrome. The method can likely be applied to other types of neurological diseases opening to the possibility of new computer-aided, precise, sensitive and non-invasive diagnosing tools.HighlightsWe identify signs of visuo-spatial neglect through an automated analysis of saccadic eye trajectories using deep convolutional neural networks (CNNs).We provide a standardized pre-processing pipeline adaptable to other task-based eye-tracker measurements.Patient-wise, we benchmark the algorithm prediction with standardized paper-and-pencil test results.We evaluate white matter tracts by using Diffusion Tensor Imaging (DTI) and find a clear correlation with the microstructure of the third branch of the superior longitudinal fasciculus.Deep CNNs can efficiently and non-invasively characterize left spatial neglect.