Using adversarial networks to extend brain computer interface decoding accuracy over time

Abstract

AbstractExisting intracortical brain computer interfaces (iBCIs) transform neural activity into control signals capable of restoring movement to persons with paralysis. However, the accuracy of the “decoder” at the heart of the iBCI typically degrades over time due to turnover of recorded neurons. To compensate, decoders can be recalibrated, but this requires the user to spend extra time and effort to provide the necessary data, then learn the new dynamics. As the recorded neurons change, one can think of the underlying movement intent signal being expressed in changing coordinates. If a mapping can be computed between the different coordinate systems, it may be possible to stabilize the original decoder’s mapping from brain to behavior without recalibration. We previously proposed a method based on Generalized Adversarial Networks (GANs), called “Adversarial Domain Adaptation Network” (ADAN), which aligns the distributions of latent signals within underlying low-dimensional neural manifolds. However, ADAN was tested on only a very limited dataset. Here we propose a method based on Cycle-Consistent Adversarial Networks (Cycle-GAN), which aligns the distributions of the full-dimensional neural recordings. We tested both Cycle-GAN and ADAN on data from multiple monkeys and behaviors and compared them to a linear method based on Procrustes Alignment of axes provided by Factor Analysis (PAF). Both GAN-based methods outperformed PAF. Cycle-GAN and ADAN (like PAF) are unsupervised and require little data, making them practical in real life. Overall, Cycle-GAN had the best performance and was easier to train and more robust than ADAN, making it ideal for stabilizing iBCI systems over time.Significance StatementThe inherent instabilities in the neural signals acquired by intracortical microelectrode arrays cause the performance of an intracortical brain computer interface (iBCI) decoder to drop over time, as the movement intent signal must essentially be recorded from neurons representing an ever-changing coordinate system. Here, we address this problem using Generative Adversarial Networks (GANs) to align these coordinates and compare their success to another, recently proposed linear method that uses Factor Analysis and Procrustes alignment. Our proposed methods are fully unsupervised, can be trained quickly, and require remarkably little new data. These methods should give iBCI users access to decoders with unchanging dynamics, and without the need for periodic supervised recalibration.