From Sensory to Perceptual Manifolds: The Twist of Neural Geometry

Abstract

AbstractTo humans, nearly everything is classifiable: whether as big or small, edible or poisonous, righteous or unjust. Similarly, classification is a central task in many machine learning applications, yet the problem of linear inseparability has long posed challenges for artificial neural networks since their inception. Here we asked how biological neural networks tackle this problem by investigating the geometric embedding of neural manifolds in macaques’V2 during orientation discrimination of motion-induced illusory contours. Specifically, we constructed a three-dimensional stimulus space that inherently made the orientation classification of these contours a linearly inseparable problem. As expected, we identified a sensory manifold, formed by the V2 neuron population, that faithfully corresponded to this stimulus space. Crucially, this sensory manifold underwent a series of twist operations, resulting in new axes orthogonal to the original ones. Within this expanded, high-dimensional perceptual manifold, the problem of linear inseparability became linearly separable. Computational models further revealed that the geometric twist operation was achieved by neurons exhibiting nonlinear mixed selectivity in networks with heterogeneous connectivity patterns. Taken together, our findings elucidate how perception arises from sensation through the lens of neural geometry, enriching our understanding of how cognitive functions derive from underlying anatomical structure.