Behavioral biometric optical tactile sensor that instantaneously decouples dynamic touch signals in real time

Abstract

Decoupling dynamic touch signals in the optical tactile sensors is highly desired for behavioral tactile applications yet challenging because typical optical sensors mostly measure only static normal force and use imprecise multi-image averaging for dynamic force sensing. Here, we report a highly sensitive upconversion nanocrystals-based behavioral biometric optical tactile sensor that instantaneously and quantitatively decomposes dynamic touch signals into individual components of vertical normal and lateral shear force from a single image in real-time. By mimicking the sensory architecture of human skin, the unique luminescence signal obtained is axisymmetric for static normal forces and non-axisymmetric for dynamic shear forces. Our sensor demonstrates high spatio-temporal screening of small objects and recognizes fingerprints for authentication with high spatial-temporal resolution. Using a dynamic force discrimination machine learning framework, we realized a Braille-to-Speech translation system and a next-generation dynamic biometric recognition system for handwriting.