A separable neural code in monkey IT enables perfect CAPTCHA decoding

Abstract

ABSTRACTReading distorted letters is easy for us but so challenging for machine vision that it is used on websites as CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart). How does our brain solve this problem? One solution is to have neurons invariant to letter distortions but selective for letter combinations. Another is for neurons to separately encode letter distortions and combinations. Here, we provide evidence for the latter using neural recordings in the monkey inferior temporal (IT) cortex. Neurons encoded letter distortions as a product of letter and distortion tuning, and letter combinations as a sum of letters. These rules were sufficient for perfect CAPTCHA decoding and were also present in neural networks trained for word recognition. Taken together, our findings suggest that a separable neural code enables efficient letter recognition.