Humanoid Intelligent Display Platform for Audiovisual Interaction and Sound Identification

Abstract

AbstractThis study proposes a rational strategy for the design, fabrication and system integration of the humanoid intelligent display platform (HIDP) to meet the requirements of highly humanized mechanical properties and intelligence for human–machine interfaces. The platform's sandwich structure comprises a middle light-emitting layer and surface electrodes, which consists of silicon elastomer embedded with phosphor and silk fibroin ionoelastomer, respectively. Both materials are highly stretchable and resilient, endowing the HIDP with skin-like mechanical properties and applicability in various extreme environments and complex mechanical stimulations. Furthermore, by establishing the numerical correlation between the amplitude change of animal sounds and the brightness variation, the HIDP realizes audiovisual interaction and successful identification of animal species with the aid of Internet of Things (IoT) and machine learning techniques. The accuracy of species identification reaches about 100% for 200 rounds of random testing. Additionally, the HIDP can recognize animal species and their corresponding frequencies by analyzing sound characteristics, displaying real-time results with an accuracy of approximately 99% and 93%, respectively. In sum, this study offers a rational route to designing intelligent display devices for audiovisual interaction, which can expedite the application of smart display devices in human–machine interaction, soft robotics, wearable sound-vision system and medical devices for hearing-impaired patients.