Human emotion recognition with a microcomb-enabled integrated optical neural network
Author:
Cheng Junwei12, Xie Yanzhao1ORCID, Liu Yu3, Song Junjie1, Liu Xinyu1, He Zhenming1, Zhang Wenkai1, Han Xinjie4, Zhou Hailong1ORCID, Zhou Ke1, Zhou Heng4, Dong Jianji12ORCID, Zhang Xinliang12
Affiliation:
1. Wuhan National Laboratory for Optoelectronics , Huazhong University of Science and Technology , Wuhan 430074 , China 2. Optics Valley Laboratory , Wuhan 430074 , China 3. School of Computer of Science and Technology , Huazhong University of Science and Technology , Wuhan 430074 , China 4. Key Lab of Optical Fiber Sensing and Communication Networks , University of Electronic Science and Technology of China , Chengdu 611731 , China
Abstract
Abstract
State-of-the-art deep learning models can converse and interact with humans by understanding their emotions, but the exponential increase in model parameters has triggered an unprecedented demand for fast and low-power computing. Here, we propose a microcomb-enabled integrated optical neural network (MIONN) to perform the intelligent task of human emotion recognition at the speed of light and with low power consumption. Large-scale tensor data can be independently encoded in dozens of frequency channels generated by the on-chip microcomb and computed in parallel when flowing through the microring weight bank. To validate the proposed MIONN, we fabricated proof-of-concept chips and a prototype photonic-electronic artificial intelligence (AI) computing engine with a potential throughput up to 51.2 TOPS (tera-operations per second). We developed automatic feedback control procedures to ensure the stability and 8 bits weighting precision of the MIONN. The MIONN has successfully recognized six basic human emotions, and achieved 78.5 % accuracy on the blind test set. The proposed MIONN provides a high-speed and energy-efficient neuromorphic computing hardware for deep learning models with emotional interaction capabilities.
Funder
National Natural Science Foundation of China Innovation Project of Optics Valley Laboratory National Key Research and Development Program of China
Publisher
Walter de Gruyter GmbH
Subject
Electrical and Electronic Engineering,Atomic and Molecular Physics, and Optics,Electronic, Optical and Magnetic Materials,Biotechnology
Reference55 articles.
1. S. Ji, W. Xu, M. Yang, and K. Yu, “3D convolutional neural networks for human action recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 221–231, 2013, https://doi.org/10.1109/tpami.2012.59. 2. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015, https://doi.org/10.1038/nature14539. 3. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, 2017, https://doi.org/10.1145/3065386. 4. S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving,” J. Field Robot., vol. 37, no. 3, pp. 362–386, 2020, https://doi.org/10.1002/rob.21918. 5. B. R. Kiran, I. Sobh, V. Talpaert, et al.., “Deep reinforcement learning for autonomous driving: a survey,” IEEE Trans. Intell. Transport. Syst., vol. 23, no. 6, pp. 4909–4926, 2022, https://doi.org/10.1109/tits.2021.3054625.
|
|