Attachable Inertial Device with Machine Learning toward Head Posture Monitoring in Attention Assessment

Abstract

The monitoring of head posture is crucial for interactive learning, in order to build feedback with learners’ attention, especially in the explosion of digital teaching that occurred during the current COVID-19 pandemic. However, conventional monitoring based on computer vision remains a great challenge in the multi-freedom estimation of head posture, owing to low-angle annotation and limited training accuracy. Here, we report a fully integrated attachable inertial device (AID) that comfortably monitors in situ head posture at the neck, and provides a machine learning-based assessment of attention. The device consists of a stretchable inertial sensing unit and a fully integrated circuit-based system, as well as mechanically compliant encapsulation. Due to the mechanical flexibility, the device can be seamlessly attach to a human neck’s epidermis without frequent user interactions, and wirelessly supports six-axial inertial measurements, thereby obtaining multidimensional tracking of individual posture. These head postures (40 types) are then divided into 10 rotation actions which correspond to diverse situations that usually occur in daily activities of teaching. Benefiting from a 2D convolutional neural network (CNN)-based machine learning model, their classification and prediction of head postures can be used to analyze and infer attention behavior. The results show that the proposed 2D CNN-based machine learning method can effectively distinguish the head motion posture, with a high accuracy of 98.00%, and three actual postures were successfully verified and evaluated in a predefined attention model. The inertial monitoring and attention evaluation based on attachable devices and machine learning will have potential in terms of learning feedback and planning for learners.