Analysing Children’s Responses from Multiple Modalities During Robot-Assisted Assessment of Mental Wellbeing

Abstract

AbstractAccording to the World Health Organization, the early identification of mental wellbeing issues in children is extremely important for children’s growth and development. However, the available health services are not sufficient to address children’s needs in this area. Literature suggests that robots can provide the support needed to promote mental wellbeing in children, but how robots can help with the assessment of mental wellbeing is relatively unexplored. Hence, this work analyses multiple data modalities collected in an exploratory study involving 41 children (8–13 years old, 21 females and 20 males) who interacted with a Nao robot for about 30–45 min. During this session, the robot delivered four tasks: (1) happy and sad memory recall, (2) the Short Moods and Feelings Questionnaire (SMFQ), (3) the picture-based task inspired by the Children Appreciation Test (CAT), and (4) the Revised Children Anxiety and Depression Scale (RCADS). We clustered the participants into three groups based on their SMFQ scores as follows: low tertile (16 participants), med tertile (12 participants), and high tertile (13 participants). Then, we described and analysed the data collected from multiple sources (i.e., questionnaires responses, audio-visual recordings, and speech transcriptions) to gather multiple perspectives for understanding how children’s responses and behaviours differ across the three clusters (low, med, vs high) and their gender (boys vs girls) for robot-assisted assessment of mental wellbeing. Our results show that: (i) the robotised mode is the most effective in the identification of wellbeing-related concerns with respect to standardised modes of administration (self-report and parent-report); (ii) children less likely to have mental wellbeing concerns displayed more expressive responses than children who are more likely to have mental wellbeing concerns; and (iii) girls who are more likely to have mental wellbeing concerns displayed more expressive responses than boys, while boys who are less likely to have mental wellbeing concerns displayed more expressive responses than girls. Findings from this work are promising for paving the way towards automatic assessment of mental wellbeing in children via robot-assisted interactions.