Smartphone-Based Cognitive Assistance of Blind People in Room Recognition and Awareness

Abstract

Blind people face multiple challenges in performing their daily activities, such as localization, navigation, and communication with other people following the social status of the ambient environment. The outdoor navigation and localization are mitigated through assistive technologies, such as white cane, smartphones, and Global Positioning System (GPS). However, little attention has been paid to assisting a blind person in judging the room environment, occupants in a room, communicating with an intended person, social status, and so on. This study attempts to provide cognitive assistance to blind people in predicting room types and finding the intended person to communicate their message by understanding their ambient environment and social status in terms of their age, gender, and the number of people inside a room. The proposed solution uses the microphone and speaker to recognize room types and a camera for understanding the ambient environment and social status. The information is conveyed to the blind person through haptic feedback. We analyzed different evaluation metrics, including the movement of people, ambient sounds, orientation, and position. We conducted an extensive user study to validate the proposed solution in real-world scenarios and achieved 87.64% accuracy in room type recognition in 11 rooms, 64.57% gender recognition, 61.73% age group, and 61.71% correctly identified the number of people.