Indoor Infrastructure Maintenance Framework Using Networked Sensors, Robots, and Augmented Reality Human Interface

Abstract

Sensing and cognition by homeowners and technicians for home maintenance are prime examples of human–building interaction. Damage, decay, and pest infestation present signals that humans interpret and then act upon to remedy and mitigate. The maintenance cognition process has direct effects on sustainability and economic vitality, as well as the health and well-being of building occupants. While home maintenance practices date back to antiquity, they readily submit to augmentation and improvement with modern technologies. This paper describes the use of networked smart technologies embedded with machine learning (ML) and presented in electronic formats to better inform homeowners and occupants about safety and maintenance issues, as well as recommend courses of remedial action. The demonstrated technologies include robotic sensing in confined areas, LiDAR scans of structural shape and deformation, moisture and gas sensing, water leak detection, network embedded ML, and augmented reality interfaces with multi-user teaming capabilities. The sensor information passes through a private local dynamic network to processors with neural network pattern recognition capabilities to abstract the information, which then feeds to humans through augmented reality and conventional smart device interfaces. This networked sensor system serves as a testbed and demonstrator for home maintenance technologies, for what can be termed Home Maintenance 4.0.