Unsupervised Factory Activity Recognition with Wearable Sensors Using Process Instruction Information

Abstract

This paper presents an unsupervised method for recognizing assembly work done by factory workers by using wearable sensor data. Such assembly work is a common part of line production systems and typically involves the factory workers performing a repetitive work process made up of a sequence of manual operations, such as setting a board on a workbench and screwing parts onto the board. This study aims to recognize the starting and ending times for individual operations in such work processes through analysis of sensor data collected from the workers along with analysis of the process instructions that detail and describe the flow of operations for each work process. We propose a particle-filter-based factory activity recognition method that leverages (i) trend changes in the sensor data detected by a nonparametric Bayesian hidden Markov model, (ii) semantic similarities between operations discovered in the process instructions, (iii) sensor-data similarities between consecutive repetitions of individual operations, and (iv) frequent sensor-data patterns (motifs) discovered in the overall assembly work processes. We evaluated the proposed method using sensor data from six workers collected in actual factories, achieving a recognition accuracy of 80% (macro-averaged F-measure).