Unsupervised Human Activity Representation Learning with Multi-task Deep Clustering

Abstract

Human activity recognition (HAR) based on sensing data from wearable and mobile devices has become an active research area in ubiquitous computing, and it envisions a wide range of application scenarios in mobile social networking, environmental context sensing, health and well-being monitoring, etc. However, activity recognition based on manually annotated sensing data is manpower-expensive, time-consuming, and privacy-sensitive, which prevents HAR systems from being really deployed in scale. In this paper, we address the problem of unsupervised human activity recognition, which infers activities from unlabeled datasets without the need of domain knowledge. We propose an end-to-end multi-task deep clustering framework to solve the problem. Taking the unlabeled multi-dimensional sensing signals as input, we firstly apply a CNN-BiLSTM autoencoder to form a compressed latent feature representation. Then we apply a K-means clustering algorithm based on the extracted features to partition the dataset into different groups, which produces pseudo labels for the instances. We further train a deep neural network (DNN) with the latent features and pseudo labels for activity recognition. The tasks of feature representation, clustering, and classification are integrated into a uniform multi-task learning framework and optimized jointly to achieve unsupervised activity classification. We conduct extensive experiments based on three public datasets. It is shown that the proposed approach outperforms shallow unsupervised learning approaches, and it performs close to the state-of-the-art supervised approaches by fine-tuning with a small number of labeled data. The proposed approach significantly reduces the cost of human-based data annotation and narrows down the gap between unsupervised and supervised human activity recognition.