Naturalistic E-Scooter Maneuver Recognition with Federated Contrastive Rider Interaction Learning

Abstract

Smart micromobility, particularly the electric (e)-scooters, has emerged as an important ubiquitous mobility option that has proliferated within and across many cities in North America and Europe. Due to the fast speed (say, ~15km/h) and ease of maneuvering, understanding how the micromobility rider interacts with the scooter becomes essential for the e-scooter manufacturers, e-scooter sharing operators, and rider communities in promoting riding safety and relevant policy or regulations. In this paper, we propose FCRIL, a novel Federated maneuver identification and Contrastive e-scooter Rider Interaction Learning system. FCRIL aims at: (i) understanding, learning, and identifying the e-scooter rider interaction behaviors during naturalistic riding (NR) experience (without constraints on the data collection settings); and (ii) providing a novel federated maneuver learning model training and contrastive identification design for our proposed rider interaction learning (RIL). Towards the prototype and case studies of FCRIL, we have harvested an NR behavior dataset based on the inertial measurement units (IMUs), e.g., accelerometer and gyroscope, from the ubiquitous smartphones/embedded IoT devices attached to the e-scooters. Based on the harvested IMU sensor data, we have conducted extensive data analytics to derive the relevant rider maneuver patterns, including time series, spectrogram, and other statistical features, for the RIL model designs. We have designed a contrastive RIL network which takes in these maneuver features with class-to-class differentiation for comprehensive RIL and enhanced identification accuracy. Furthermore, to enhance the dynamic model training efficiency and coping with the emerging micromobility rider data privacy concerns, we have designed a novel asynchronous federated maneuver learning module, which asynchronously takes in multiple sets of model gradients (e.g., based on the IMU data from the riders' smartphones) for dynamic RIL model training and communication overhead reduction. We have conducted extensive experimental studies with different smartphone models and stand-alone IMU sensors on the e-scooters. Our experimental results have demonstrated the accuracy and effectiveness of FCRIL in learning and recognizing the e-scooter rider maneuvers.