CoSense: Deep Learning Augmented Sensing for Coexistence with Networking in Millimeter-Wave Picocells

Abstract

We present CoSense , a system that enables coexistence of networking and sensing on next-generation millimeter-wave (mmWave) picocells for traffic monitoring and pedestrian safety at intersections in all weather conditions. Although existing wireless signal-based object detection systems are available, they suffer from limited resolution and their outputs may not provide sufficient discriminatory information in complex scenes, such as traffic intersections. CoSense proposes using 5G picocells, which operate at mmWave frequency bands and provide higher data rates and higher sensing resolution than traditional wireless technology. However, it is difficult to run sensing applications and data transfer simultaneously on mmWave devices due to potential interference, and using special-purpose sensing hardware can prohibit deployment of sensing applications to a large number of existing and future inexpensive mmWave devices. Additionally, mmWave devices are vulnerable to weak reflectivity and specularity challenges, which may result in loss of information about objects and pedestrians. To overcome these challenges, CoSense design customized deep learning models that not only can recover missing information about the target scene but also enable coexistence of networking and sensing. We evaluate CoSense on diverse data samples captured at traffic intersections and demonstrate that it can detect and locate pedestrians and vehicles, both qualitatively and quantitatively, without significantly affecting the networking throughput.