Real-time batch processing at a GPU-based edge with a passive optical network

Abstract

In recent years, advances in deep learning technology have significantly improved the research and services relating to artificial intelligence. Real-time object recognition is an important technique in smart cities, in which low-cost network deployment and low-latency data transfer are key technologies. In this study, we focus on time- and wavelength-division multiplexed passive optical network (TWDM-PON)-based inference systems to deploy cost-efficient networks that accommodate several network cameras. A significant issue for a graphics processing unit (GPU)-based inference system via a TWDM-PON is the optimal allocation of the upstream wavelength and bandwidth to enable real-time inference. However, an increase in the batch size of the arrival data at the edge servers, thereby ensuring low-latency transmission, has not been considered previously. Therefore, this study proposes the concept of an inference system in which a large number of cameras periodically upload image data to a GPU-based server via the TWDM-PON. Moreover, we propose a cooperative wavelength and bandwidth allocation algorithm to ensure low-latency and time-synchronized data arrivals at the edge. The performance of the proposed scheme is verified through a computer simulation.