SLEXNet: Adaptive Inference Using Slimmable Early Exit Neural Networks

Abstract

Deep learning is a proven method in many applications. However, it requires high computation resources and usually has a constant architecture. Mobile systems are good candidates to benefit from deep learning applications since they are closely integrated in people’s life. However, mobile systems experience varying conditions for the same reason. Constant deep learning architectures against varying resources cannot satisfy the requirements of the applications, so dynamic deep learning architectures are needed. In this work, we propose SLEXNet, a slimmable early exit neural network architecture. SLEXNet combines dynamic depth and width architectures to adapt to varying time and power conditions. Moreover, we propose a runtime scheduling algorithm that can estimate inference time and power consumption of SLEXNet variations on runtime. We train SLEXNet on real aerial drone images and implement the runtime on NVIDIA Jetson Orin. We show that our approach achieves significantly better responses to time and power requirements in varying conditions than baseline dynamic depth and width techniques in a wide range of experiments.