Efficient Design of Low Bitwidth Convolutional Neural Networks on FPGA with Optimized Dot Product Units

Abstract

Designing hardware accelerators to run the inference of convolutional neural networks (CNN) is under intensive research. Several different architectures have been proposed along with hardware-oriented optimizations of the neural network models. One of the most used optimizations is quantization since it reduces the memory requirements to store weights and layer maps, the memory bandwidth requirements and the hardware complexity. As a consequence, the inference throughput has improved and the computing cost has been reduced, allowing inference to be executed on embedded devices. In this work, we propose highly efficient dot-product arithmetic units for ternary and non-ternary convolutional neural networks on FPGA. The non-ternary dot-product unit uses a fused multiply-add that avoids expensive adder trees, while the ternary dot-product unit uses a dual product unit followed by an optimized conditional adder tree structure. In both cases, designs with and without embedded DSP are considered. The solution is configurable and can be adapted to the available number of resources of the FPGA to achieve the best efficiency. A CNN architecture was developed and characterized using the proposed dot product units. The results show a performance improvement of 1.8 × with a 2× more area efficiency for low bit-width quantizations when compared to previous works running large CNNs in FPGA.