A High Performance Framework for Large-Scale 2D Convolution Operation on FPGA

摘要

Convolutional neural network (CNN), as the focus in the artificial intelligence field, has attracted more and more attention resent years. Various accelerators based on FPGA platform have been proposed because of the high performance and high energy efficiency. However, limited to the on-chip memory resources, only small networks can be accelerated by these accelerators. Due to the fact that 2D convolution operation is the most computing-intensive part, accelerating large-scale 2D convolution operation has become the key to accelerating large-scale CNNs on FPGA platforms. This paper presents a method of rotary data-storage and a design of the new PE (processing unit). Compared to the traditional Z-type PE, it performs non-redundant calculations when the stride of convolution operation is greater than 1, which significantly reduces the calculating time. At the same time, in order to reduce the external memory bandwidth with limited on-chip resources, two optimization architectures and a block-calculation method have been proposed, which can reduce the usage of on-chip memory resources. As a case study, we select some convolution layers of ResNet-34 and compare it with a previous accelerator. Under the condition of same consumption of resources, the result shows that the proposed framework needs only 29.9% time and 51% external memory bandwidth of the previous accelerator.