HW/SW Co-Design of Cost-Efficient CNN Inference for Cognitive IoT

摘要

Cognitive Internet of things (IoT) is a novel paradigm that outfits the contemporary IoT with a “brain” to impart high-level intelligence. Convolutional neural networks (CNNs) are an integral part of cognitive IoT that support inference and decision-making. In this paper, we demonstrate a resource-efficient hardware/software (HW/SW) co-design of a CNN architecture for cognitive IoT. We only offload image-tocolumn (im2col) and general matrix multiply (GEMM), which are the most time- and energy-consuming part of convolution layer operations, to the field-programmable gate array (FPGA)-based accelerator. We also exploit the parallelism in the operations of convolution layers to efficiently hide a non-negligible portion of execution time required for bias and activation. Experimental results demonstrate the resource, performance, and energy efficiency of our HW/SW co-design. Results indicate a speedup of $1.3mathrm{X} sim 2.0mathrm{X}$ and energy reduction of 19.4%$sim 44.3$% as compared to using only a general-purpose processor.