FDFuzz: Applying Feature Detection to Fuzz Deep Learning Systems

摘要

In the past years, many resources have been allocated to research on deep learning networks for better classification and recognition. These models have higher accuracy and wider application contexts, but the weakness of easily being attacked by adversarial examples has raised our concern. It is widely acknowledged that the reliability of many safety-critical systems must be confirmed. However, not all systems have sufficient robustness, which makes it necessary to test these models before going into service. In this work, we introduce FDFuzz, an automated fuzzing technique that exposes incorrect behaviors of neural networks. Under the guidance of the neuron coverage metric, the fuzzing process aims to find those examples to let the network make mistakes via mutating inputs, which are then correctly classified. FDFuzz employs a feature detection technique to analyze input images and improve the efficiency of mutation by features of keypoints. Compared with TensorFuzz, the state-of-the-art open source library for neural network testing, FDFuzz demonstrates higher efficiency in generating adversarial examples and makes better use of elements in corpus. Although our mutation function consumes more time to generate new elements, it can generate 250% more adversarial examples and save testing time.