Sketch4Image: a novel framework for sketch-based image retrieval based on product quantization with coding residuals

摘要

Sketch-based Image Retrieval (SBIR) is one important branch of Content-based Image Retrieval (CBIR). SBIR means dealing with retrieval using simple edge or contour images. However, SBIR is more difficult than CBIR due to the lack of visual information, this makes the Bag-of-Words (BoW) or codebook in SBIR hard to construct. In this paper, we propose a novel SBIR framework based on Product Quantization (PQ) with sparse coding (SC) to construct an optimized codebook. By using state-of-the-art local descriptors, we transform sketch images into features and then build the optimized codebook using PQ-based SC. In the retrieval stage, we can obtain a better representation of the query sketch and testing images by the optimized codebook with coding quantization residuals, by which the information loss during feature encoding process can be reduced; similarity computing is implemented by comparing the feature histograms between a query sketch and the testing data for the final results. We demonstrate the superiority and effectiveness of the proposed SBIR by comparing it with several state-of-the-art methods on three public sketch datasets.