A decisive content based image retrieval approach for feature fusion in visual and textual images

摘要

Image content analysis plays a dynamic role in various computer vision applications. These contents can be either visual (i.e. color, shape, texture) or the textual (i.e. text appearing within images). Both the contents involve fundamental characteristics of an image and thus can be an enormous asset for any intelligent application. For content based image retrieval (CBIR) systems, most of the art methods are either annotated text based or the visual search based. Due to high demand of multitasking, there is a great need of a system that can combine visual as well as textual features. Consequently, this work proposes a decisive CBIR approach that combines visual and textual features to retrieve similar images. Firstly, the method classifies the query image as textual and non-textual. If any text appears within the image then the query image is classified as textual, and the text is detected and formed as Bag of Textual words. If the query image is classified as non-textual, the visual salient features are extracted and formed as Bag of Visual words. Next, the method fuses the visual and textual features, and top similar images are retrieved based on the fused feature vector. It supports three modes of retrieval: Image query, Keywords, and a combination of both. The experimental results on four datasets show the efficiency and accuracy of the proposed approach for visual and textual images. (C) 2019 Elsevier B.V. All rights reserved.