Panel Labels Extraction from Multi-panel Figures for Facilitating Multi-modal Information Retrieval

摘要

The association of subfigures in the multi-panel figure with related text in the accompanying caption and research article is necessary for the implementation of multi-modal information retrieval system. The panel labels in the multi-panel figure are used as a source for making this kind of association. In this paper, we propose a novel method for the detection of panel labels in the multi-panel figures. The proposed method uses segmentation of multi-panel figure and its accompanying caption into subfigures and sub captions, respectively, as a preprocessing step. Next, the features of panel label, i.e., area and its distance from the borders in the upper left most subfigure of the multi panel figure are computed. These features are then used for detecting panel labels located in the rest of subfigures of the same multi-panel figure. Experiments on multi-panel figures selected from imageCLEF2013 dataset show promising results.