Term expansion on the categorization of summarized documents

摘要

Several researches emphasize on the application of using documents' summaries as feature vector inputs in text categorization tasks. The performance of this kind of approach is often poor when the coverage rate of summarization is low with oversimplified feature vectors. Therefore, in this study we propose to expand the terms in the summaries using supervised distributional clustering method to improve the categorization performance. In the training stage of our approach, we input documents' summaries to generate classifiers (KNN and Naive Bayes) and term clusters (using KL divergence as dissimilarity measure) as well. In the test stage, we classify a new document by inputting its expanded feature vector of its summary into the generated classifiers. That is, terms in the feature vector will be expanded using related terms in the same cluster in order to alleviate the term mismatch problem. Three experiments are conducted accordingly. The results show that our proposed approach can effectively resolve the problem of term mismatch problem and improve the categorization accuracy. In a word, our approach makes the idea of using automatic summarization to replace for the feature selection in text categorization tasks more practical and feasible.