High performance in minimizing of term-document matrix representation for document clustering

摘要

Document clustering usually involves high dimensional term space, which makes it difficult for organizing data into a small number of meaningful clusters. Clustering based on similar terms without considering the content or meaning is often unsatisfactory as it ignores the relationship between important terms that do not co-occur literally. In this paper, we propose to integrate the latent semantic indexing (LSI) concept to our document clustering. This involves the use of singular value decomposition (SVD) which creates a new abstract and uses a way of finding pattern document collection in matrix representation, so that it can identify between the terms and documents which are similar. By using various numbers of patterns (rank) of SVD, the proposed method is applied to cluster documents using the fuzzy C-means algorithm. The results of the experiment show that the performance of document clustering to be better when applied to the LSI method.