一种基于LDA的潜在语义区划分及Web文档聚类算法

摘要

该文应用LDA模型进行文档的潜在语义分析,将语义分布划分成低频、中频、高频语义区,以低频语义区的语义进行Web游离文档检测,以中、高频语义区的语义作为文档特征进行文档聚类,采用文档类别与语义互作用机制对聚类结果进行修正.与相关工作比较.该文不仅应用LDA模型表示文档.而且进行了深入的语义分布区域划分,并将分析结果应用于Web文档聚类.实验表明,该文提出的基于LDA的文档类别与语义互作用聚类算法获得了更好的聚类结果.%This paper applies the LDA model to analyze latent semantics of documents and partition the semantic space into low, middle and high frequency space. The semantics in low frequency space are used to detect outlier web documents. The semantics in middle and high frequency space are devoted to document clustering as features of the documents. The quality of clustering results is improved by a mutual-action mechanism between document clusters and semantics. Compared with related work, this paper not only applies LDA model to represent documents,but also analyzes the semantic distribution in depth and applies the results of analysis to web document clustering.Experiments show that the clustering algorithm of the mutual-action between LDA-based document class and semantic in this paper deserve better effects in document clustering.