面向维吾尔文不平衡数据分类的特征选择方法

摘要

为解决维吾尔文文本分类中不平衡数据集问题,提出了一种改进的卡方特征选择方法.结合维吾尔文的语言特性对文本进行预处理,降低特征空间维度；运用卡方和逆文档频数相结合的方法进行特征选择,进一步降低特征空间维数；使用朴素贝叶斯分类器进行分类.在维吾尔文不平衡语料库上进行的实验表明,提出的特征选择方法在不平衡数据集中要优于卡方和信息增益特征选择方法.%To solve the problem of imbalance data sets in Uyghur text classification, an improved CHI (Chi-square) feature selection method is proposed. Firstly, text pre-processing is carried out to reduce the feature space based on the Uyghur language features; then the combining feature selection method of CHI and IDF (inverse document frequency) is used to further reduce the feature space dimension. Finally Na? ve Bayesian is used to be the classifier. The experimental results show that the effectiveness and the feasibility of the proposed feature selection method in imbalanced data set and it is superior to CHI and IG.