基于不完备数据聚类的缺失数据填补方法

摘要

缺失数据的处理是数据挖掘领域进行数据预处理的一个重要问题.传统的缺失数据填补方法大部分是基于概率分布等一些统计假设,对于大数据集的数据挖掘不一定是最适合的方法.受不完备数据分析(ROUSTIDA)未采用传统的概率统计学方法启发,提出基于不完备数据聚类的缺失数据填补方法(MIBOI),针对分类变量不完备数据集定义约束容差集合差异度,直接计算不完备数据对象集合内所有对象的总体相异程度,以不完备数据聚类的结果为基础进行缺失数据的填补.采用UCI机器学习基准数据集进行实验表明,MIBOI对缺失数据的填补是有效可行的.%Missing data processing is an important problem of data pre-processing in data mining field. Traditional missing data filling methods are mostly based on some statistical hypothesis, such as probability distribution,which might not be the most applicable approaches for data mining of large data set. Inspired by ROUSTIDA,an incomplete data analysis approach not using probability statistical methods, MIBOI is proposed for missing data imputation based on incomplete data clustering. Constraint Tolerance Set Dissimilarity is defined for incomplete data set of categorical variables,so the general dissimilarity of all the incomplete data objects in a set can be directly computed,and the missing data is imputed according to the incomplete data clustering results. The empirical tests using UCI benchmark data sets show that MIBOI is effective and feasible.