基于MapReduce的ID3决策树分类算法研究

摘要

决策树算法是经典的分类挖掘算法之一,具有广泛的实际应用价值.经典的ID3决策树算法是内存驻留算法,只能处理小数据集,在面对海量数据集时显得无能为力.为此,对经典ID3决策树生成算法的可并行性进行了深入分析和研究,利用云计算的MapReduce编程技术,提出并实现面向海量数据的ID3决策树并行分类算法.实验结果表明该算法是有效可行的.%Decision tree is widely used in data mining which is one of the typical classification algorithms. Traditional ID3 tree learning algorithms require training data to reside in memory on a single machine, so they cannot deal with massive datasets. To solve this problem, this paper analyzes the parallel algorithm of ID3 decision tree based on MapReduce model, then proposes a parallel and distributed algorithm for ID3 decision tree learning. The experimental results demonstrate the algorithm can scale well and efficiently process large-scale datasets on commodity computers.