BRPS: A Big Data Placement Strategy for Data Intensive Applications

摘要

The Market of Data is an environment where data are reasonably deal with. Some data in the market of data are large and hard to analyze. How to efficiently analyze and organize such large scale data in the market of data is a difficult problem. When using Hadoop to analyze these massive data, if input data of a data mining task are not locally available in a processing node, data have to be migrated via network interconnects to node that performs the data processing operations. These data movement obviously has a bad effect on system performance. In this paper, we propose BRPS (Big data Replicas Placement Strategy), a strategy that improves data intensive tasks parallel execution performance by reducing data movement across multiple machines. The simulation results show that BRPS can greatly reduce the data movement cost and promote workload balance slightly.