LOG DATA MINING METHOD AND SYSTEM BASED ON HADOOP

摘要

A log data mining method based on Hadoop, comprising: saving a first log data set over an acquired current time period into a Hadoop database (101); if the number of first log data sets saved in the Hadoop database satisfies a pre-set numerical value, performing parallel aggregation processing on the first log data sets in the Hadoop database using a pre-set parallel arithmetic model, so as to get a second log data set (102); and according to the dimension of the log data in the second log data set, performing dimension division on the log data in the second log data set, and saving a third log data set corresponding to obtained different dimensions into the Hadoop database(103). The method can rapidly and effectively realize mass data mining, and satisfy the storage and operation requirement for the mass data mining.