大数据关键处理技术综述

摘要

Big data is a disruptive technological revolution ,in IT field ,after the cloud computing and EPC system net‐work ,and big data development and research will change the world .The conceptions and characteristics of big data ,its de‐velopment course ,and the relationship between big data and cloud computing are introduced .Then the more mature plat‐form ,Spark and Hadoop of big data analysis and processing are described .And some key techniques for big data processing are systematically analyzed ,summarized and discussed ,such as big data acquisition ,big data preprocessing ,big data storage and management ,big data analysis and mining ,and statistical analysis of big data .%大数据是继云计算、物联网之后 IT 产业又一次颠覆性的技术革命，大数据的发展、研究必将改变世界。先简介大数据的概念及其特征、大数据发展历程、大数据与云计算的关系；接着叙述了大数据分析和处理的比较成熟的平台：Spark 和 Hadoop ；然后对大数据处理的若干关键技术：大数据采集、大数据预处理、大数据的存储及管理、大数据的分析和挖掘、大数据的统计分析等进行了较系统的分析、归纳和探讨。