数据预处理技术在地学大数据中应用

摘要

大数据时代随着数据的爆发式增长,在带来可供研究的海量数据的同时,也带来巨量的噪声和冗余数据.在地学领域,由于研究方向和技术方法手段的多样化,产生了数据量巨大和类型众多的地学数据集合.在地学信息的研究过程中,经常碰到地学信息孤岛,分图幅地学数据边界系统误差和地学文档的非结构化问题.在对地学数据进行信息的提取和挖掘之前,必须根据研究目的对地学大数据进行预处理,使冗余、复杂的大数据转为结构化、准确、可用的数据.本文以地学大数据的预处理技术为切入点,从地学数据交互标准与语义、数据调平、地质图接边和文本结构化等四个研究方面,分析阐述目前地学大数据挖掘方面存在的问题及主要的解决手段,同时也对多元数据融合在大数据中的应用进行了阐述.希望通过本文对地学大数据预处理技术的探讨,能对地学大数据的挖掘有所帮助.%The big data era brings an explosive increment of data.Although it provides large amount of useful data for study,it also produces massive noise information and redundancy data.In the field of geosciences,diversified research interests and techniques result in enormous and various geoscience data collections.In the research of Geoinformatics,we face the challenges of information islands,system error along boundaries of different map sheets or non-structural information in geoscience literature.Before data mining and information extraction,it is necessary to conduct data pre-processing,such that transfer the redundancy,complex big data into structured,correct and usable data that will address the research objectives.In this paper,we analyze the above challenges in big data application,and discuss methods and technologies of data pre-processing in geosciences.It includes standard of geoscience data and semantics,data leveling,edge processing of geological map and structuralization of text literature from the perspective of data preprocessing.The paper also discusses the application of data fusing in geoscience big data.We hope the discussion about big data preprocessing will provide a few new thoughts for the geoscience community and can benefit efforts on data mining for geoscience big data.