基于Hadoop技术的数字化校园海量数据存储系统研究与设计

摘要

With the rapid growth of digital teaching resources of colleges and universities ,the storage and application of big data have become a new topic in the study of digital campus technology in military colleges and universities .There are many shortcomings in traditional data processing methods ,such as high cost ,data management difficulty ,low reliability ,inefficiency ,parallel processing programming difficulty ,etc .Hadoop , which is an open source ,distributed data processing framework being used to handle huge amounts of data efficiently ,has the scalability ,high reliability ,low cost ,high efficiency ,etc .In view of the problems existing in traditional storage architecture ,the big data storage model based on Hadoop technology is proposed ,and on the basis ,the Hadoop distributed data storage system is designed .%基于 Hadoop技术开发了数字化校园海量数据存储系统，提出了基于 Hadoop的大数据存储模型。将整个系统设计为系统管理、业务应用、数据处理、数据采集等4个大模块，采用开源分布式数据处理框架，高效处理海量数据。系统具有可伸缩性、高可靠性、低成本和高效性等优点，解决了传统数据处理成本高、数据管理困难、可靠性低、效率低、并行处理程序编写困难等诸多问题。