气象网格环境下大数据的端到端传输机制研究

摘要

以八个区域中心建立气象网格计算环境(CMAGrid)，有效整合利用各区域中心现有的高性能计算资源，以提高气象行业资源利用率和气象作业运行效率。气象作业运行过程中的输入和输出都需要在各区域中心之间进行数据传输，大量的气象数据文件较大，因此，在 CAMGrid 环境下实现高效的气象大数据传输是个值得研究的课题。本文重点研究CMAGrid环境中各区域中心间端到端的气象大数据的高效传输机制。在气象网格环境搭建的基础上，根据气象大数据的特点，分析影响大数据传输效率的关键问题，基于缓冲区、数据切分、多路并行传输等机制研究一种大数据的端到端高效传输机制，有效利用现有的网络带宽，以提高气象骨干网环境下的数据传输能力。%In order to improve the using efficiency of existed meteorological resources and the running efficiency of mete-orological jobs,we established a Meteorological Grid (CMAGrid)environment with the high performance computing re-sources of eight regional centers.The input and output of meteorological jobs in the running process need transfer data in the regional centers.The capacity of the meteorological data file is always big,therefore,it is a worthy study topic to improve transmission efficiency of the meteorological data in the CMAGrid environment.This paper focuses on the End-to-End effi-cient of transmission mechanism of the big meteorological data between the regional centers in the CMAGrid environment. According to the characteristics of meteorological data and he CMAGrid environment,this paper analyzes the key problems that affects the transfer efficiency of the big data and designs an efficient end-to-end multipath parallel data transfer mecha-nism based on the technology of buffer area,data segmentation and multipath parallel transfer and so on.It aims to fully use the transfer capacity of the entire network and improve the data transmission efficiency in the CMAGrid environment.