HMF: High-available Message-passing Framework for Cluster File System

摘要

In large-scale cluster systems, the failure rate of network connection is non-negligibly high. A cluster file system must have the ability to handle network failures in order to provide high-available data accesses service. Traditionally, network failure handling is only guaranteed by network protocol, or implemented within the file system semantic layer. We present the high-available message-passing framework which is called HMF. Based on the operation hierarchy in cluster file system, HMF guarantees the availability of each pair of network transmissions and their interaction with the file system sub-operations. It separates the network fault-tolerance design from the file system and keeps a simple interface between them. HMF could handle a lot of network failures internally, which greatly simplifies the implementation of file system semantic layer. Performance results show that HMF can increase the availability of message passing and reduce the cost of recovery from network failures. When there are two network channels, HMF also improves aggregate I/O bandwidth by 80% in normal condition while the performance degradation due to recovery is below 10%.