The impact of applications' I/O strategies on the performance of the Lustre parallel file system

摘要

Parallel applications present multiple approaches regarding the management of data. Due to specific characteristics of parallel file systems, some approaches will provide better performance than others due to a better matching to the system's internals.One common situation is when each instance of an application accesses exclusive data stored in the file system. This paper studies some I/O techniques for this situation and evaluates them on the Lustre file system. We provide a guide to help developers tune their application to extract the best performance out of Lustre. Our results show expressive gains in performance related with the choice of access pattern of the application. We present considerations on operation granularity, intra-node concurrency and temporal behaviour of the application.