Design and implementation of an adaptive job allocation strategy for heterogeneous multi-cluster computing systems

摘要

Cluster computing is an attractive approach to provide high-performance computing for solving large-scale applications. Owing to the advances in processor and networking technology, expanding clusters have resulted in the system heterogeneity; thus, it is crucial to dispatch jobs to heterogeneous computing resources for better resource utilization. In this paper, we propose a new job allocation system for heterogeneous multi-cluster environments named the Adaptive Job Allocation Strategy (AJAS), in which a self-scheduling scheme is applied in the scheduler to dispatch jobs to the most appropriate computing resources. Our strategy focuses on increasing resource utility by dispatching jobs to computing nodes with similar performance capacities. By doing so, execution times among all nodes can be equalized. The experimental results show that AJAS can improve the system performance.