Meta-model Evolution with Self-Adaptation: An Empirical Development Approach for Distributed Parallel Computing Framework

摘要

Distributed parallel computing platform performs well in processing big data. However, due to the platform's complexity and distributed characteristics, it is hard to design and achieve. For example, during the platform's design phase, variations are unpredictable. To address these issues, a highly structured object-oriented framework for systematic modeling, which has high flexibility, reusability and maintainability, is needed. In this paper, we present and implement an empirical development approach to achieve the specific modeling framework, based on meta-model evolution with hot-spots (variable parts) recognition. Our work expands this idea into the following directions. Firstly, we define a generic and stable meta-model. We give out concept models, object models and dynamic models of main objects. General data partitioning, job division and inter-node communication mechanism are designed. Secondly, based on the meta-model, we complete framework's incremental adaptation using an iterative process according to design variations. Practical experience of our empirical case study shows, the approach gets a relatively mature framework for design theory instruction. The integrated framework clearly expresses main objects and their relationship. This work promotes the development of parallel computing framework to have a qualitative leap, and eventually have more extensive application and popularization of the established framework.