Dynamic job scheduling system and method for supporting real-time stream data processing in distributed in-memory environment

摘要

In the dynamic job scheduling method in the dynamic job scheduling system for real-time stream data processing in the distributed in-memory environment of the present invention, when the stream data is input, the CPU usage, the size of the idle memory, A step of calculating a load of each node, a step of assigning priority to each node according to a load of each node, and performing static job scheduling to distribute the job, Managing the load information consumed by the load table in the load table and periodically checking the information of the load table to search for the cause of the load and determining the CPU usage, And redistributing the job through dynamic job scheduling. According to the present invention, it is possible to prevent a job delay and loss due to a load on a node and improve data processing performance.