KAFKA System and method for transmission of data in real time using multiple KAFKA

摘要

A real-time data transmission system using a multiple KAFKA, according to the present invention, comprises: a sensing data layer in which sensing data is collected from a PLC, a message is created by organizing the collected data into data with a human readable structure, and a KAFKA producer encrypts the message with a public key and transmits the same to a KAFKA server; a KAFKA server layer as a broker layer in which the KAFKA server notifies a KAFKA consumer to subscribe to a newly arrived message when the message is received from the KAFKA producer, and stores the received message; and a data consumption and processing layer in which the KAFKA consumer subscribes to the new message, decrypts the message received from the KAFKA server using a private key, and analyzes the message using a spark engine. According to the present invention, data streaming can be performed very efficiently by transmitting the sensing data, collected from the plurality of PLCs, in real time by using the multiple KAFKA.