RTA: Real Time Actionable Events Detection as a Service

摘要

Nowadays vast amounts of data are being produced in continuous ways. They may come from sensors, smart meters, application logs, monitoring software etc. The data need to be processed in realtime to gain actionable insights. Services like smart grid load balancing, cloud platform maintenance, can be carried out in an efficient way. Stream processing is the programming paradigm that answers such demand. When talking about stream processing, we can easily recall several famous open-source software frameworks such as Spark Streaming, Samza, Flink and Storm. Although they provide distributed, robust, low-latency stream processing engines, it's still difficult for an end user to set up a usable stream processing application from scratch. Firstly, users are required to write code to define their business related stream processing logic. Secondly, the submission and update of the stream processing logic require service restart, therefore it may lead to service unavailability for minutes. Thirdly, extra operation effort are required for handling scaling and failover issues. In this paper, we present RTA, a released research service on realtime data processing. The RTA service fills the gap between the stream processing requester and the existing software stacks. It offers a SQL-like stream query language for defining stream processing logic definition over streaming data. It allows users easily define their stream processing logic without programming. In RTA service, stream processing logic is also treated as a type of input, which enables online logic update without service downtime. The RTA service also provides scalability, high availability and resource isolation for serving multiple tenants. In this paper, we also provide a comprehensive evaluation of our service through a case study.