An Incremental Outlier Detection Model for Transaction Data Streams

摘要

Outliers are large deviate from others data points. They are often not the errors, and may carry important information. This paper considers the outlier detection problem for transaction data streams. We present IODM (Incremental Outlier Detection Model), a fully automated transaction data streams outlier detection model. The IODM designs three crucial parts. First, we choose a transaction dataset from database. Then, mining association rules in the dataset, and building a basic Association Rule Warehouse (ARW). Second, we design a synopsis data structure to monitor and maintain the constantly coming transactions in a sliding window. Meanwhile, we introduce the concept of Match Difference Degree (MDD) as a measurement standard for detecting outlier transactions. Third, in the data streams environment, the infrequent items may be become frequent later on and hence cannot be ignored. Therefore, the ARW needs to be dynamically adjusted to adapt to the evolution of data streams. So we maintain and incrementally update the ARW with the arrived transaction data streams. Besides, we design a scalable outlier detection algorithm by the IODM to analyze and detect outlier transactions. Finally, we have conducted experiment using real datasets and the result show that the approach by the IODM can detect outlier transactions efficiently from concept-drift data streams in a fully automated manner.