Extracting Information From Heterogeneous Internet of Things Data Streams

摘要

Recent advancements in sensing, networking technologies and collecting real-world data on a large scale and from various environments have created an opportunity for new forms of services and applications. This is known under the umbrella term of the Internet of Things (IoT). Physical sensor devices constantly produce very large amounts of data. Methods are needed which give the raw sensor measurements a meaningful interpretation for building automated decision support systems. One of the main research challenges in this domain is to extract actionable information from real-world data, that is information that can readily be used to make informed automatic decisions in intelligent systems. Most existing approaches are application or domain dependent or are only able to deal with specific data sources of one kind. This PhD research concerns multiple approaches for analysing IoT data streams. We propose a method which determines how many different clusters can be found in a stream based on the data distribution. After selecting the number of clusters, we use an online clustering mechanism to cluster the incoming data from the streams. Our approach remains adaptive to drifts by adjusting itself as the data changes. The work is benchmarked against state-of-the art stream clustering algorithms on data streams with data drift. We show how our method can be applied in a use case scenario involving near real-time traffic data. Our results allow to cluster, label and interpret IoT data streams dynamically according to the data distribution. This enables to adaptively process large volumes of dynamic data online based on the current situation. We show how our method adapts itself to the changes and we demonstrate how the number of clusters in a real-world data stream can be determined by analysing the data distributions.;Using the ideas and concepts of this approach as a starting point we designed another novel dynamic and adaptable clustering approach that is more suitable for multi-variate time-series data clustering. Our solution uses probability distributions and analytical methods to adjust the centroids as the data and feature distributions change over time. We have evaluated our work against some well-known time-series clustering methods and have shown how the proposed method can reduce the complexity and perform efficient in multi-variate datastreams.;Finally we propose a method that uncovers hidden structures and relations between multiple IoT data streams. Our novel solution uses Latent Dirichlet Allocation (LDA), a topic extraction method that is generally used in text analysis. We apply LDA on meaningful labels that describe the numerical data in human understandable terms. To create the labels we use Symbolic Aggregate approXimation (SAX), a method that converts raw data into string-based patterns. The extracted patterns are then transformed with a rule engine into the labels.;The work investigates how heterogeneous sensory data from multiple sources can be processed and analysed to create near real-time intelligence and how our proposed method provides an efficient way to interpret patterns in the data streams. The proposed method provides a novel way to uncover the correlations and associations between different pattern in IoT data streams. The evaluation results show that the proposed solution is able to identify the correlation with high efficiency with an F-measure up to 90%.;Overall, this PhD research has designed, implemented and evaluated unsupervised adaptive algorithms to analyse, structure and extract information from dynamic and multi-variate sensory data streams. The results of this research has significant impact in designing flexible and scalable solutions in analysing real-world sensory data streams and specially in cases where labelled and annotated data is not available or it is too costly to be collected. Research and advancements in healthcare and smarter cities are two key areas that can directly from this research.