Data Preprocessing and Outlier Detection in Multivariate Data

摘要

The n-dimensional input structure in an artificial neural network for the classification of objects consists of different signal sources, such as signals from temperature sensors, position encoders, microphones, cameras, spectrometers or data from other measurement instruments. The raw data, delivered from these measurement data sources do not have the same structure; rather, they have a different number of dimensions and they can be presented in a parameterized form as linked values, as data packets or as discrete values at the input. The data distribution of these values will be analyzed in the data pre-preparation procedures; the statistical values were identified, non-applicable values will be eliminated and the characteristic values will be saved in a model for the training data set (TDS) for the underlying object. In order to match these values to the input structure of the artificial network, different methods were used. Sensor signals were quantized, filtered and standardized. Signal trends presented in a parameterized manner can be transformed via wavelet transform into spectral components, filtered and standardized. To create the input vectors, single measuring values, packets of spectral measuring data and other data sources of information can be combined in order to get reliable characteristic properties for the trained object. To obtain a pre-defined exactness for the mapping to a low dimensional space, the training vectors should have a limited maximum distance to their nearest neighbors (NN). This and a continuous manifold without gaps will be assured by the data pre-preparation procedure so that the data can be processed as an entirety in the following work steps.