A Machine Learning Algorithm in Automated Text Categorization of Legacy Archives

摘要

The goal of this research is to develop an algorithm to automatically retrieve criticalinformation from raw data files in NASA’s airborne measurement data archive. The product hasto meet specific metrics in term of accuracy, robustness and usability, as the initial decision-treebased development has shown limited applicability due to its resource intensive characteristics.We have developed an innovative solution that is much less resource intensive while offeringcomparable performance. As with many practical applications, the data available are noisy andcorrelated; and there is a wide range of features that are associated with the information to beretrieved. The proposed algorithm uses a decision tree to select features and determine theirweights. A weighted Naive Bayes is used due to the presence of highly correlated inputs. Thedevelopment has been successfully deployed in an industrial scale, and the results show that thedevelopment is well-balanced in term of performance and resource requirements .