Natural Language Text Classification and Filtering with Trigrams and Evolutionary Nearest Neighbour Classifiers. Software Engineering (SEN).

摘要

N grams offer fast language independent multi-class text categorization. Text is reduced in a single pass to ngram vectors. These are assigned to one of several classes by (1) nearest neighbour (KNN) and (2) genetic algorithm operating on weights in a nearest neighbour classifier. 91% accuracy is found on binary classification on short multi-author technical English documents. This falls if more categories are used but 69% is obtained with 8 classes. Zipf law is found not to apply to trigrams.