Generative Models for Similarity-based Classification

摘要

This work proposes a generative framework for similarity-based classification: similarity discriminant analysis (SDA). The classifiers in the SDA framework are similarity-based, because they classify based on the pairwise similarities of samples, and they are generative, because they build class-conditional probability models of the pairwise similarities. The problem of estimating the class-conditional similarity probability models is solved by applying the maximum entropy principle, under the constraint that the mean similarities be equal to the average similarities observed in a set of training samples. Thus, the class-conditional distributions in the SDA framework are exponential functions of the similarities. Within the SDA framework, several classifiers are analyzed in detail: the SDA classifier, the local SDA classifier, the nnSDA classifier, and the mixture SDA classifier. Their performance is evaluated on simulated and benchmark data sets, and compared to the performance of existing similarity-based classifiers which are not generative.