A probabilistic prior knowledge integration method: Application to generative and discriminative models

摘要

Prior knowledge integration aims at taking advantage of cheap plentiful data to improve the efficiency of supervised learning procedures. For parametric models, however, it is a challenging task. In this contribution, we introduce a novel simple methodology to incorporate prior knowledge into a supervised objective function. We propose to introduce the prior knowledge in the form of joint probability of observations and labels. We discuss the nature of features in discriminative and generative models and hence, differences in priors integration. We illustrate the efficiency of the proposed method both by synthetic data sets and by our results on a realistic large-scale sequence labeling task.