On an optimization problem arising from probability density estimation

摘要

We consider a class of optimization problems arising from statistical density estimation of high dimensional data from projections on lower dimensional subspaces. Two criteria are used for optimal model selection, namely, maximum entropy and maximum likelihood estimation. In each case, our approach requires univariate density estimators and in this regard we explore the use of mixture models of gaussian densities as well as Parzcn estimators, for the projected data. An expectation maximization strategy is used to update means and covariances as described in Dempster et al. [7]. However, the computation of best directions leads to a challenging class of nonlinear optimization problems which is the focus of our study here. Special eases of this optimization problem are studied analytically and an algorithm to numerically solve the general case is proposed. We provide numerical evidence, on data coming from speech recognition as well as on synthetically generated data, that validates the efficacy of the proposed method.