Distribution of Shannon statistic from normal sample

摘要

Several articles treated asymptotic normality of MLE for entropy measures and applied it in several inferential problems. To study the effect of sample size on such an approximation, this paper provides the exact distribution of a standardized version of MLE of Shannon entropy of normal distribution and compares its exact quantiles with those of the quantiles of the asymptotic distribution of this statistic. Moreover it suggests a modified version of that statistic and shows that this modified one is much closer to normality than the statistic which is given in the literature even for small samples. It also provides formulas for quantiles of the exact distribution of both statistics through the quantiles of chi-square distribution. Moreover, it derives some properties of the MLE of Shannon entropy of normal distribution.