English

MLE's bias pathology, Model Updated Maximum Likelihood Estimates and Wallace's Minimum Message Length method

Statistics Theory 2014-09-11 v5 Statistics Theory

Abstract

The inherent bias pathology of the maximum likelihood (ML) estimation method is confirmed for models with unknown parameters θ\theta and ψ\psi when MLE ψ^\hat \psi is function of MLE θ^.\hat \theta. To reduce ψ^\hat \psi's bias the likelihood equation to be solved for ψ\psi is updated using the model for the data YY in it. Model updated (MU) MLE, ψ^MU,\hat \psi_{MU}, often reduces either totally or partially ψ^\hat \psi's bias when estimating shape parameter ψ.\psi. For the Pareto model ψ^MU\hat \psi_{MU} reduces also ψ^\hat \psi's variance. The results explain the difference that puzzled R. A. Fisher, between biased ψ^\hat \psi and the unbiased estimate he obtained for two models with the "2-stage procedure". MUMLE's implementation is equivalent to the abandoned 2-stage procedure thus justifying its use. MUMLE and Firth's bias correcting likelihood are also obtained with the Minimum Message Length method thus motivating its use in frequentist inference and, more generally, model updating with a prior distribution.

Keywords

Cite

@article{arxiv.1312.7713,
  title  = {MLE's bias pathology, Model Updated Maximum Likelihood Estimates and Wallace's Minimum Message Length method},
  author = {Yannis G. Yatracos},
  journal= {arXiv preprint arXiv:1312.7713},
  year   = {2014}
}
R2 v1 2026-06-22T02:36:51.959Z