Related papers: The Epic Story of Maximum Likelihood
Philosophers now seem to agree that frequentism is an untenable strategy to explain the meaning of probabilities. Nevertheless, I want to revive frequentism, and I will do so by grounding probabilities on typicality in the same way as the…
We address the fundamental problem of selection under uncertainty by modeling it from the perspective of Bayesian persuasion. In our model, a decision maker with imperfect information always selects the option with the highest expected…
This note discusses the problem of choosing between hypotheses in a situation with many, correlated non-normal variables. A new method is introduced to shrink the many variables into a smaller subset of variables with zero mean, unit…
Fisher information is a measure of the best precision with which a parameter can be estimated from statistical data. It can also be defined for a continuous random variable without reference to any parameters, in which case it has a…
Maximum likelihood (ML) estimation is widely used in statistics. The h-likelihood has been proposed as an extension of Fisher's likelihood to statistical models including unobserved latent variables of recent interest. Its advantage is that…
The three-parameter generalized extreme value distribution arises from classical univariate extreme value theory and is in common use for analyzing the far tail of observed phenomena. Curiously, important asymptotic properties of…
This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The discussion emphasizes the pragmatic elements in the derivation. An epistemic notion of information is defined in…
The fiducial argument of Fisher (1973) has been described as his biggest blunder, but the recent review of Hannig et al. (2016) demonstrates the current and increasing interest in this brilliant idea. This short note analyses an example…
Maximum entropy (MAXENT) method has a large number of applications in theoretical and applied machine learning, since it provides a convenient non-parametric tool for estimating unknown probabilities. The method is a major contribution of…
In his classical article[3](1953), J.Kiefer introduced the Fibonacci search as a direct optimal method. The optimality was proved under the restriction: the total number of tests is given in advance and fixed. To avoid this restriction,…
For latent class models where the class weights depend on individual covariates, we derive a simple expression for computing the score vector and a convenient hybrid between the observed and the expected information matrices which is always…
Human decision-making deviates from the optimal solution, that maximizes cumulative rewards, in many situations. Here we approach this discrepancy from the perspective of bounded rationality and our goal is to provide a justification for…
Kagan and Shepp (2005, Amer. Statist.) presented an elegant example of a mixture model for which an insufficient statistic preserves Fisher information. This note uses the regularity property of differentiability in quadratic mean to…
Randomness in scientific estimation is generally assumed to arise from unmeasured or uncontrolled factors. However, when combining subjective probability estimates, heterogeneity stemming from people's cognitive or information diversity is…
By using ideas on complexity and randomness originally suggested by the mathematician-philosopher Gottfried Leibniz in 1686, the modern theory of algorithmic information is able to show that there can never be a "theory of everything" for…
This paper develops a unified estimation framework, the Maximum Ideal Likelihood Estimation (MILE), for general parametric models with latent variables. Unlike traditional approaches relying on the marginal likelihood of the observed data,…
After experimenting with a number of non-probabilistic methods for dealing with uncertainty many researchers reaffirm a preference for probability methods [1] [2], although this remains controversial. The importance of being able to form…
Random utility theory models an agent's preferences on alternatives by drawing a real-valued score on each alternative (typically independently) from a parameterized distribution, and then ranking the alternatives according to scores. A…
Maximum likelihood estimates and corresponding confidence regions of the estimates are commonly used in statistical inference. In practice, people often construct approximate confidence regions with the Fisher information at given sample…
The theory of equidistribution is about hundred years old, and has been developed primarily by number theorists and theoretical computer scientists. A motivated uninitiated peer could encounter difficulties perusing the literature, due to…