Related papers: Maximum Probability and Relative Entropy Maximizat…
Likelihood-free methods, such as approximate Bayesian computation, are powerful tools for practical inference problems with intractable likelihood functions. Markov chain Monte Carlo and sequential Monte Carlo variants of approximate…
The asymptotically optimal hypothesis testing problem with the general sources as the null and alternative hypotheses is studied under exponential-type error constraints on the first kind of error probability. Our fundamental philosophy in…
Various methods for solving the inverse reinforcement learning (IRL) problem have been developed independently in machine learning and economics. In particular, the method of Maximum Causal Entropy IRL is based on the perspective of entropy…
Model inadequacy and measurement uncertainty are two of the most confounding aspects of inference and prediction in quantitative sciences. The process of scientific inference (the inverse problem) and prediction (the forward problem)…
In this paper, we study the linear transformation model in the most general setup. This model includes many important and popular models in statistics and econometrics as special cases. Although it has been studied for many years, the…
Parameter estimation in Markov random fields (MRFs) is a difficult task, in which inference over the network is run in the inner loop of a gradient descent procedure. Replacing exact inference with approximate methods such as loopy belief…
We apply recent ideas about complexity and randomness to the philosophy of laws and chances. We develop two ways to use algorithmic randomness to characterize probabilistic laws of nature. The first, a generative chance* law, employs a…
In statistics, there are a variety of methods for performing model selection that all stem from slightly different paradigms of statistical inference. The reasons for choosing one particular method over another seem to be based entirely on…
There is an increasing amount of literature focused on Bayesian computational methods to address problems with intractable likelihood. One approach is a set of algorithms known as Approximate Bayesian Computational (ABC) methods. One of the…
We present a theoretical framework of probabilistic learning derived by Maximum Probability (MP) Theorem shown in the current paper. In this probabilistic framework, a model is defined as an event in the probability space, and a model or…
The article concerns hybrid combinations of empirical and parametric likelihood functions. Combining the two allows classical parametric likelihood to be crucially modified via the nonparametric counterpart, making possible model…
We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…
Covariate balance is a conventional key diagnostic for methods used estimating causal effects from observational studies. Recently, there is an emerging interest in directly incorporating covariate balance in the estimation. We study a…
Given a sample of independent and identically distributed random variables, a novel nonparametric maximum entropy method is presented to estimate the underlying continuous univariate probability density function (pdf). Estimates are found…
Empirical analysis serves as an important complement to theoretical analysis for studying practical Bayesian optimization. Often empirical insights expose strengths and weaknesses inaccessible to theoretical analysis. We define two metrics…
In Bayesian theory, calculating a posterior probability distribution is highly important but usually difficult. Therefore, some methods have been put forward to deal with such problem, among which, the most popular one is the asymptotic…
We develop a method for multidimensional optimisation using flow equations. This method is based on homotopy continuation in combination with a maximum entropy approach. Extrema of the optimising functional correspond to fixed points of the…
We investigate convergence of the expectation maximization algorithm by representing it as a generalized proximal method. Convergence of iterates and not just in value is investigated under natural hypotheses such as definability of the…
Maximum likelihood estimation is a valuable tool often applied to inverse problems in quantum theory. Estimation from small data sets can, however, have non unique solutions. We discuss this problem and propose to use Jaynes maximum entropy…
We study the convergence rates of empirical Bayes posterior distributions for nonparametric and high-dimensional inference. We show that as long as the hyperparameter set is discrete, the empirical Bayes posterior distribution induced by…