Related papers: On Tsallis Entropy Bias and Generalized Maximum En…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
Value-based reinforcement-learning algorithms have shown strong results in games, robotics, and other real-world applications. Overestimation bias is a known threat to those algorithms and can sometimes lead to dramatic performance…
This paper applies the recently axiomatized Optimum Information Principle (minimize the Kullback-Leibler information subject to all relevant information) to nonparametric density estimation, which provides a theoretical foundation as well…
This paper proposes a new Bayesian approach for analysing moment condition models in the situation where the data may be contaminated by outliers. The approach builds upon the foundations developed by Schennach (2005) who proposed the…
Standard maximum-likelihood estimators for binary-star and exoplanet eccentricities are biased high, in the sense that the estimated eccentricity tends to be larger than the true eccentricity. As with most non-trivial observables, a simple…
Heckman selection model is the most popular econometric model in analysis of data with sample selection. However, selection models with Normal errors cannot accommodate heavy tails in the error distribution. Recently, Marchenko and Genton…
In applications of Bayesian procedures, once a class of priors has been chosen, it may be tempting to fix the prior's hyperparameters from the data, in an empirical Bayes (EB) fashion, usually by their maximum marginal likelihood estimates…
We address the challenge of performing Targeted Maximum Likelihood Estimation (TMLE) after an initial Highly Adaptive Lasso (HAL) fit. Existing approaches that utilize the data-adaptive working model selected by HAL-such as the relaxed HAL…
Mutual Information (MI) is a crucial measure for capturing dependencies between variables, but exact computation is challenging in high dimensions with intractable likelihoods, impacting accuracy and robustness. One idea is to use an…
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the…
In this paper we introduce an easy to compute upper bound on the Tsallis entropy of a density matrix describing a system coupled to a noise source. This suggests that the Tsallis entropy is most natural in the context of quantum information…
During the MaxEnt 2002 workshop in Moscow, Idaho, Tony Vignaux asked again a few simple questions about using Maximum Entropy or Bayesian approaches for the famous Dice problems which have been analyzed many times through this workshop and…
Recently there has been growing interest in the use of Maximum Relative Entropy (MaxREnt) as a tool for statistical inference in ecology. In contrast, here we propose MaxREnt as a tool for applying statistical mechanics to ecology. We use…
To address the common problem of high dimensionality in tensor regressions, we introduce a generalized tensor random projection method that embeds high-dimensional tensor-valued covariates into low-dimensional subspaces with minimal loss of…
This paper outlines a Bayesian approach to estimate finite mixtures of Tobit models. The method consists of an MCMC approach that combines Gibbs sampling with data augmentation and is simple to implement. I show through simulations that the…
We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel…
In a recent paper, the authors proposed a general methodology for probabilistic learning on manifolds. The method was used to generate numerical samples that are statistically consistent with an existing dataset construed as a realization…
Given a probability density $P({\bf x}|{\boldsymbol \lambda})$, where $\bf x$ represents continuous degrees of freedom and $\lambda$ a set of parameters, it is possible to construct a general identity relating expectations of observable…
Prior information often takes the form of parameter constraints. Bayesian methods include such information through prior distributions having constrained support. By using posterior sampling algorithms, one can quantify uncertainty without…
This paper addresses the need for deep learning models to integrate well-defined constraints into their outputs, driven by their application in surrogate models, learning with limited data and partial information, and scenarios requiring…