Related papers: Using Relative Entropy to Find Optimal Approximati…
We show that a simple geometric result suffices to derive the form of the optimal solution in a large class of finite and infinite-dimensional maximum entropy problems concerning probability distributions, spectral densities and covariance…
Works, briefly surveyed here, are concerned with two basic methods: Maximum Probability and Bayesian Maximum Probability; as well as with their asymptotic instances: Relative Entropy Maximization and Maximum Non-parametric Likelihood.…
We examine the minimization of information entropy for measures on the phase space of bounded domains, subject to constraints that are averages of grand canonical distributions. We describe the set of all such constraints and show that it…
Given two discrete random variables $X$ and $Y$, with probability distributions ${\bf p} =(p_1, \ldots , p_n)$ and ${\bf q}=(q_1, \ldots , q_m)$, respectively, denote by ${\cal C}({\bf p}, {\bf q})$ the set of all couplings of ${\bf p}$ and…
Importance sampling of target probability distributions belonging to a given convex class is considered. Motivated by previous results, the cost of importance sampling is quantified using the relative entropy of the target with respect to…
The problem addressed concerns the determination of the average number of successive attempts of guessing a word of a certain length consisting of letters with given probabilities of occurrence. Both first- and second-order approximations…
Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally…
Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical…
The Matrix-based Renyi's entropy enables us to directly measure information quantities from given data without the costly probability density estimation of underlying distributions, thus has been widely adopted in numerous statistical…
The classical Density Functional Theory (DFT) is introduced as an application of entropic inference for inhomogeneous fluids at thermal equilibrium. It is shown that entropic inference reproduces the variational principle of DFT when…
Semi-continuous data comes from a distribution that is a mixture of the point mass at zero and a continuous distribution with support on the positive real line. A clear example is the daily rainfall data. In this paper, we present a novel…
We study the relative entropy between the empirical estimate of a discrete distribution and the true underlying distribution. If the minimum value of the probability mass function exceeds an $\alpha > 0$ (i.e. when the true underlying…
Several approximations are made to study the microcanonical formalism that are valid in the thermodynamics limit. Usually it is assumed that: 1)Stirling approximation can be used to evaluate the number of microstates; 2) the surface entropy…
Maximum-entropy moment methods allow for the modelling of gases from the continuum regime to strongly rarefied conditions. The development of approximated solutions to the entropy maximization problem has made these methods computationally…
In a recent paper, the authors proposed a general methodology for probabilistic learning on manifolds. The method was used to generate numerical samples that are statistically consistent with an existing dataset construed as a realization…
The general framework of entropic dynamics is used to formulate a relational quantum dynamics. The main new idea is to use tools of information geometry to develop an entropic measure of the mismatch between successive configurations of a…
We present a novel artificial diffusion method to circumvent the instabilities associated with the standard finite element approximation of convection-diffusion equations. Motivated by the micromorphic approach, we introduce an auxiliary…
This paper proposes a new method of bandwidth selection in kernel estimation of density and distribution functions motivated by the connection between maximisation of the entropy of probability integral transforms and maximum likelihood in…
The focus of the present work is the application of the random phase approximation (RPA), derived for inhomogeneous fluids [Frydel and Ma, Phys. Rev. E 93, 062112 (2016)], to penetrable-spheres. As penetrable-spheres transform into…
Motivated by recent work on approximation of diffusion equations by deterministic interacting particle systems, we develop a nonlocal approximation for a range of linear and nonlinear diffusion equations and prove convergence of the method…