Related papers: Steepest entropy ascent paths towards the Max-Ent …
We establish higher-order nonasymptotic expansions for a difference between probability distributions of sums of i.i.d. random vectors in a Euclidean space. The derived bounds are uniform over two classes of sets: the set of all Euclidean…
We develop a new randomized iterative algorithm---stochastic dual ascent (SDA)---for finding the projection of a given vector onto the solution space of a linear system. The method is dual in nature: with the dual being a non-strongly…
The need to estimate smooth probability distributions (a.k.a. probability densities) from finite sampled data is ubiquitous in science. Many approaches to this problem have been described, but none is yet regarded as providing a definitive…
The one dimensional symmetric simple exclusion process (SSEP) is one of the very few exactly soluble models of non-equilibrium statistical physics. It describes a system of particles which diffuse with hard core repulsion on a one…
This paper investigates a function of macroscopic variables known as the singular potential, building on previous work by Ball and Majumdar. The singular potential is a function of the admissible statistical averages of probability…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
The notion of entropy-regularized optimal transport, also known as Sinkhorn divergence, has recently gained popularity in machine learning and statistics, as it makes feasible the use of smoothed optimal transportation distances for data…
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
As written by statistician George Box "All models are wrong, but some are useful", standard diffusion derivation or Feynman path ensembles use nonphysical infinite velocity/kinetic energy nowhere differentiable trajectories - what seems…
The maximum entropy production (MEP) principle is a hypothetical law of physics which dictates that complex systems, far from equilibrium, evolve into an ordered dissipative structure (DS) which generates as much entropy per second as…
Modern deep neural networks achieved remarkable progress in medical image segmentation tasks. However, it has recently been observed that they tend to produce overconfident estimates, even in situations of high uncertainty, leading to…
We use the delta method and Stein's method to derive, under regularity conditions, explicit upper bounds for the distributional distance between the distribution of the maximum likelihood estimator (MLE) of a $d$-dimensional parameter and…
The Expectation Maximization (EM) algorithm is of key importance for inference in latent variable models including mixture of regressors and experts, missing observations. This paper introduces a novel EM algorithm, called…
We presented background information about various entropies in the literature. The pathway idea of Mathai (2005) is shown to be inferable from the maximization of a certain generalized entropy measure and established connections to…
We consider the problem of estimating a probability distribution that maximizes the entropy while satisfying a finite number of moment constraints, possibly corrupted by noise. Based on duality of convex programming, we present a novel…
In this article we investigate consistency and asymptotic normality of the maximum likelihood and the posterior distribution of the parameters in the context of state space stochastic differential equations (SDEs). We then extend our…
The totally asymmetric simple exclusion process (TASEP) is a stochastic model for the unidirectional dynamics of interacting particles on a $1$D-lattice that is much used in systems biology and statistical physics. Its master equation…
We review here {\it Maximum Caliber} (Max Cal), a general variational principle for inferring distributions of paths in dynamical processes and networks. Max Cal is to dynamical trajectories what the principle of {\it Maximum Entropy} (Max…
We propose a novel neural algorithm for the fundamental problem of computing the entropic optimal transport (EOT) plan between continuous probability distributions which are accessible by samples. Our algorithm is based on the saddle point…
The maximum-entropy remote sampling problem (MERSP) is to select a subset of s random variables from a set of n random variables, so as to maximize the information concerning a set of target random variables that are not directly…