Related papers: Approximate maximum entropy principles via Goemans…
Kappa distributions are widely used in space plasma physics to model velocity distribution functions with heavy tails. Parameter estimation in these distributions is, however, complicated by the fact that the kappa distribution does not…
The foundations of the Boltzmann-Gibbs (BG) distributions for describing equilibrium statistical mechanics of systems are examined. Broadly, they fall into: (i) probabilistic paaroaches based on the principle of equal a priori probability…
Herein we define a measure of similarity between classification distributions that is both principled from the perspective of statistical pattern recognition and useful from the perspective of machine learning practitioners. In particular,…
Consider two correlated sources $X$ and $Y$ generated from a joint distribution $p_{X,Y}$. Their G\'acs-K\"orner Common Information, a measure of common information that exploits the combinatorial structure of the distribution $p_{X,Y}$,…
The operational characterization of quantum coherence is the corner stone in the development of resource theory of coherence. We introduce a new coherence quantifier based on max-relative entropy. We prove that max-relative entropy of…
In this work, we introduce a notion of reachability entropy to characterize the smallest data rate which is sufficient enough to enforce reach-while-stay specification. We also define data rates of coder-controllers that can enforce this…
The estimation of categorical distributions under marginal constraints summarizing some sample from a population in the most-generalizable way is key for many machine-learning and data-driven approaches. We provide a parameter-agnostic…
Dissipative processes cause collisionless plasmas in many systems to develop nonthermal particle distributions with broad power-law tails. The prevalence of power-law energy distributions in space/astrophysical observations and kinetic…
This paper presents a general asymptotic theory of sequential Bayesian estimation giving results for the strongest, almost sure convergence. We show that under certain smoothness conditions on the probability model, the greedy information…
We construct the generalized entropy optimized by a given arbitrary statistical distribution with a finite linear expectation value of a random quantity of interest. This offers, via the maximum entropy principle, a unified basis for a…
A novel formalism, called H-theory, is applied to the problem of statistical equilibrium of a hierarchical complex system with multiple time and length scales. In this approach, the system is formally treated as being composed of a small…
The maximum entropy principle is a powerful tool for solving underdetermined inverse problems. This paper considers the problem of discretizing a continuous distribution, which arises in various applied fields. We obtain the approximating…
In this paper we develop an Expectation Maximization(EM) algorithm to estimate the parameter of a Yule-Simon distribution. The Yule-Simon distribution exhibits the "rich get richer" effect whereby an 80-20 type of rule tends to dominate.…
In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of…
The purpose of this note is to show how the method of maximum entropy in the mean (MEM) may be used to improve parametric estimation when the measurements are corrupted by large level of noise. The method is developed in the context on a…
The entropic region is formed by the collection of the Shannon entropies of all subvectors of finitely many jointly distributed discrete random variables. For four or more variables, the structure of the entropic region is mostly unknown.…
Statistical inference on the mean of a Poisson distribution is a fundamentally important problem with modern applications in, e.g., particle physics. The discreteness of the Poisson distribution makes this problem surprisingly challenging,…
The maximum entropy approach operating with quite general entropy measure and constraint is considered. It is demonstrated that for a conditional or parametrized probability distribution $f(x|\mu)$ there is a "universal" relation among the…
We derive entropy factorization estimates for spin systems using the stochastic localization approach proposed by Eldan and Chen-Eldan, which, in this context, is equivalent to the renormalization group approach developed independently by…
The maximum entropy principle (MEP) is one of the most prominent methods to investigate and model complex systems. Despite its popularity, the standard form of the MEP can only generate Boltzmann-Gibbs distributions, which are ill-suited…