Related papers: Using Relative Entropy to Find Optimal Approximati…
We propose a systematic expansion method which is applied to freely evolving granular fluids contained in sufficiently small systems. Restricting ourselves to small systems, we show that there exists a small parameter which characterizes a…
The classical problem of moments is addressed by the maximum entropy approach for one-dimensional discrete distributions. The numerical technique of adaptive support approximation is proposed to reconstruct the distributions in the region…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…
Analytical expressions for radial distribution function (RDF) are of critical importance for various applications, such as development of the perturbation theories for equilibrium properties. Theoretically, RDF expressions for…
In many problems in data mining and machine learning, data items that need to be clustered or classified are not points in a high-dimensional space, but are distributions (points on a high dimensional simplex). For distributions, natural…
The method of maximum entropy (ME) is extended to address the following problem: Once one accepts that the ME distribution is to be preferred over all others, the question is to what extent are distributions with lower entropy supposed to…
A formalism is presented for analytically obtaining the probability density function, (P_{n}(s)), for the random distance (s) between two random points in an (n)-dimensional spherical object of radius (R). Our formalism allows (P_{n}(s)) to…
We develop a theory of optimal transport for stationary random measures with a focus on stationary point processes and construct a family of distances on the set of stationary random measures. These induce a natural notion of interpolation…
Consider the following problem: given two arbitrary densities $q_1,q_2$ and a sample-access to an unknown target density $p$, find which of the $q_i$'s is closer to $p$ in total variation. A remarkable result due to Yatracos shows that this…
The principle of maximum entropy provides a useful method for inferring statistical mechanics models from observations in correlated systems, and is widely used in a variety of fields where accurate data are available. While the assumptions…
The aim of this paper is to provide several novel upper bounds on the excess risk with a primal focus on classification problems. We suggest two approaches and the obtained bounds are represented via the distribution dependent local…
The maximum entropy principle advocates to evaluate events' probabilities using a distribution that maximizes entropy among those that satisfy certain expectations' constraints. Such principle can be generalized for arbitrary decision…
The maximum entropy method has been used to reconstruct images in a wide range of astronomical fields, but in its traditional form it is restricted to the reconstruction of strictly positive distributions. We present an extension of the…
We show that the natural scaling of measurement for a particular problem defines the most likely probability distribution of observations taken from that measurement scale. Our approach extends the method of maximum entropy to use…
Entropy and its various generalizations are important in many fields, including mathematical statistics, communication theory, physics and computer science, for characterizing the amount of information associated with a probability…
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased"…
We present a general framework for the approximation of systems of hyperbolic balance laws. The novelty of the analysis lies in the construction of suitable relaxation systems and the derivation of a delicate estimate on the relative…
Model or variable selection is usually achieved through ranking models according to the increasing order of preference. One of methods is applying Kullback-Leibler distance or relative entropy as a selection criterion. Yet that will raise…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
General hyperbolic systems of balance laws with inhomogeneity in space and time in all constitutive functions are studied in the context of relative entropy. A framework is developed in this setting that contributes to a measure-valued weak…