Related papers: Maximum entropy methods as the bridge between macr…
We consider the problem of defining free energy and other thermodynamic functions when the entropy is given as a general function of the probablity distribution, including that for non extensive forms. We find that the free energy, which is…
A marginal problem asks whether a given family of marginal distributions for some set of random variables arises from some joint distribution of these variables. Here we point out that the existence of such a joint distribution imposes…
Density functional theory offers a very accurate way of computing materials properties from first principles. However, it is too expensive for modelling large-scale molecular systems whose properties are, in contrast, computed using…
The maximum entropy technique (MENT) is used to determine the distribution functions of physical values. MENT naturally combines required maximum entropy, the properties of a system and connection conditions in the form of restrictions…
We show that the natural scaling of measurement for a particular problem defines the most likely probability distribution of observations taken from that measurement scale. Our approach extends the method of maximum entropy to use…
In most data-scientific approaches, the principle of Maximum Entropy (MaxEnt) is used to a posteriori justify some parametric model which has been already chosen based on experience, prior knowledge or computational simplicity. In a…
This paper proposes a probabilistic approach to investigate the shape of landscapes of multi-dimensional potential functions. Under a suitable coupling scheme, two copies of the overdamped Langevin dynamics associated with the potential…
I show that the maximum entropy principle can be replaced by a more natural assumption, that there exists a phenomenological function of entropy consistent with the microscopic model. The requirement of existence provides then a unique…
In this paper, we present a maximum likelihood estimation approach to determine the value vector in transformer models. We model the sequence of value vectors, key vectors, and the query vector as a sequence of Gaussian distributions. The…
We show that a simple telescoping sum trick, together with the triangle inequality and a tensorisation property of expected-contractive coefficients of random channels, allow us to achieve general simultaneous decoupling for multiple users…
The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
We investigate the overdamped stochastic dynamics of a particle in an asymptotically flat external potential field, in contact with a thermal bath. For an infinite system size, the particles may escape the force field and diffuse freely at…
Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is…
In a recent paper, the authors proposed a general methodology for probabilistic learning on manifolds. The method was used to generate numerical samples that are statistically consistent with an existing dataset construed as a realization…
We give some results relating asymptotic characterisations of maximum entropy probability measures to characterisations of Bayes optimal classifiers. Our main theorems show that maximum entropy is a universally Bayes optimal decision rule…
Spaces of convex and concave functions appear naturally in theory and applications. For example, convex regression and log-concave density estimation are important topics in nonparametric statistics. In stochastic portfolio theory, concave…
In this work, we consider a recently proposed entropy S (called varentropy) defined by a variational relationship dI=beta*(d<x>-<dx>) as a measure of uncertainty of random variable x. By definition, varentropy underlies a generalized…
We show that the principle of maximum entropy, a variational method appearing in statistical inference, statistical physics, and the analysis of stochastic dynamical systems, admits a geometric description from gauge theory. Using the…
We propose a family of "exactly solvable" probability distributions to approximate partition functions of two-dimensional statistical mechanics models. While these distributions lie strictly outside the mean-field framework, their free…