Related papers: Maximum Relative Divergence Principle for Grading …
Ill-posed inverse problems of the form y = X p where y is J-dimensional vector of a data, p is m-dimensional probability vector which cannot be measured directly and matrix X of observable variables is a known J,m matrix, J < m, are…
The maximum correntropy criterion (MCC) has recently been successfully applied in robust regression, classification and adaptive filtering, where the correntropy is maximized instead of minimizing the well-known mean square error (MSE) to…
The irregularities of a distribution of $N$ points in the unit interval are often measured with various notions of discrepancy. The discrepancy function can be defined with respect to intervals of the form $[0,t)\subset [0,1)$ or arbitrary…
A formulation of the density functional theory is constructed on the foundations of entropic inference. The theory is introduced as an application of maximum entropy for inhomogeneous fluids in thermal equilibrium. It is shown that entropic…
The maximum entropy principle, as applied to quantum systems, is a fundamental prescript positing that for a quantum system for which we only have partial knowledge, the maximum entropy state consistent with the partial knowledge is a…
Distribution functions of many static transport equations are found using the Maximum Entropy Principle. The equations of constraint which contain the relevant dynamical information are simply the low-lying moments of the distributions.…
We introduce a comprehensive theoretical and algorithmic framework that bridges formal group theory and group entropies with modern machine learning, paving the way for an infinite, flexible family of Mirror Descent (MD) optimization…
We introduce an independence criterion based on entropy regularized optimal transport. Our criterion can be used to test for independence between two samples. We establish non-asymptotic bounds for our test statistic and study its…
The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at…
Maximum Entropy (MaxEnt) reinforcement learning is a powerful learning paradigm which seeks to maximize return under entropy regularization. However, action entropy does not necessarily coincide with state entropy, e.g., when multiple…
In this paper, we present a novel and general framework called {\it Maximum Entropy Discrimination Markov Networks} (MaxEnDNet), which integrates the max-margin structured learning and Bayesian-style estimation and combines and extends…
Partial orders have been used to model several experimental setups, going from classical thermodynamics and general relativity to the quantum realm with its resource theories. In order to study such experimental setups, one typically…
Relative entropy is the standard measure of distinguishability in classical and quantum information theory. In the classical case, its loss under channels admits an exact chain rule, while in the quantum case only asymptotic, regularized…
It is possible to derive the maximum entropy principle from thermodynamic stability requirements. Using as a starting point the equilibrium probability distribution, currently used in non-extensive thermostatistics, it turns out that the…
We define an entropy based on a chosen governing probability distribution. If a certain kind of measurements follow such a distribution it also gives us a suitable scale to study it with. This scale will appear as a link function that is…
In this paper we address the problem of systems under an external feedback. This is performed using a large deviation approach and rate distortion from information theory. In particular we define a lower boundary for the maximum entropy…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
The concept of the value-gradient is introduced and developed in the context of reinforcement learning. It is shown that by learning the value-gradients exploration or stochastic behaviour is no longer needed to find locally optimal…
We consider an isomorphism invariant for measure-preserving systems - types of generalized entropy convergence rates. We show the connections of this invariant with the types of Shannon entropy convergence rates. In the case when they…
Entropy production is the crucial quantity characterizing irreversible phenomena and the second law of thermodynamics. Yet, a ubiquitous definition eludes consensus. Given that entropy production arises from incomplete access to…