Related papers: Entropic Inequalities and Marginal Problems
Many partially-successful attempts have been made to find the most natural discrete-variable version of Shannon's entropy power inequality (EPI). We develop an axiomatic framework from which we deduce the natural form of a discrete-variable…
Entropy is useful in statistical problems as a measure of irreversibility, randomness, mixing, dispersion, and number of microstates. However, there remains ambiguity over the precise mathematical formulation of entropy, generalized beyond…
A common theme underlying many problems in statistics and economics involves the determination of a systematic method of selecting a joint distribution consistent with a specified list of categorical marginals, some of which have an ordinal…
Local and category-theoretical entropies associated with an endomorphism of finite length (i.e., with zero-dimensional closed fiber) of a commutative Noetherian local ring are compared. Local entropy is shown to be less than or equal to…
This work deals with the scattering entropy of quantum graphs in many different circumstances. We first consider the case of the Shannon entropy and then the R\'enyi and Tsallis entropies, which are more adequate to study distinct…
Any unconstrained information inequality in three or fewer random variables can be written as a linear combination of instances of Shannon's inequality I(A;B|C) >= 0 . Such inequalities are sometimes referred to as "Shannon" inequalities.…
We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor $-/\mathsf{LProb}_{\rho}$ (where…
Using statistical thermodynamics, we derive a general expression of the stationary probability distribution for thermodynamic systems driven out of equilibrium by several thermodynamic forces. The local equilibrium is defined by imposing…
A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…
The problem of sampling according to the probability distribution minimizing a given free energy, using interacting particles unadjusted kinetic Langevin Monte Carlo, is addressed. In this setting, three sources of error arise, related to…
We study the random binary symmetric perceptron problem, focusing on the behavior of rare high-margin solutions. While most solutions are isolated, we demonstrate that these rare solutions are part of clusters of extensive entropy,…
The Entropy method provides a powerful framework for proving scalar concentration inequalities by establishing functional inequalities like Poincare and log-Sobolev inequalities. These inequalities are especially useful for deriving…
The Sinkhorn algorithm is the most popular method for solving the entropy minimization problem called the Schr\"odinger problem: in the non-degenerate cases, the latter admits a unique solution towards which the algorithm converges…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
By solving a differential-functional equation inposed by the MaxEnt principle we obtain a class of two-parameter deformed logarithms and construct the corresponding two-parameter generalized trace-form entropies. Generalized distributions…
A common task in physics, information theory, and other fields is the analysis of properties of subsystems of a given system. Given the covariance matrix $M$ of a system of $n$ coupled variables, the covariance matrices of the subsystems…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
We derive a novel chain rule for a family of channel conditional entropies, covering von Neumann and sandwiched R\'{e}nyi entropies. In the process, we show that these channel conditional entropies are equal to their regularized version,…
Sampling algorithms, hypergraph degree sequences, and polytopes play a crucial role in statistical analysis of network data. This article offers a brief overview of open problems in this area of discrete mathematics from the point of view…
We consider a singular stochastic control problem, which is called the Monotone Follower Stochastic Control Problem and give sufficient conditions for the existence and uniqueness of a local-time type optimal control. To establish this…