Related papers: Information-theoretic convergence of extreme value…
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class…
The precise one-shot characterisation of operational tasks in classical and quantum information theory relies on different forms of smooth entropic quantities. A particularly important connection is between the hypothesis testing relative…
Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses…
It is well known that in Information Theory and Machine Learning the Kullback-Leibler divergence, which extends the concept of Shannon entropy, plays a fundamental role. Given an {\it a priori} probability kernel $\hat{\nu}$ and a…
Max stable laws are limit laws of linearly normalized partial maxima of independent identically distributed random variables. Saeb (2014) proves that the Renyi entropy of order b (b > 1) of linear normalized maximum of iid random variables…
In this paper we perform an analytical and numerical study of Extreme Value distributions in discrete dynamical systems. In this setting, recent works have shown how to get a statistics of extremes in agreement with the classical Extreme…
We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's…
The extreme event statistics plays a very important role in the theory and practice of time series analysis. The reassembly of classical theoretical results is often undermined by non-stationarity and dependence between increments.…
Hilhorst and Schehr recently presented a straight forward computation of limit distributions of sufficiently correlated random numbers \cite{hilhorst}. Here we present the analytical form of entropy which --under the maximum entropy…
We establish that entropy production, which is crucial to the characterization of thermodynamic irreversibility, is obtained through a variational principle involving the Kulback-Leibler divergence. A simple application of this…
We study the statistical distribution of the closest encounter between observations computed along different trajectories of a mixing dynamical system. At the limit of large trajectories, the distribution is of Gumbel type and depends on…
Statistical inference for extreme values of random events is difficult in practice due to low sample sizes and inaccurate models for the studied rare events. If prior knowledge for extreme values is available, Bayesian statistics can be…
In this document, we aim to gather various results related to a compositional/categorical approach to rigorous Statistical Mechanics. Rigorous Statistical Mechanics is centered on the mathematical study of statistical systems. Central…
The asymptotic convergence of probability density function (pdf) and convergence of differential entropy are examined for the non-stationary processes that follow the maximum entropy principle (MaxEnt) and maximum entropy production…
In this paper we discuss about the validity of the Shannon entropy functional in connection with the correct Gibbs-Hertz probability distribution function. We show that there is no contradiction in using the Shannon-Gibbs functional and…
We present a new method of estimating the dispersion of a distribution which is based on the surprising property of a function that measures information processing intensity. It turns out that this function has a maximum at its fixed point.…
Two estimates for the inverse binary entropy function are derived using the property of information entropy to estimate combinatorics of sequences as well as related formulas from population genetics for the effective number of alleles. The…
In this paper we derive an integral (with respect to time) representation of the relative entropy (or Kullback-Leibler Divergence) between measures mu and P on the space of continuous functions from time 0 to T. The underlying measure P is…
The functional characterization of a measure, an essential but delicate aspect of Stein's method, is shown to be accessible for stable probability distributions on convex cones. This notion encompasses the usual stable distributions…
The quantum relative entropy is a measure of the distinguishability of two quantum states, and it is a unifying concept in quantum information theory: many information measures such as entropy, conditional entropy, mutual information, and…