Related papers: Logarithmic Asymptotic Relations Between $p$-Value…
The statistical analysis of data stemming from dynamical systems, including, but not limited to, time series, routinely relies on the estimation of information theoretical quantities, most notably Shannon entropy. To this purpose, possibly…
Mutual information $I(X;Y)$ is a useful definition in information theory to estimate how much information the random variable $Y$ holds about the random variable $X$. One way to define the mutual information is by comparing the joint…
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
Mutual information is a well-known tool to measure the mutual dependence between variables. In this paper, a Bayesian nonparametric estimation of mutual information is established by means of the Dirichlet process and the $k$-nearest…
We show how Fisher's information already known particular character as the fundamental information geometric object which plays the role of a metric tensor for a statistical differential manifold, can be derived in a relatively easy manner…
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter…
We consider the behaviour of the Fisher information of scaled sums of independent and identically distributed random variables in the Central Limit Theorem regime. We show how this behaviour can be related to the second-largest non-trivial…
The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and…
Conditional Mutual Information (CMI) is a measure of conditional dependence between random variables X and Y, given another random variable Z. It can be used to quantify conditional dependence among variables in many data-driven inference…
Recently, the importance of analysing data and collecting valuable insight efficiently has been increasing in various fields. Estimating mutual information (MI) plays a critical role to investigate the relationship among multiple random…
The article studies the almost surely asymptotics of extreme values $\bar{\xi}_n = \max_{1\leq i \leq n} \xi_i$, where $ \xi , \xi_1 , \xi_2 , \ldots$ are discrete identically distributed random variables. One of the main results on this…
There is a vast body of recent literature on the reliability of communication through noisy channels, the recovery of community structures in the stochastic block model, the limiting behavior of the free entropy in spin glasses and the…
We characterize mutual information as the unique map on ordered pairs of random variables satisfying a set of axioms similar to those of Faddeev's characterization of the Shannon entropy. There is a new axiom in our characterization however…
Testing mutual independence among multiple random variables is a fundamental problem in statistics, with wide applications in genomics, finance, and neuroscience. In this paper, we propose a new class of tests for high-dimensional mutual…
It is of importance to investigate the significance of a subset of covariates $W$ for the response $Y$ given covariates $Z$ in regression modeling. To this end, we propose a significance test for the partial mean independence problem based…
In information theory, Fisher information and Shannon information (entropy) are respectively used to quantify the uncertainty associated with the distribution modeling and the uncertainty in specifying the outcome of given variables. These…
The large-scale multiple testing inherent to high throughput biological data necessitates very high statistical stringency and thus true effects in data are difficult to detect unless they have high effect sizes. One solution to this…
Relative Fisher information (IR), which is a measure of correlative fluctuation between two probability densities, has been pursued for a number of quantum systems, such as, 1D quantum harmonic oscillator (QHO) and a few central potentials…
Most existing works on disentangled representation learning are solely built upon an marginal independence assumption: all factors in disentangled representations should be statistically independent. This assumption is necessary but…
It is very difficult to solve the Maximum Mutual Information (MMI) or Maximum Likelihood (ML) for all possible Shannon Channels or uncertain rules of choosing hypotheses, so that we have to use iterative methods. According to the Semantic…