相关论文: A new approach to mutual information

A representation of heterogeneous stochastic populations that are composed of sub-populations with different levels of distinguishability is introduced together with an analysis of its properties. It is demonstrated that any instance of…

Following our previous work on copula-based nonsymmetric dependence measures, we introduce similar measures for discrete random variables. The measures cover the range between two extremes: independence and complete dependence, which take…

The main objective of this paper is to show that two asymptotically stable steady states which belong to an analytic path of asymptotically stable steady states can be gradually transferred one to the other by successive changes of the…

We prove that mutual information is actually negative copula entropy, based on which a method for mutual information estimation is proposed.

We strengthen and put in a broader perspective previous results of the first two authors on colliding permutations. The key to the present approach is a new non-asymptotic invariant for graphs.

Information-theoretic quantities have received significant attention as system-independent measures of correlations in many-body quantum systems, e.g., as universal order parameters of synchronization. In this work, we present a method to…

Reconstructing the structural connectivity between interacting units from observed activity is a challenge across many different disciplines. The fundamental first step is to establish whether or to what extent the interactions between the…

We take an information theoretic perspective on a classical sparse-sampling noisy linear model and present an analytical expression for the mutual information, which plays central role in a variety of communications/processing problems.…

We review recent progress in understanding the meaning of mutual information in natural language. Let us define words in a text as strings that occur sufficiently often. In a few previous papers, we have shown that a power-law distribution…

Learning disentangled representations of textual data is essential for many natural language tasks such as fair classification, style transfer and sentence generation, among others. The existent dominant approaches in the context of text…

We propose a new approach for estimating the parameters of a probability distribution. It consists on combining two new methods of estimation. The first is based on the definition of a new distance measuring the difference between…

We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that…

We prove the existence of (one-way) communication tasks with a subconstant versus superconstant asymptotic gap, which we call "doubly infinite," between their quantum information and communication complexities. We do so by studying the…

The paper presents a novel asymptotic distribution for a mle when the log--likelihood is strictly concave in the parameter for all data points; for example, the exponential family. The new asymptotic distribution can be seen as a refinement…

We study p-adic counterparts of stable distributions, that is limit distributions for sequences of normalized sums of independent identically distributed p-adic-valued random variables. In contrast to the classical case, non-degenerate…

In this paper, an alternative mixed Poisson distribution is proposed by amalgamating Poisson distribution and a modification of the Quasi Lindley distribution. Some fundamental structural properties of the new distribution, namely the shape…

This paper is part of series on self-contained papers in which a large part, if not the full extent, of the asymptotic limit theory of summands of independent random variables is exposed. Each paper of the series may be taken as review…

In this paper, we investigate the asymptotic error distributions of symplectic methods for stochastic Hamiltonian systems and further provide Hamiltonian-specific analysis that clarifies the superiority of symplectic methods. Our…

This paper develops upper and lower bounds for the probability of Boolean expressions by treating multiple occurrences of variables as independent and assigning them new individual probabilities. Our technique generalizes and extends the…

The problem of how to properly quantify redundant information is an open question that has been the subject of much recent research. Redundant information refers to information about a target variable S that is common to two or more…