Related papers: Computing the Unique Information
Robust optimization is a popular paradigm for modeling and solving two- and multi-stage decision-making problems affected by uncertainty. In many real-world applications, the time of information discovery is decision-dependent and the…
We consider a model of a data broker selling information to a single agent to maximize his revenue. The agent has a private valuation of the additional information, and upon receiving the signal from the data broker, the agent can conduct…
We propose a method to construct a joint statistical model for mixed-domain data to analyze their dependence. Multivariate Gaussian and log-linear models are particular examples of the proposed model. It is shown that the functional…
Information-maximization clustering learns a probabilistic classifier in an unsupervised manner so that mutual information between feature vectors and cluster assignments is maximized. A notable advantage of this approach is that it only…
We analyze a distributed information network in which each node has access to the information contained in a limited set of nodes (its neighborhood) at a given time. A collective computation is carried out in which each node calculates a…
A simple method is shown to provide optimal variational bounds on $f$-divergences with possible constraints on relative information extremums. Known results are refined or proved to be optimal as particular cases.
In this paper, we present a distributed algorithm for solving convex, constraint-coupled, optimization problems over peer-to-peer networks. We consider a network of processors that aim to cooperatively minimize the sum of local cost…
Information flow is the branch of security that studies the leakage of information due to correlation between secrets and observables. Since in general such correlation cannot be avoided completely, it is important to quantify the leakage.…
A recursive approach for shrinking coefficients of an atomic decomposition is proposed. The corresponding algorithm evolves so as to provide at each iteration a) the orthogonal projection of a signal onto a reduced subspace and b) the index…
We consider a linear inverse problem whose solution is expressed as a sum of two components: one smooth and the other sparse. This problem is addressed by minimizing an objective function with a least squares data-fidelity term and a…
We consider the mixed regression problem with two components, under adversarial and stochastic noise. We give a convex optimization formulation that provably recovers the true solution, and provide upper bounds on the recovery errors for…
Randomness in scientific estimation is generally assumed to arise from unmeasured or uncontrolled factors. However, when combining subjective probability estimates, heterogeneity stemming from people's cognitive or information diversity is…
The dynamic environment in the real world calls for the adaptive techniques for information filtering, namely to provide real-time responses to the changes of system data. Where many incremental algorithms are designed for this purpose,…
In this paper a numerical method is presented, which finds a lower bound for the mutual information between a binary and an arbitrary finite random variable with joint distributions that have a variational distance not greater than a known…
This paper investigates two issues on identification of switched linear systems: persistence of excitation and numerical algorithms. The main contribution is a much weaker condition on the regressor to be persistently exciting that…
The amount of information exchanged per unit of time between two nodes in a dynamical network or between two data sets is a powerful concept for analysing complex systems. This quantity, known as the mutual information rate (MIR), is…
We investigate the information distribution among different entities in the weak measurements protocol. Focusing on multilevel, decaying systems under continuous (no-click) monitoring, we derive exact, conservation-type information…
We build information geometry for a partially ordered set of variables and define the orthogonal decomposition of information theoretic quantities. The natural connection between information geometry and order theory leads to efficient…
In this paper we establish lower bounds on information divergence from a distribution to certain important classes of distributions as Gaussian, exponential, Gamma, Poisson, geometric, and binomial. These lower bounds are tight and for…
Estimating Mutual Information (MI), a key measure of dependence of random quantities without specific modelling assumptions, is a challenging problem in high dimensions. We propose a novel mutual information estimator based on parametrizing…