Related papers: Partial Information Decomposition via Deficiency f…
Consider the problem of Private Information Retrieval (PIR), where a user wishes to retrieve a single message from $N$ non-communicating and non-colluding databases (servers). All servers store the same set of $M$ messages and they respond…
Computable Information Density (CID), the ratio of the length of a losslessly compressed data file to that of the uncompressed file, is a measure of order and correlation in both equilibrium and nonequilibrium systems. Here we show that…
In Private Information Retrieval (PIR), one wants to download a file from a database without revealing to the database which file is being downloaded. Much attention has been paid to the case of the database being encoded across several…
Unsupervised learning plays an important role in many fields, such as artificial intelligence, machine learning, and neuroscience. Compared to static data, methods for extracting low-dimensional structure for dynamic data are lagging. We…
We propose a probability distribution for multivariate binary random variables. The probability distribution is expressed as principal minors of the parameter matrix, which is a matrix analogous to the inverse covariance matrix in the…
We discuss a Bayesian methodology for the solution of the inverse problem underlying the determination of parton distribution functions (PDFs). In our approach, Gaussian Processes (GPs) are used to model the PDF prior, while Bayes theorem…
This paper proposes a joint segmentation and deconvolution Bayesian method for medical ultrasound (US) images. Contrary to piecewise homogeneous images, US images exhibit heavy characteristic speckle patterns correlated with the tissue…
The Gray and Wyner lossy source coding for a simple network for sources that generate a tuple of jointly Gaussian random variables (RVs) $X_1 : \Omega \rightarrow {\mathbb R}^{p_1}$ and $X_2 : \Omega \rightarrow {\mathbb R}^{p_2}$, with…
Bayesian networks have been used as a mechanism to represent the joint distribution of multiple random variables in a flexible yet interpretable manner. One major challenge in learning the structure of a Bayesian network is how to model…
Recently, sparsity has become a key concept in various areas of applied mathematics, computer science, and electrical engineering. One application of this novel methodology is the separation of data, which is composed of two (or more)…
In the practice of information extraction, the input data are usually arranged into pattern matrices, and analyzed by the methods of linear algebra and statistics, such as principal component analysis. In some applications, the tacit…
We propose a method to restore and to segment simultaneously images degraded by a known point spread function (PSF) and additive white noise. For this purpose, we propose a joint Bayesian estimation framework, where a family of…
Mutual information is widely used, in a descriptive way, to measure the stochastic dependence of categorical random variables. In order to address questions such as the reliability of the descriptive value, one must consider…
Multivariate information decompositions hold promise to yield insight into complex systems, and stand out for their ability to identify synergistic phenomena. However, the adoption of these approaches has been hindered by there being…
A random Gaussian density field contains a fixed amount of Fisher information on the amplitude of its power spectrum. For a given smoothing scale, however, that information is not evenly distributed throughout the smoothed field. We…
Information transmission over channels with transceiver distortion is investigated via generalized mutual information (GMI) under Gaussian input distribution and nearest-neighbor decoding. A canonical transceiver structure in which the…
Continuous-variable Gaussian entanglement is an attractive notion, both as a fundamental concept in quantum information theory, based on the well-established Gaussian formalism for phase-space variables, and as a practical resource in…
An important notion of common information between two random variables is due to Wyner. In this paper, we derive a lower bound on Wyner's common information for continuous random variables. The new bound improves on the only other general…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
Partial information decomposition has recently found applications in biological signal processing and machine learning. Despite its impacts, the decomposition was introduced through an informal and heuristic route, and its exact operational…