Related papers: Information-Theoretic Extensions of the Shannon-Ny…

It has been generally recognized that stochasticity can play an important role in the information processing accomplished by reaction networks in biological cells. Most treatments of that stochasticity employ Gaussian noise even though it…

We present an experimental approach to construct a dephrasure channel, which contains both dephasing and erasure noises, and can be used as an efficient tool to study the superadditivity of coherent information. By using a three-fold…

We investigate how to exploit intermittent feedback for interference management by studying the two-user Gaussian interference channel (IC). We approximately characterize (within a universal constant) the capacity region for the Gaussian IC…

In this paper, we consider some long-standing problems in communication systems with access to noisy feedback. We introduce a new notion, the residual directed information, to capture the effective information flow (i.e. mutual information…

We formulate an info-clustering paradigm based on a multivariate information measure, called multivariate mutual information, that naturally extends Shannon's mutual information between two random variables to the multivariate case…

Non-orthogonal communications play an important role in future communication architectures. In such scenarios, the received signal is corrupted by an interfering signal which is typically cyclostationary in continuous-time. If the period of…

Gaussian smoothing combined with a probabilistic framework for denoising via the empirical Bayes formalism, i.e., the Tweedie-Miyasawa formula (TMF), are the two key ingredients in the success of score-based generative models in Euclidean…

In the field of signal processing, the sampling theorem plays a fundamental role for signal reconstruction as it bridges the gap between analog and digital signals. Following the celebrated Nyquist-Shannon sampling theorem, generalizing the…

We address the problem of universal communications over an unknown channel with an instantaneous noiseless feedback, and show how rates corresponding to the empirical behavior of the channel can be attained, although no rate can be…

It is well known that, in general, feedback may enlarge the capacity region of Gaussian broadcast channels. This has been demonstrated even when the feedback is noisy (or partial-but-perfect) and only from one of the receivers. The only…

We present a new theoretical perspective of data noising in recurrent neural network language models (Xie et al., 2017). We show that each variant of data noising is an instance of Bayesian recurrent neural networks with a particular…

In many scenarios, the communication system suffers from both Gaussian white noise and non-Gaussian impulsive noise. In order to design optimal signal detection method, it is necessary to estimate the parameters of mixed Gaussian-impulsive…

For the discrete-time additive white generalized Gaussian noise channel with a generalized input power constraint, with the respective shape and power parameters >= 1, we derive an upper bound on the optimal block error exponent. Explicit…

We present the Causal Gaussian Process Convolution Model (CGPCM), a doubly nonparametric model for causal, spectrally complex dynamical phenomena. The CGPCM is a generative model in which white noise is passed through a causal,…

For vector Gaussian channels, a precise differential connection between channel capacity and a quantity termed normalized optimal detection error (NODE) is presented. Then, this C-NODE relationship is extended to continuous-time Gaussian…

We prove an exact relationship between the optimal denoising function and the data distribution in the case of additive Gaussian noise, showing that denoising implicitly models the structure of data allowing it to be exploited in the…

An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…

We propose a novel class of Gaussian processes (GPs) whose spectra have compact support, meaning that their sample trajectories are almost-surely band limited. As a complement to the growing literature on spectral design of covariance…

We use white Gaussian noise as a test signal for single-mode and multimode transmission links and estimate the link capacity based on a calculation of mutual information. We also extract the complex amplitude channel estimations and…

Estimation of parameters of a diffusion based on discrete time observations poses a difficult problem due to the lack of a closed form expression for the likelihood. From a Bayesian computational perspective it can be casted as a missing…