Related papers: Minimum MS. E. Gerber's Lemma
We analyze the best achievable performance of Bayesian learning under generative models by defining and upper-bounding the minimum excess risk (MER): the gap between the minimum expected loss attainable by learning from data and the minimum…
We consider the estimation of a scalar parameter, when two estimators are available. The first is always consistent. The second is inconsistent in general, but has a smaller asymptotic variance than the first, and may be consistent if an…
For a Markov process the detailed balance condition is equivalent to the time-reversibility of the process. For stochastic differential equations (SDE's) time discretization numerical schemes usually destroy the property of…
We comment on some conceptual and and technical problems related to computational mechanics, point out some errors in several papers, and straighten out some wrong priority claims. We present explicitly the correct algorithm for…
We demonstrate an efficient formula to compute the entropy rate $H(\mu)$ of a hidden Markov process with $q$ output symbols where at least one symbol is unambiguously received. Using an approximation to $H(\mu)$ to the first $N$ terms we…
In random parameter estimation, Bayesian lower bounds (BLBs) for the mean-square error have been noticed to not be tight in a number of cases, even when the sample size, or the signal-to-noise ratio, grow to infinity. In this paper, we…
We show that the Schnakenberg's entropy production rate in a master equation is lower bounded by a function of the weight of the Markov graph, here defined as the sum of the absolute values of probability currents over the edges. The result…
In this paper the Output Statistics of Random Binning (OSRB) framework is used to prove a new inner bound for the problem of secure channel simulation. Our results subsume some recent results on the secure function computation. We also…
This work presents joint minimum mean-square error (MMSE) consensus algorithm and relay selection algorithms for distributed beamforming. We propose joint MMSE consensus relay and selection schemes with a total power constraint and local…
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from…
The distortion-rate performance of certain randomly-designed scalar quantizers is determined. The central results are the mean-squared error distortion and output entropy for quantizing a uniform random variable with thresholds drawn…
For open systems described by the quantum master equation (QME), we investigate the excess entropy production under quasistatic operations between nonequilibrium steady states. The average entropy production is composed of the time integral…
The channel output entropy of a transmitted word is the entropy of the possible channel outputs and similarly, the input entropy of a received word is the entropy of all possible transmitted words. The goal of this work is to study these…
We consider the problem of Linear Programming (LP) decoding of binary linear codes. The LP excess lemma was introduced by the first author, B. Ghazi, and R. Urbanke (IEEE Trans. Inf. Th., 2014) as a technique to trade crossover probability…
We study the problem of estimating a random process from the observations collected by a network of sensors that operate under resource constraints. When the dynamics of the process and sensor observations are described by a state-space…
The Landauer principle bridges the energetic cost and information processing, showing that irreversible computation inevitably demands energy dissipation. As energy demands from computation continue to rise, approximate computing has…
Optimal estimation of a coin's bias using noisy data is surprisingly different from the same problem with noiseless data. We study this problem using entropy risk to quantify estimators' accuracy. We generalize the "add Beta" estimators…
Measuring entropy production of a system directly from the experimental data is highly desirable since it gives a quantifiable measure of the time-irreversibility for non-equilibrium systems and can be used as a cost function to optimize…
State estimation is a classical problem in quantum information. In optimization of estimation scheme, to find a lower bound to the error of the estimator is a very important step. So far, all the proposed tractable lower bounds use…
The error coefficient of a linear code is defined as the number of minimum-weight codewords. In an additive white Gaussian noise channel, optimal linear codes with the smallest error coefficients achieve the best possible asymptotic frame…