Related papers: Minimum MS. E. Gerber's Lemma
For sequence-to-sequence tasks it is challenging to combine individual system outputs. Further, there is also often a mismatch between the decoding criterion and the one used for assessment. Minimum Bayes' Risk (MBR) decoding can be used to…
Empirical Bayes estimators are based on minimizing the average risk with the hyper-parameters in the weighting function being estimated from observed data. The performance of an empirical Bayes estimator is typically evaluated by its mean…
We investigate the theoretical foundations of a recently introduced entropy-based formulation of weighted least squares for the approximation of overdetermined linear systems, motivated by robust data fitting in the presence of sparse gross…
Minimum mean square error (MMSE) estimation of block sparse signals from noisy linear measurements is considered. Unlike in the standard compressive sensing setup where the non-zero entries of the signal are independently and uniformly…
As one of the central tasks in machine learning, regression finds lots of applications in different fields. An existing common practice for solving regression problems is the mean square error (MSE) minimization approach or its regularized…
Small area estimators that ignore the sampling design lack design consistency when the sampling mechanism is complex and may be severely biased under informative designs. Existing procedures that account for the survey weights under…
Kernel techniques are among the most popular and flexible approaches in data science allowing to represent probability measures without loss of information under mild conditions. The resulting mapping called mean embedding gives rise to a…
We examine codes, over the additive Gaussian noise channel, designed for reliable communication at some specific signal-to-noise ratio (SNR) and constrained by the permitted minimum mean-square error (MMSE) at lower SNRs. The maximum…
We establish some new non-asymptotical lower bounds for deviation of regular unbiased estimation of unknown parameter from its true value in different norms, alike the classical Rao-Kramer's inequality. We show that if the new norm is…
The performance of kernel density estimators is usually studied via Taylor expansions and asymptotic approximation arguments, in which the bandwidth parameter tends to zero with increasing sample size. In contrast, this paper focusses…
In this paper we find, for a class of bipartite quantum states, a nontrivial lower bound on the entropy gain resulting from the action of a tensor product of identity channel with an arbitrary channel. By means of that we then estimate…
"Bounds on information combining" are entropic inequalities that determine how the information (entropy) of a set of random variables can change when these are combined in certain prescribed ways. Such bounds play an important role in…
Consider the task of estimating a 3-order $n \times n \times n$ tensor from noisy observations of randomly chosen entries in the sparse regime. We introduce a similarity based collaborative filtering algorithm for estimating a tensor from…
The goal of this paper is to characterize the best achievable performance for the problem of estimating an unknown parameter having a sparse representation. Specifically, we consider the setting in which a sparsely representable…
We consider the problem of estimating an input signal from noisy measurements in both parallel scalar Gaussian channels and linear mixing systems. The performance of the estimation process is quantified by the $\ell_\infty$ norm error…
We present a neural network based predictor which is derived by starting from the linear minimum mean squared error (LMMSE) predictor and by further making two key assumptions. With these assumptions, we first derive a weighted sum of LMMSE…
Post-processing of the raw bits produced by a true random number generator (TRNG) is always necessary when the entropy per bit is insufficient for security applications. In this paper, we derive a tight bound on the output min-entropy of…
We derive fundamental limitations on the performances of intrinsic averaging algorithms in open multi-agent systems, which are systems subject to random arrivals and departures of agents. Each agent holds a value, and their goal is to…
We establish exact asymptotic expressions for the normalized mutual information and minimum mean-square-error (MMSE) of sparse linear regression in the sub-linear sparsity regime. Our result is achieved by a generalization of the adaptive…
We consider the discrete-time intersymbol interference (ISI) channel model, with additive Gaussian noise and fixed i.i.d. inputs. In this setting, we investigate the expression put forth by Shamai and Laroia as a conjectured lower bound for…