Related papers: A Bilateral Bound on the Mean-Square Error for Est…
Generalized linear models are a popular tool in applied statistics, with their maximum likelihood estimators enjoying asymptotic Gaussianity and efficiency. As all models are wrong, it is desirable to understand these estimators' behaviours…
Bayesian inference typically requires the computation of an approximation to the posterior distribution. An important requirement for an approximate Bayesian inference algorithm is to output high-accuracy posterior mean and uncertainty…
In fitting a mixture of linear regression models, normal assumption is traditionally used to model the error and then regression parameters are estimated by the maximum likelihood estimators (MLE). This procedure is not valid if the normal…
Projection-based model order reduction of dynamical systems usually introduces an error between the high-fidelity model and its counterpart of lower dimension. This unknown error can be bounded by residual-based methods, which are typically…
The small sample universal hypothesis testing problem is investigated in this paper, in which the number of samples $n$ is smaller than the number of possible outcomes $m$. The goal of this work is to find an appropriate criterion to…
Multilayer perceptrons (MLP) with one hidden layer have been used for a long time to deal with non-linear regression. However, in some task, MLP's are too powerful models and a small mean square error (MSE) may be more due to overfitting…
We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other…
Mutual information is fundamentally important for measuring statistical dependence between variables and for quantifying information transfer by signaling and communication mechanisms. It can, however, be challenging to evaluate for…
Consistent reconstruction is a method for producing an estimate $\widetilde{x} \in \mathbb{R}^d$ of a signal $x\in \mathbb{R}^d$ if one is given a collection of $N$ noisy linear measurements $q_n = \langle x, \varphi_n \rangle +…
This paper derives lower bounds for the mean square errors of parameter estimators in the case of Poisson distributed data subjected to multiple abrupt changes. Since both change locations (discrete parameters) and parameters of the Poisson…
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find…
In this paper, we improve the PAC-Bayesian error bound for linear regression derived in Germain et al. [10]. The improvements are twofold. First, the proposed error bound is tighter, and converges to the generalization loss with a…
Two-sample inference for the difference of population means typically relies upon a Central Limit Theorem approximation. When data are drawn from a Negative Binomial distribution, previous work of Shilane et al. (2010) showed that a Normal…
Motivated by emerging technologies for energy efficient analog computing and continuous-time processing, this paper proposes continuous-time minimum mean squared error estimation for multiple-input multiple-output (MIMO) systems based on an…
Bayesian error analysis paves the way to the construction of credible and plausible error regions for a point estimator obtained from a given dataset. We introduce the concept of region accuracy for error regions (a generalization of the…
Bagging can significantly improve the generalization performance of unstable machine learning algorithms such as trees or neural networks. Though bagging is now widely used in practice and many empirical studies have explored its behavior,…
We noisily observe solutions of an ordinary differential equation $\dot u = f(u)$ at given times, where $u$ lives in a $d$-dimensional state space. The model function $f$ is unknown and belongs to a H\"older-type smoothness class with…
In this paper we propose a semi-parametric Bayesian Generalized Least Squares estimator. In a generic setting where each error is a vector, the parametric Generalized Least Square estimator maintains the assumption that each error vector…
We consider the estimation of a scalar parameter, when two estimators are available. The first is always consistent. The second is inconsistent in general, but has a smaller asymptotic variance than the first, and may be consistent if an…
We study learning algorithms when there is a mismatch between the distributions of the training and test datasets of a learning algorithm. The effect of this mismatch on the generalization error and model misspecification are quantified.…