Related papers: Generic Variance Bounds on Estimation and Predicti…
Safety-critical navigation applications require that estimation errors be reliably quantified and bounded. This can be challenging for linear dynamic systems if the process noise or measurement errors have uncertain time correlation. In…
The existing upper and lower bounds between entropy and error are mostly derived through an inequality means without linking to joint distributions. In fact, from either theoretical or application viewpoint, there exists a need to achieve a…
At the nanoscale, random effects govern not only the dynamics of a physical system but may also affect its observation. This work introduces a novel paradigm for coarse graining that eschews the assignment of a unique coarse-grained…
The goal of this paper is to obtain expectation bounds for the deviation of large sample autocovariance matrices from their means under weak data dependence. While the accuracy of covariance matrix estimation corresponding to independent…
We present an information-theoretic lower bound for the problem of parameter estimation with time-uniform coverage guarantees. Via a new a reduction to sequential testing, we obtain stronger lower bounds that capture the hardness of the…
New upper and lower bounds for the error probability over an erasure channel are provided, making use of Wei's generalized weights, hierarchy and spectra. In many situations the upper and lower bounds coincide and this allows improvement of…
Prediction intervals in supervised Machine Learning bound the region where the true outputs of new samples may fall. They are necessary in the task of separating reliable predictions of a trained model from near random guesses, minimizing…
In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two…
Nonequilibrium processes break time-reversal symmetry and generate entropy. Living systems are driven out-of-equilibrium at the microscopic level of molecular motors that exploit chemical potential gradients to transduce free energy to…
Statistical learning theory has largely focused on learning and generalization given independent and identically distributed (i.i.d.) samples. Motivated by applications involving time-series data, there has been a growing literature on…
The classic Landauer bound can be lowered when erasure errors are permitted. Here we point out that continuous phase transitions characterized by an order parameter can also be viewed as information erasure by resetting a certain number of…
We propose a new method of estimation in high-dimensional linear regression model. It allows for very weak distributional assumptions including heteroscedasticity, and does not require the knowledge of the variance of random errors. The…
We introduce a Gaussian process-based model for handling of non-stationarity. The warping is achieved non-parametrically, through imposing a prior on the relative change of distance between subsequent observation inputs. The model allows…
We study time-uniform statistical inference for parameters in stochastic approximation (SA), which encompasses a bunch of applications in optimization and machine learning. To that end, we analyze the almost-sure convergence rates of the…
Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios.…
This study focuses on conditional entropy frameworks based on the Kolmogorov--Nagumo (KN) mean. First, $(\eta, \psi)$-KN averaging (\texttt{EPKNAVG}), a KN-mean extension of the $\eta$-averaging (\texttt{EAVG}) framework for $(\eta,…
This paper considers the quantification of the prediction performance in Gaussian process regression. The standard approach is to base the prediction error bars on the theoretical predictive variance, which is a lower bound on the mean…
We provide a new information-theoretic generalization error bound that is exactly tight (i.e., matching even the constant) for the canonical quadratic Gaussian (location) problem. Most existing bounds are order-wise loose in this setting,…
Configurational entropy has been revealed as a reliable method for constraining some parameters of a given model [Phys. Rev. D \textbf{92} (2015) 126005, Eur. Phys. J. C \textbf{76} (2016) 100]. In this letter we calculate the…
The study considers advantages of the introduced measure of time based on the entropy change under irreversible processes (entropy production). Using the example of non-equilibrium expansion of an ideal gas in vacuum, such a measure is…