Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective
Information Theory
2021-05-12 v5 Machine Learning
Signal Processing
math.IT
Statistics Theory
Machine Learning
Statistics Theory
Abstract
In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach. It is seen in general that the error bounds are determined by the conditional entropy of the data point to be estimated or predicted given the side information or past observations. Additionally, we discover that in order to achieve the prediction error bounds asymptotically, the necessary and sufficient condition is that the "innovation" is asymptotically white Gaussian. When restricted to Gaussian processes and 1-step prediction, our bounds are shown to reduce to the Kolmogorov-Szeg\"o formula and Wiener-Masani formula known from linear prediction theory.
Cite
@article{arxiv.1904.04765,
title = {Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective},
author = {Song Fang and Mikael Skoglund and Karl Henrik Johansson and Hideaki Ishii and Quanyan Zhu},
journal= {arXiv preprint arXiv:1904.04765},
year = {2021}
}