On Constructing Confidence Region for Model Parameters in Stochastic Gradient Descent via Batch Means
Machine Learning
2020-02-03 v2 Machine Learning
Statistics Theory
Statistics Theory
Abstract
In this paper, we study a simple algorithm to construct asymptotically valid confidence regions for model parameters using the batch means method. The main idea is to cancel out the covariance matrix which is hard/costly to estimate. In the process of developing the algorithm, we establish process-level functional central limit theorem for Polyak-Ruppert averaging based stochastic gradient descent estimators. We also extend the batch means method to accommodate more general batch size specifications.
Cite
@article{arxiv.1911.01483,
title = {On Constructing Confidence Region for Model Parameters in Stochastic Gradient Descent via Batch Means},
author = {Yi Zhu and Jing Dong},
journal= {arXiv preprint arXiv:1911.01483},
year = {2020}
}