Estimating the Fundamental Limits is Easier than Achieving the Fundamental Limits

Jiantao Jiao; Yanjun Han; Irena Fischer-Hwang; Tsachy Weissman

Estimating the Fundamental Limits is Easier than Achieving the Fundamental Limits

Information Theory 2017-10-03 v2 math.IT Machine Learning

Authors: Jiantao Jiao , Yanjun Han , Irena Fischer-Hwang , Tsachy Weissman

Abstract

We show through case studies that it is easier to estimate the fundamental limits of data processing than to construct explicit algorithms to achieve those limits. Focusing on binary classification, data compression, and prediction under logarithmic loss, we show that in the finite space setting, when it is possible to construct an estimator of the limits with vanishing error with $n$ samples, it may require at least $n\ln n$ samples to construct an explicit algorithm to achieve the limits.

Keywords

fundamental limits binary classification data compression prediction under logarithmic loss finite space setting sample complexity explicit algorithms

Cite

@article{arxiv.1707.01203,
  title  = {Estimating the Fundamental Limits is Easier than Achieving the Fundamental Limits},
  author = {Jiantao Jiao and Yanjun Han and Irena Fischer-Hwang and Tsachy Weissman},
  journal= {arXiv preprint arXiv:1707.01203},
  year   = {2017}
}

Estimating the Fundamental Limits is Easier than Achieving the Fundamental Limits

Abstract

Keywords

Cite

Related papers