Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds
Machine Learning
2023-11-01 v1 Information Theory
Machine Learning
math.IT
Abstract
We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. Our approach yields sharper bounds that improve upon previous information-theoretic bounds in various learning scenarios. Notably, these bounds address the limitations of existing information-theoretic bounds in the context of stochastic convex optimization (SCO) problems, as explored in the recent work by Haghifam et al. (2023).
Keywords
Cite
@article{arxiv.2310.20102,
title = {Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Bounds},
author = {Ziqiao Wang and Yongyi Mao},
journal= {arXiv preprint arXiv:2310.20102},
year = {2023}
}
Comments
Accepted at NeurIPS 2023