Federated PAC-Bayesian Learning on Non-IID data
Machine Learning
2023-09-14 v1 Distributed, Parallel, and Cluster Computing
Abstract
Existing research has either adapted the Probably Approximately Correct (PAC) Bayesian framework for federated learning (FL) or used information-theoretic PAC-Bayesian bounds while introducing their theorems, but few considering the non-IID challenges in FL. Our work presents the first non-vacuous federated PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique prior knowledge for each client and variable aggregation weights. We also introduce an objective function and an innovative Gibbs-based algorithm for the optimization of the derived bound. The results are validated on real-world datasets.
Keywords
Cite
@article{arxiv.2309.06683,
title = {Federated PAC-Bayesian Learning on Non-IID data},
author = {Zihao Zhao and Yang Liu and Wenbo Ding and Xiao-Ping Zhang},
journal= {arXiv preprint arXiv:2309.06683},
year = {2023}
}