Federated Learning for Non-IID Data via Client Variance Reduction and Adaptive Server Update
Abstract
Federated learning (FL) is an emerging technique used to collaboratively train a global machine learning model while keeping the data localized on the user devices. The main obstacle to FL's practical implementation is the Non-Independent and Identical (Non-IID) data distribution across users, which slows convergence and degrades performance. To tackle this fundamental issue, we propose a method (ComFed) that enhances the whole training process on both the client and server sides. The key idea of ComFed is to simultaneously utilize client-variance reduction techniques to facilitate server aggregation and global adaptive update techniques to accelerate learning. Our experiments on the Cifar-10 classification task show that ComFed can improve state-of-the-art algorithms dedicated to Non-IID data.
Cite
@article{arxiv.2207.08391,
title = {Federated Learning for Non-IID Data via Client Variance Reduction and Adaptive Server Update},
author = {Hiep Nguyen and Lam Phan and Harikrishna Warrier and Yogesh Gupta},
journal= {arXiv preprint arXiv:2207.08391},
year = {2022}
}