Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning

Maximilian Egger; Mayank Bakshi; Rawad Bitar

Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning

Machine Learning 2025-02-04 v1 Cryptography and Security Distributed, Parallel, and Cluster Computing Machine Learning

Authors: Maximilian Egger , Mayank Bakshi , Rawad Bitar

Abstract

We introduce CyBeR-0, a Byzantine-resilient federated zero-order optimization method that is robust under Byzantine attacks and provides significant savings in uplink and downlink communication costs. We introduce transformed robust aggregation to give convergence guarantees for general non-convex objectives under client data heterogeneity. Empirical evaluations for standard learning tasks and fine-tuning large language models show that CyBeR-0 exhibits stable performance with only a few scalars per-round communication cost and reduced memory requirements.

Keywords

byzantine fault tolerance

Cite

@article{arxiv.2502.00193,
  title  = {Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning},
  author = {Maximilian Egger and Mayank Bakshi and Rawad Bitar},
  journal= {arXiv preprint arXiv:2502.00193},
  year   = {2025}
}

Related papers

View all related →