English

FedHPro: Federated Hyper-Prototype Learning via Gradient Matching

Computer Vision and Pattern Recognition 2026-05-21 v2

Abstract

Federated Learning (FL) enables collaborative training of distributed clients while protecting privacy. To enhance generalization capability in FL, prototype-based FL is in the spotlight, since shared global prototypes offer semantic anchors for aligning client-specific local prototypes. However, existing methods update global prototypes at the prototype-level via averaging local prototypes or refining global anchors, which often leads to semantic drift across clients and subsequently yields a misaligned global signal. To alleviate this issue, we introduce hyper-prototypes, defined by a set of learnable global class-wise prototypes to preserve underlying semantic knowledge across clients. The hyper-prototypes are optimized via gradient matching to align with class-relevant characteristics distilled directly from clients' real samples, rather than prototype-level descriptors. We further propose FedHPro, a Federated Hyper-Prototype Learning framework, to leverage hyper-prototypes to promote inter-class separability via mutual-contrastive learning with client-specific margin, while encouraging intra-class uniformity through a consistency penalty. Comprehensive experiments under diverse heterogeneous scenarios confirm that 1) hyper-prototypes produce a more semantically consistent global signal, and 2) FedHPro achieves state-of-the-art performance on several benchmark datasets. Code is available at \href{https://github.com/mala-lab/FedHPro}{https://github.com/mala-lab/FedHPro}.

Keywords

Cite

@article{arxiv.2605.13475,
  title  = {FedHPro: Federated Hyper-Prototype Learning via Gradient Matching},
  author = {Huan Wang and Jun Shen and Haoran Li and Zhenyu Yang and Jun Yan and Ousman Manjang and Yanlong Zhai and Di Wu and Guansong Pang},
  journal= {arXiv preprint arXiv:2605.13475},
  year   = {2026}
}

Comments

23 pages, ICML 2026 Camera-ready Version