English

Aggregation Design for Personalized Federated Multi-Modal Learning over Wireless Networks

Information Theory 2024-06-13 v1 Signal Processing math.IT

Abstract

Federated Multi-Modal Learning (FMML) is an emerging field that integrates information from different modalities in federated learning to improve the learning performance. In this letter, we develop a parameter scheduling scheme to improve personalized performance and communication efficiency in personalized FMML, considering the non-independent and nonidentically distributed (non-IID) data along with the modality heterogeneity. Specifically, a learning-based approach is utilized to obtain the aggregation coefficients for parameters of different modalities on distinct devices. Based on the aggregation coefficients and channel state, a subset of parameters is scheduled to be uploaded to a server for each modality. Experimental results show that the proposed algorithm can effectively improve the personalized performance of FMML.

Keywords

Cite

@article{arxiv.2406.07915,
  title  = {Aggregation Design for Personalized Federated Multi-Modal Learning over Wireless Networks},
  author = {Benshun Yin and Zhiyong Chen and Meixia Tao},
  journal= {arXiv preprint arXiv:2406.07915},
  year   = {2024}
}

Comments

accepted by IEEE Communications Letters

R2 v1 2026-06-28T17:02:39.625Z