FEATHERS:联邦架构与超参数搜索
机器学习
2023-03-28 v3
摘要
深度神经架构对当今许多 AI 任务所取得的性能有深远影响,然而其设计仍严重依赖人类的先验知识与经验。神经架构搜索(NAS)与超参数优化(HO)有助于减少这种依赖。然而,最先进的 NAS 与 HO 随着以分布式方式存储的数据量增长而迅速变得不可行,且通常违反 GDPR 与 CCPA 等数据隐私法规。作为补救,我们引入 FEATHERS——derated rchiecture and ypparameter earch,一种不仅在分布式数据设置中联合优化神经架构与优化相关超参数,而且通过使用差分隐私(DP)进一步遵守数据隐私的方法。我们展示 FEATHERS 能高效优化架构与优化相关超参数,同时在分类任务上展现出收敛性,且在符合隐私约束时对模型性能无损害。
引用
@article{arxiv.2206.12342,
title = {FEATHERS: Federated Architecture and Hyperparameter Search},
author = {Jonas Seng and Pooja Prasad and Martin Mundt and Devendra Singh Dhami and Kristian Kersting},
journal= {arXiv preprint arXiv:2206.12342},
year = {2023}
}
备注
Main paper: 8 pages, References: 2 pages, Supplement: 4.5 pages, Main paper: 3 figures, 2 tables, 1 algorithm, Supplement: 2 figure, 4 algorithms, extended previous version by Differential Privacy, theoretical results and more experiments. Updated author list as it was incomplete