Rethinking the Setting of Semi-supervised Learning on Graphs
Abstract
We argue that the present setting of semisupervised learning on graphs may result in unfair comparisons, due to its potential risk of over-tuning hyper-parameters for models. In this paper, we highlight the significant influence of tuning hyper-parameters, which leverages the label information in the validation set to improve the performance. To explore the limit of over-tuning hyperparameters, we propose ValidUtil, an approach to fully utilize the label information in the validation set through an extra group of hyper-parameters. With ValidUtil, even GCN can easily get high accuracy of 85.8% on Cora. To avoid over-tuning, we merge the training set and the validation set and construct an i.i.d. graph benchmark (IGB) consisting of 4 datasets. Each dataset contains 100 i.i.d. graphs sampled from a large graph to reduce the evaluation variance. Our experiments suggest that IGB is a more stable benchmark than previous datasets for semisupervised learning on graphs.
Cite
@article{arxiv.2205.14403,
title = {Rethinking the Setting of Semi-supervised Learning on Graphs},
author = {Ziang Li and Ming Ding and Weikai Li and Zihan Wang and Ziyu Zeng and Yukuo Cen and Jie Tang},
journal= {arXiv preprint arXiv:2205.14403},
year = {2022}
}
Comments
To appear in IJCAI 2022