Classical Sequence Match is a Competitive Few-Shot One-Class Learner
Abstract
Nowadays, transformer-based models gradually become the default choice for artificial intelligence pioneers. The models also show superiority even in the few-shot scenarios. In this paper, we revisit the classical methods and propose a new few-shot alternative. Specifically, we investigate the few-shot one-class problem, which actually takes a known sample as a reference to detect whether an unknown instance belongs to the same class. This problem can be studied from the perspective of sequence match. It is shown that with meta-learning, the classical sequence match method, i.e. Compare-Aggregate, significantly outperforms transformer ones. The classical approach requires much less training cost. Furthermore, we perform an empirical comparison between two kinds of sequence match approaches under simple fine-tuning and meta-learning. Meta-learning causes the transformer models' features to have high-correlation dimensions. The reason is closely related to the number of layers and heads of transformer models. Experimental codes and data are available at https://github.com/hmt2014/FewOne
Cite
@article{arxiv.2209.06394,
title = {Classical Sequence Match is a Competitive Few-Shot One-Class Learner},
author = {Mengting Hu and Hang Gao and Yinhao Bai and Mingming Liu},
journal= {arXiv preprint arXiv:2209.06394},
year = {2022}
}
Comments
COLING 2022; Added a missing citation compared with the last version