Like Humans to Few-Shot Learning through Knowledge Permeation of Vision and Text
Abstract
Few-shot learning aims to generalize the recognizer from seen categories to an entirely novel scenario. With only a few support samples, several advanced methods initially introduce class names as prior knowledge for identifying novel classes. However, obstacles still impede achieving a comprehensive understanding of how to harness the mutual advantages of visual and textual knowledge. In this paper, we propose a coherent Bidirectional Knowledge Permeation strategy called BiKop, which is grounded in a human intuition: A class name description offers a general representation, whereas an image captures the specificity of individuals. BiKop primarily establishes a hierarchical joint general-specific representation through bidirectional knowledge permeation. On the other hand, considering the bias of joint representation towards the base set, we disentangle base-class-relevant semantics during training, thereby alleviating the suppression of potential novel-class-relevant information. Experiments on four challenging benchmarks demonstrate the remarkable superiority of BiKop. Our code will be publicly available.
Keywords
Cite
@article{arxiv.2405.12543,
title = {Like Humans to Few-Shot Learning through Knowledge Permeation of Vision and Text},
author = {Yuyu Jia and Qing Zhou and Wei Huang and Junyu Gao and Qi Wang},
journal= {arXiv preprint arXiv:2405.12543},
year = {2024}
}