We introduce SONO, a novel method leveraging Second-Order Neural Ordinary Differential Equations (Second-Order NODEs) to enhance cross-modal few-shot learning. By employing a simple yet effective architecture consisting of a Second-Order NODEs model paired with a cross-modal classifier, SONO addresses the significant challenge of overfitting, which is common in few-shot scenarios due to limited training examples. Our second-order approach can approximate a broader class of functions, enhancing the model's expressive power and feature generalization capabilities. We initialize our cross-modal classifier with text embeddings derived from class-relevant prompts, streamlining training efficiency by avoiding the need for frequent text encoder processing. Additionally, we utilize text-based image augmentation, exploiting CLIP's robust image-text correlation to enrich training data significantly. Extensive experiments across multiple datasets demonstrate that SONO outperforms existing state-of-the-art methods in few-shot learning performance.
@article{arxiv.2412.15813,
title = {Cross-Modal Few-Shot Learning with Second-Order Neural Ordinary Differential Equations},
author = {Yi Zhang and Chun-Wun Cheng and Junyi He and Zhihai He and Carola-Bibiane Schönlieb and Yuyan Chen and Angelica I Aviles-Rivero},
journal= {arXiv preprint arXiv:2412.15813},
year = {2024}
}