基于对象中心学习的分布迁移下自助分割基石模型
摘要
基石模型在实现零样本或少样本泛化方面取得了显著进展，利用提示工程模仿人类智能的问题解决方法。然而，对于诸如分段感知模型（Segment Anything）等基石模型而言，在处理分布外数据（包括伪装图像和医学图像）时仍面临挑战。微调和测试过程中提示策略不一致，这进一步加剧了该问题，导致性能下降。drawing inspiration from how human cognition processes new environments, we introduce SlotSAM, a method that reconstructs features from the encoder in a self-supervised manner to create object-centric representations. These representations are then integrated into the foundation model, bolstering its object-level perceptual capabilities while reducing the impact of distribution-related variables. The beauty of SlotSAM lies in its simplicity and adaptability to various tasks, making it a versatile solution that significantly enhances the generalization abilities of foundation models. Through limited parameter fine-tuning in a bootstrap manner, our approach paves the way for improved generalization in novel environments. The code is available at github.com/lytang63/SlotSAM.
引用
@article{arxiv.2408.16310,
title = {Bootstrap Segmentation Foundation Model under Distribution Shift via Object-Centric Learning},
author = {Luyao Tang and Yuxuan Yuan and Chaoqi Chen and Kunze Huang and Xinghao Ding and Yue Huang},
journal= {arXiv preprint arXiv:2408.16310},
year = {2024}
}
备注
This work is accepted by ECCV 2024 EVAL-FoMo Workshop