OCC-MLLM-Alpha：通过自监督测试时学习赋能多模态大语言模型以理解被遮挡对象
计算机视觉与模式识别
2024-10-04 v1
摘要
现有大规模视觉语言多模态模型在理解被遮挡对象方面存在差距。当前最先进的多模态模型无法通过通用视觉编码器和监督学习策略提供令人满意的结果。因此，我们引入了一个多模态大型语言框架和相应的自监督学习策略，支持3D生成。我们首先在大规模数据集SOMVideo[18]的评估上与最先进的模型进行比较。初始结果表明，与最先进的VLM模型相比，改进了16.92%。
引用
@article{arxiv.2410.01861,
title = {OCC-MLLM-Alpha:Empowering Multi-modal Large Language Model for the Understanding of Occluded Objects with Self-Supervised Test-Time Learning},
author = {Shuxin Yang and Xinhan Di},
journal= {arXiv preprint arXiv:2410.01861},
year = {2024}
}
备注
Accepted by ECCV 2024 Observing and Understanding Hands in Action Workshop (5 pages, 3 figures, 2 tables). arXiv admin note: substantial text overlap with arXiv:2410.01261