big.LITTLE Vision Transformer用于高效视觉识别
摘要
本文介绍了big.LITTLE Vision Transformer，这是一种旨在实现高效视觉识别的创新架构。该双Transformer系统由两个 distinct 块组成：big性能块具有高容量和巨大的计算需求，LITTLE效率块设计用于速度，具有较低的容量。我们方法的关键创新在于其动态推理机制。处理图像时，系统确定每个token的重要性并分配处理：essential tokens由高性能big模型处理，而不重要的tokens由更高效的little模型处理。这种选择性处理显著减少了计算负载，而不牺牲模型整体性能，因为它确保最重要的信息得到详细分析。为了验证big.LITTLE Vision Transformer的有效性，我们在图像分类和segment anything任务上进行了全面实验。我们的结果表明，big.LITTLE架构不仅保持高准确率，还实现了显著的计算节省。 Specifically, our approach enables the efficient handling of large-scale visual recognition tasks by dynamically balancing the trade-offs between performance and efficiency. The success of our method underscores the potential of hybrid models in optimizing both computation and performance in visual recognition tasks, paving the way for more practical and scalable deployment of advanced neural networks in real-world applications.
引用
@article{arxiv.2410.10267,
title = {big.LITTLE Vision Transformer for Efficient Visual Recognition},
author = {He Guo and Yulong Wang and Zixuan Ye and Jifeng Dai and Yuwen Xiong},
journal= {arXiv preprint arXiv:2410.10267},
year = {2024}
}