文本分组适配器:为布局分析适配预训练文本检测器
摘要
自深度学习兴起以来,场景文本检测模型取得了显著进展,但旨将检测到的文本实例分组为段落的场景文本布局分析尚未跟上。以往的工作要么将文本检测和分组分别处理,要么使用统一模型从头训练。它们尚未充分利用已经很好地训练好的文本检测器以及容易获得的数据集。在本文中,我们提出了文本分组适配器(Text Grouping Adapter, TGA),这是一个可以使各种预训练文本检测器用于学习布局分析的模块,使我们能够直接使用已经训练好的文本检测器或有效地微调它。 designed to be compatible with various text detector architectures, TGA takes detected text regions and image features as universal inputs to assemble text instance features. To capture broader contextual information for layout analysis, we propose to predict text group masks from text instance features by one-to-many assignment. Our comprehensive experiments demonstrate that, even with frozen pre-trained models, incorporating our TGA into various pre-trained text detectors and text spotters can achieve superior layout analysis performance, simultaneously inheriting generalized text detection ability from pre-training. In the case of full parameter fine-tuning, we can further improve layout analysis performance.
引用
@article{arxiv.2405.07481,
title = {Text Grouping Adapter: Adapting Pre-trained Text Detector for Layout Analysis},
author = {Tianci Bi and Xiaoyi Zhang and Zhizheng Zhang and Wenxuan Xie and Cuiling Lan and Yan Lu and Nanning Zheng},
journal= {arXiv preprint arXiv:2405.07481},
year = {2024}
}
备注
Accepted to CVPR 2024