Digital archiving is becoming widespread owing to its effectiveness in protecting valuable books and providing knowledge to many people electronically. In this paper, we propose a novel approach to leverage digital archives for machine learning. If we can fully utilize such digitized data, machine learning has the potential to uncover unknown insights and ultimately acquire knowledge autonomously, just like humans read books. As a first step, we design a dataset construction pipeline comprising an optical character reader (OCR), an object detector, and a layout analyzer for the autonomous extraction of image-text pairs. In our experiments, we apply our pipeline on old photo books to construct an image-text pair dataset, showing its effectiveness in image-text retrieval and insight extraction.
@article{arxiv.2310.01936,
title = {Constructing Image-Text Pair Dataset from Books},
author = {Yamato Okamoto and Haruto Toyonaga and Yoshihisa Ijiri and Hirokatsu Kataoka},
journal= {arXiv preprint arXiv:2310.01936},
year = {2023}
}
Comments
Accepted at ICCV 2023 workshop, Towards the Next Generation of Computer Vision Datasets: General DataCentric Submission Track