Automated analysis of chest radiography using deep learning has tremendous potential to enhance the clinical diagnosis of diseases in patients. However, deep learning models typically require large amounts of annotated data to achieve high performance -- often an obstacle to medical domain adaptation. In this paper, we build a data-efficient learning framework that utilizes radiology reports to improve medical image classification performance with limited labeled data (fewer than 1000 examples). Specifically, we examine image-captioning pretraining to learn high-quality medical image representations that train on fewer examples. Following joint pretraining of a convolutional encoder and transformer decoder, we transfer the learned encoder to various classification tasks. Averaged over 9 pathologies, we find that our model achieves higher classification performance than ImageNet-supervised and in-domain supervised pretraining when labeled training data is limited.
@article{arxiv.2208.03218,
title = {RadTex: Learning Efficient Radiograph Representations from Text Reports},
author = {Keegan Quigley and Miriam Cha and Ruizhi Liao and Geeticka Chauhan and Steven Horng and Seth Berkowitz and Polina Golland},
journal= {arXiv preprint arXiv:2208.03218},
year = {2023}
}
Comments
Awarded Best Paper at Resource Efficient Medical Image Analysis (REMIA) Workshop, MICCAI 2022