Radiology reporting is a complex task requiring detailed medical image understanding and precise language generation, for which generative multimodal models offer a promising solution. However, to impact clinical practice, models must achieve a high level of both verifiable performance and utility. We augment the utility of automated report generation by incorporating localisation of individual findings on the image - a task we call grounded report generation - and enhance performance by incorporating realistic reporting context as inputs. We design a novel evaluation framework (RadFact) leveraging the logical inference capabilities of large language models (LLMs) to quantify report correctness and completeness at the level of individual sentences, while supporting the new task of grounded reporting. We develop MAIRA-2, a large radiology-specific multimodal model designed to generate chest X-ray reports with and without grounding. MAIRA-2 achieves state of the art on existing report generation benchmarks and establishes the novel task of grounded report generation.
Cite
@article{arxiv.2406.04449,
title = {MAIRA-2: Grounded Radiology Report Generation},
author = {Shruthi Bannur and Kenza Bouzid and Daniel C. Castro and Anton Schwaighofer and Anja Thieme and Sam Bond-Taylor and Maximilian Ilse and Fernando Pérez-García and Valentina Salvatelli and Harshita Sharma and Felix Meissen and Mercy Ranjit and Shaury Srivastav and Julia Gong and Noel C. F. Codella and Fabian Falck and Ozan Oktay and Matthew P. Lungren and Maria Teodora Wetscherek and Javier Alvarez-Valle and Stephanie L. Hyland},
journal= {arXiv preprint arXiv:2406.04449},
year = {2024}
}
Comments
72 pages, 21 figures. v2 updates the model and adds results on the PadChest-GR dataset