PadChest-GR: A Bilingual Chest X-Ray Dataset for Grounded Radiology Report Generation

Artificial intelligence (AI)-powered radiology report generation (RRG) aims to create free-text radiology reports from clinical imaging. Grounded radiology report generation (GRRG) augments RRG by including the localization of individual findings on the image. Currently, to our knowledge, no manually annotated chest x-ray (CXR) datasets exist on which to train GRRG models.

In this article, we present a dataset called PadChest-GR (grounded reporting), which is derived from the CXR dataset, PadChest, and aimed at training GRRG models to analyze CXR images. First, we selected a subset of studies from PadChest that contained images with frontal projection; studies that were originally labeled as suboptimal and those involving pediatric patients were excluded. Then, using Generative Pretrained Transformer 4 in Microsoft Azure OpenAI Service, we processed reports to extract sentences with single findings, translate them from Spanish into English, link them to the existing PadChest finding and location labels, and classify the finding progression. A team of 14 radiologists discarded studies with poor image quality or issues relating to the report or findings list and then manually annotated the findings using bounding boxes to surround regions of interest in each image.

We curated a public bilingual dataset of 4555 CXR studies with grounded reports, of which 3099 were abnormal and 1456 were normal. Each report contains complete lists of sentences describing individual present (positive) findings and absent (negative) findings in English and Spanish. In total, PadChest-GR contains 7037 positive-finding sentences and 3422 negative-finding sentences. Every positive-finding sentence is associated with up to two independent sets of bounding boxes labeled by different readers and has categorical labels for finding type, locations, and progression.

PadChest-GR is a manually curated dataset designed to train GRRG models to understand and interpret radiological images and generated text. By including detailed localization and comprehensive annotations of all clinically relevant findings, PadChest-GR provides a valuable resource for developing and evaluating GRRG models from CXR images. (Funded by the Microsoft Corporation.).