Can Prompt Learning Benefit Radiology Report Generation?

by   Jun Wang, et al.

Radiology report generation aims to automatically provide clinically meaningful descriptions of radiology images such as MRI and X-ray. Although great success has been achieved in natural scene image captioning tasks, radiology report generation remains challenging and requires prior medical knowledge. In this paper, we propose PromptRRG, a method that utilizes prompt learning to activate a pretrained model and incorporate prior knowledge. Since prompt learning for radiology report generation has not been explored before, we begin with investigating prompt designs and categorise them based on varying levels of knowledge: common, domain-specific and disease-enriched prompts. Additionally, we propose an automatic prompt learning mechanism to alleviate the burden of manual prompt engineering. This is the first work to systematically examine the effectiveness of prompt learning for radiology report generation. Experimental results on the largest radiology report generation benchmark, MIMIC-CXR, demonstrate that our proposed method achieves state-of-the-art performance. Code will be available upon the acceptance.


page 4

page 8


Knowledge Matters: Radiology Report Generation with General and Specific Knowledge

Automatic radiology report generation is critical in clinics which can r...

Prior Knowledge Enhances Radiology Report Generation

Radiology report generation aims to produce computer-aided diagnoses to ...

Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment

Generating radiology reports is time-consuming and requires extensive ex...

Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation

Automatically generating radiology reports can improve current clinical ...

Competence-based Multimodal Curriculum Learning for Medical Report Generation

Medical report generation task, which targets to produce long and cohere...

Confidence-Guided Radiology Report Generation

Medical imaging plays a pivotal role in diagnosis and treatment in clini...

Reading Radiology Imaging Like The Radiologist

Automated radiology report generation aims to generate radiology reports...

Please sign up or login with your details

Forgot password? Click here to reset