Medical Report Generation for Chest X-rays Using Convolutional Recurrent and Attention-Based Architectures

Articles in Press

Document Type : Computer Article

Authors

1 MSc, Biomedical Engineering Department, Hamedan University of Technology, Hamedan, Iran

2 Assistant Professor, Biomedical Engineering Department- Hamedan University of Technology

3 Associate Professor, Biomedical Engineering Department, Hamedan University of Technology, Hamedan, Iran

Abstract

Medical images are extensively used in medical science for diagnosis and treatment protocol design. Writing medical reports in text form can be error-prone for inexperienced physicians due to the deep understanding of the disease and its analysis. It is also time-consuming and laborious for experts due to the large number of patients they see in a day. Also, the existence of template reports for physicians can significantly increase their accuracy in diagnosing diseases and reduce errors caused by inattention to details. This research presents a deep learning-based model for the automatic generation of radiology reports. This model is based on a combination of a convolutional recurrent structure and an attention-based architecture called Res-LSTM-Attn. In this model, features are first extracted from medical images using a convolutional residual network, and based on a multi-label word model, a report is predicted. Then, using the LSTM recurrent neural network and multi-head attention layers, the final report is generated. The performance of the proposed models was evaluated based on the BLEU 1-4, ROUGE-L, and CIDEr-D criteria. The results showed that the proposed model outperformed previous studies in generating long reports in terms of CIDEr-D and ROUGE-L metrics, with improvements of 7.2% and 3.2%, respectively.

Keywords

Main Subjects

Journal of Modeling in Engineering

Articles in Press, Accepted Manuscript
Available Online from 14 September 2025

Files

History

  • Receive Date: 05 November 2024
  • Revise Date: 30 May 2025
  • Accept Date: 21 June 2025

Share

How to cite

Statistics