Study on Medical Image Report Generation Based on Improved Encoding-Decoding Method

摘要

The automatic description of images has made good advances, but limited by the little-sample dataset, that the automatic generation of medical imaging reports is still a challenging problem. Aiming at the problem of training the joint model (CNN-RNN) on little-sample datasets, this paper proposes an improved encoding-decoding mode, in which the encoder uses less parameter in FCN (Fully Convolutional Network) for identifying lesions in mammography, and encoding it into a semantic vector. The decoder uses a LSTM (Long Short-Term Memory network) for solving, thereby reducing sample requirements. In addition, this paper combines multi-label classification (MLC) to assist the semantic coding process and uses post-processing such as the beam search to make the output fit in the natural language description better. Compared to existing models, our improved model on public mammography dataset (INbreast) with real-world data supplement achieved the BLEU score improvements by two points.