Generate classical Chinese poems with theme-style from images

摘要

The automatic generation of poems from images is a classic task in natural language processing. Recently, it has gained tremendous research interest due to the help of neural sequence-to-sequence networks. The current methods are not qualified to generate poems under the joint control of the theme and style conditions, while these two key factors are critical to the quality of Chinese poetry. This paper proposes an image-based Chinese poem generation network (ICPGN), which can obtain the holistic content and sentiments in images and transform them into theme and style representations. A poem theme and style control module (PTSC) is designed to ensure that the generated poems meet the theme and style requirements simultaneously. A new dataset CQC2020 is constructed for training and testing of the proposed network and other related methods. Extensive manual and machine experiment results demonstrate the effectiveness and competitiveness of the proposed method. (c) 2021 Elsevier B.V. All rights reserved.