Aesthetic-Aware Text to Image Synthesis

摘要

Synthesizing an image from natural language description is an important task to many applications such as photo-editing, art generation, and computer aided-design. However, to synthesize an appealing image from the text, image aesthetics criteria should be maintained. In this study, we propose a new framework which first generates a set of mask maps from the input text via mask map generator (MG), and then we compute and rank the image aesthetics score for all generated mask maps via Pre-IG Aesthetic Ranking that contains two composition rules, i.e., the rule of thirds along with the rule of formal balance. At the next stage, we feed the subset of the mask maps, which are the highest, lowest, and the average aesthetic scores, to image generator (IG). The photorealistic images are ranked at the second round through, namely Post-IG Aesthetic Ranking, to determine the lowest aesthetic score and return the most appealing generated image. The experiments on COCO-stuff dataset demonstrate that our framework yields better results compared to previous text-to- image models.