Improving Image Aesthetic Assessment via Multiple Image Joint Learning

Abstract

I mage A esthetic A ssessment (IAA) is an emerging paradigm that predicts aesthetic score as the popular aesthetic taste for an image. Previous IAA approaches take a single image as input to predict the aesthetic score of the image. However, we discover that most existing IAA methods fail dramaticlly to predict the images with large variance of aesthetic voting distribution. Motivated by the practice that people considers similar experiences to improve the consistence of the voting result, we present a novel M ultiple I mage joint L earning Net work (MILNet) to mimic this natural process. Our novelty is mainly three-fold: (a) Semantic-based retrieval method that constructs aesthetic similarity (the similarity of aesthetic attribution) to select reference images; (b) Graph network reasoning that initializes and updates the weight of intrinsic relationships among multiple images; (c) Adaptive Earth Mover’s Distance (AdaEMD) loss function that adjusts weight for easy and hard instances to mitigate unbalanced distribution of aesthetic datasets. Our evaluation with the benchmark AVA and TAD datasets demonstrates that the proposed MILNet outperforms state-of-the-art IAA methods. The code is available at https://github.com/flyingbird93/MILNet .