Automatic Computation of Meaning in Authored Images Such as Artworks: A Grand Challenge for AI

Abstract

We discuss preliminary successes and major outstanding challenges in extracting messages, stories, morals, and especially meaning in crafted or “authored” images, such as artworks. Traditional semantic image understanding seeks to summarize an image (such as through a caption), or to answer basic questions expressed in free-text natural language, but can neither infer plausible reasons the creator made the artwork nor compute a high-level message or meaning it conveys. Such meaning is often abstract, dependent upon genre, period, and art movement, and exploits visual conventions such as special objects and signs (“signifiers”), composition, and possibly non-realistic style. Several of these properties have no counterpart in the natural photographs that are studied in most semantic image analysis, nor in the application-specific image analysis applied to robotics, autonomous driving, medical diagnosis, or remote sensing. For such reasons, the extraction of meaning from works in a variety of styles and subjects will be both a tool for art scholars and a grand challenge to artificial intelligence research.