Summarizing multimodal documents in popular media for people with visual impairments

Abstract

People without visual impairments often skim a document and look at the graphics it contains before deciding whether to read it in detail. While this is easy for sighted people, it is far more challenging for people with visual impairments, especially since the graphical content is largely inaccessible. We propose to increase the universal accessibility of articles in popular media containing both text and graphical elements for people with visual impairments by automatically generating an abstractive summary of the entire multimodal document. Our goal is to enable users to quickly access the high-level content of a multimodal document, and to decide whether to commit to reading the full article.