Abstract
AbstractAutomatic storytelling is a broad challenge in research contexts such as Natural Language Processing and Contend Based Image Analysis. Despite the considerable achievements of machine learning techniques in these research fields, combining different approaches to fill the gap between an automatic generated story and human handwriting is hard. This work proposes a novel storytelling framework in the Cultural Heritage domain. We developed our framework based on a Multimedia Knowledge Graph (MKG), a crucial point of our work. Furthermore, we populated our Multimedia Knowledge Graph with a focused crawler that employs deep learning techniques to recognise a multimedia object from web resources. Furthermore, we used a combined approach of deep learning techniques and Linked Open Data (LOD) to retrieve information about images and depicted figures using Instance Segmentation. The system has a dynamic, user-friendly interface that guides the user during the storytelling process. Finally, we evaluated the system from a qualitative and quantitative point of view.
Funder
Università degli Studi di Napoli Federico II
Publisher
Springer Science and Business Media LLC
Subject
Computer Networks and Communications,Hardware and Architecture,Media Technology,Software
Reference46 articles.
1. Aneja J, Deshpande A, Schwing AG (2018) Convolutional image captioning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5561–5570
2. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z (2007) Dbpedia: a nucleus for a web of open data. In: The semantic web. Springer, pp 722–735
3. Bauer F, Kaltenböck M (2011) Linked open data: The essentials. Edition mono/monochrom, Vienna 710
4. Capuano A, Rinaldi AM, Russo C (2020) An ontology-driven multimedia focused crawler based on linked open data and deep learning techniques. Multimed Tools Appl 79(11):7577–7598
5. DB A (2022) Arango DB. https://www.arangodb.com/ Accessed 01 Mar 2022