Chapter 19. Neuro-Symbolic Spatio-Temporal Reasoning

Abstract

Knowledge about space and time is necessary to solve problems in the physical world. Spatio-temporal knowledge, however, is required beyond interacting with the physical world, and is also often transferred to the abstract world of concepts through analogies and metaphors. As spatial and temporal reasoning is ubiquitous, different attempts have been made to integrate this into AI systems. In the area of knowledge representation, spatial and temporal reasoning has been largely limited to modeling objects and relations and developing reasoning methods to verify statements about objects and relations. On the other hand, neural network researchers have tried to teach models to learn spatial relations from data with limited reasoning capabilities. Bridging the gap between these two approaches in a mutually beneficial way could allow us to tackle many complex real-world problems. In this chapter, we view this integration problem from the perspective of Neuro-Symbolic AI. Specifically, we propose a synergy between logical reasoning and machine learning that will be grounded on spatial and temporal knowledge. A (symbolic) spatio-temporal knowledge base and a base of possibly grounded examples could provide a dependable causal seed upon which machine learning models could generalize. Describing some successful applications, remaining challenges, and evaluation datasets pertaining to this direction is the main topic of this contribution.