Deep Understanding of Technical Documents: Part I. Diagrams Structural-functional Modeling

Abstract

The automatic deep understanding of technical documents is a privilege only to humans so far, since it requires knowledge coming from many different modalities, like text, diagrams, formulas, tables, graphics, pictures, etc. Thus, in response to this very large and complex challenge, this paper investigates the synergistic association of only two modalities, the diagrams as main modality and natural language text as an assistive one in an effort to combine them together for deeper understanding of technical documents. In particular, it presents the formal modelling of a hybrid methodology capable to automatically extract the structural and functional behavior of a system described in a technical document without the use of original code. By system here we mean the block diagram(s) of a system. The methodology presented here is based on a formal language, called Synergy, to efficiently represent and synthesize the structural features of the system, and convert them into a Stochastic Petri-nets (SPN) model as for expressing the functional behavior of the understudy system. The overall methodology will contribute to an automatic deep understanding of technical documents (TD) without the main involvement of human users.