Transforming Hand-Drawn Sketches of Linkage Mechanisms Into Their Digital Representation

Abstract

Abstract This paper introduces a new method using deep neural networks for the interactive digital transformation and simulation of n-bar planar linkages, which consist of revolute and prismatic joints, based on hand-drawn sketches. Instead of relying solely on computer vision, our approach combines topological knowledge of linkage mechanisms with the outcomes of a convolutional deep neural network. This creates a framework for recognizing hand-drawn sketches. We generate a dataset of synthetic images that resemble hand-drawn sketches of linkage mechanisms. Next, we fine-tune a state-of-the-art deep neural network to detect discrete objects using building blocks that represent joints and links in various positions, sizes, and orientations within these sketches. We then conduct a topological analysis on the detected objects to construct a kinematic model of the sketched mechanisms. The results demonstrate the effectiveness of our algorithm in handling hand-drawn sketches and converting them into digital representations. This has practical implications for improving communication, analysis, organization, and classification of planar mechanisms.