CoR-SketchAR: Cooperative Sketch-Based Real-Time Augmented Reality Authoring Tool for Crowd Simulation

Abstract

In this study, we propose CoR-SketchAR, an augmented reality (AR) environment authoring tool that uses dry-erase markers for real-time collaboration among users. The most important requirement for multi-user collaboration in an AR environment is that the 3D virtual object must be placed at a specific location and can be easily changed by multiple users. Because marker-based registration techniques, which are widely used for matching virtual objects with real ones, require a marker for each object, and creating a crowd simulation environment with objects of various shapes and sizes requires the use of a large number of markers, which is time consuming and expensive. CoR-SketchAR, instead, creates an urban AR environment by drawing sketches with easily altered dry-erase marker. Then, system recognizes the shapes and colors of the sketches automatically. Those recognized shapes and colors provide the exact positions for overlaying the 3D virtual objects, which are the environment factors, on the real environment in augmented reality manner. We can even specify the path the crowd are moving along with a simple sketch stroke. We apply the computer vision technique to recognize the colors and shapes of sketches. By altering the size, shape and color sketches, the system is able to create a wide variety of dynamic urban environments. To validate the proposed techniques, we built two stand-alone software systems to check the usability of the proposed system (a 2D screen-based environmental authoring tool and a sketch-based environmental authoring tool) and conducted experiments in which two users collaborated with each other to create an environment with a specific authoring tool and then report surveys. In the experiments, users collaborated in pairs to create environmental elements, such as highways, buildings, trees, and the starting and goal positions of crowds. After recognizing them, the system then automatically creates a 3D environment, and crowds are animated accordingly. Based on a user survey, we observed that participants who used sketch-based environmental authoring tools were more active and accessible than those who used 2D screen-based authoring tools. The results of the study show that CoR-SketchAR can be further used to create a dynamical crowd simulation on a large scale using beam projectors or portable devices by simply adding sketches based on different scenarios.