Virtual Test Beds for Image-Based Control Simulations Using Blender

Abstract

Process systems engineering research often utilizes virtual testbeds consisting of physicsbased process models. As machine learning and image processing become more relevant sensing frameworks for control, it becomes important to address how process systems engineers can research the development of control and analysis frameworks that utilize images of physical processes. One method for achieving this is to develop experimental systems; another is to use software that integrates the visualization of systems, as well as modeling of the physics, such as three-dimensional graphics software. The prior work in our group analyzed image-based control for the small-scale example of level in a tank and hinted at some of its potential extensions, using Blender as the graphics software and programming the physics of the tank level via the Python programming interface. The present work focuses on exploring more practical applications of image-based control. Specifically, in this work, we first utilize Blender to demonstrate how a process like zinc flotation, where images of the froth can play a key role in assessing the quality of the process, can be modeled in graphics software through the integration of visualization and programming of the process physics. Then, we demonstrate the use of Blender for testing image-based controllers applied to two other processes: (1) control of the stochastic motion of a nanorod as a precursor simulation toward image-based control of colloidal self-assembly using a virtual testbed; and (2) controller updates based on environment recognition to modify the controller behavior in the presence of different levels of sunlight to reduce the impacts of environmental disturbances on the controller performance. Throughout, we discuss both the setup used in Blender for these systems, as well as some of the features when utilizing Blender for such simulations, including highlighting cases where non-physical parameters of the graphics software would need to be assumed or tuned to the needs of a given process for the testbed simulation. These studies highlight benefits and limitations of this framework as a testbed for image-based controllers and discuss how it can be used to derive insights on image-based control functionality without the development of an experimental testbed.