Deep reinforcement learning-based digital twin for droplet microfluidics control

Abstract

This study applied deep reinforcement learning (DRL) with the Proximal Policy Optimization (PPO) algorithm within a two-dimensional computational fluid dynamics (CFD) model to achieve closed-loop control in microfluidics. The objective was to achieve the desired droplet size with minimal variability in a microfluidic capillary flow-focusing device. An artificial neural network was utilized to map sensing signals (flow pressure and droplet size) to control actions (continuous phase inlet pressure). To validate the numerical model, simulation results were compared with experimental data, which demonstrated a good agreement with errors below 11%. The PPO algorithm effectively controlled droplet size across various targets (50, 60, 70, and 80 μm) with different levels of precision. The optimized DRL + CFD framework successfully achieved droplet size control within a coefficient of variation (CV%) below 5% for all targets, outperforming the case without control. Furthermore, the adaptability of the PPO agent to external disturbances was extensively evaluated. By subjecting the system to sinusoidal mechanical vibrations with frequencies ranging from 10 Hz to 10 KHz and amplitudes between 50 and 500 Pa, the PPO algorithm demonstrated efficacy in handling disturbances within limits, highlighting its robustness. Overall, this study showcased the implementation of the DRL+CFD framework for designing and investigating novel control algorithms, advancing the field of droplet microfluidics control research.