Multi-Agent Reinforcement Learning for Stand Structure Collaborative Optimization of Pinus yunnanensis Secondary Forests

Abstract

This study aims to investigate the potential and advantages of multi-agent reinforcement learning (MARL) in forest management, offering innovative insights and methodologies for achieving sustainable management of forest ecosystems. Focusing on the Pinus yunnanensis secondary forests in Southwest China, we formulated the objective function and constraints based on both spatial and non-spatial structural indices of the forest stand structure (FSS). The value of the objective function (VOF) served as an indicator for assessing FSS. Leveraging the random selection method (RSM) to select harvested trees, we propose the replanting foreground index (RFI) to enhance replanting optimization. The decision-making processes involved in selection harvest optimization and replanting were modeled as actions within MARL. Through iterative trial-and-error and collaborative strategies, MARL optimized agent actions and collaboration to address the collaborative optimization problem of FSS. We conducted optimization experiments for selection felling and replanting across four circular sample plots, comparing MARL with traditional combinatorial optimization (TCO) and single-agent reinforcement learning (SARL). The findings illustrate the superior practical efficacy of MARL in collaborative optimization of FSS. Specifically, replanting optimization based on RFI outperformed the classical maximum Delaunay generator area method (MDGAM). Across different plots (P1, P2, P3, and P4), MARL consistently improved the maximum VOFs by 54.87%, 88.86%, 41.34%, and 22.55%, respectively, surpassing those of the TCO (38.81%, 70.04%, 41.23%, and 18.73%) and SARL (54.38%, 70.04%, 41.23%, and 18.73%) schemes. The RFI demonstrated superior performance in replanting optimization experiments, emphasizing the importance of considering neighboring trees’ influence on growth space and replanting potential. Following selective logging and replanting adjustments, the FSS of each sample site exhibited varying degrees of improvement. MARL consistently achieved maximum VOFs across different sites, underscoring its superior performance in collaborative optimization of logging and replanting within FSS. This study presents a novel approach to optimizing FSS, contributing to the sustainable management of Pinus yunnanensis secondary forests in southwestern China.