A comparative study of plant phenotyping workflows based on three-dimensional reconstruction from multi-view images

Abstract

AbstractWith the world facing escalating food demand, limited agricultural land, and rapid environmental change, there is a growing need for data-driven sustainable agricultural management approaches. The proliferation of next-generation sequencers and sensor networks has reduced the cost of acquiring genomic and environmental data, respectively. However, collecting phenotypic data, which is crucial for monitoring plant growth trajectories and detecting pests and diseases, continues to be a labor-intensive endeavor. Technological advances have enabled an efficient collection of three-dimensional (3D) data, yet this process currently involves a set of intricate steps. Therefore, the development of an effective phenotyping method is essential. In this study, we developed a phenotyping process based on 3D reconstruction, including mask image generation using deep neural network models, 3D reconstruction using the Structure from Motion/Multi-View Stereo (SfM/MVS) pipeline, and leaf surface reconstruction for leaf area estimation. Our investigation using soybean datasets into the optimal magnification for input images in mask generation models revealed that a 1/5.4× magnification was most effective for segmenting thin structures. Using mask images in the SfM/MVS pipeline limited the region of interest, and this could decrease the processing time and improve the quality of the point-cloud data. We assessed four scenarios regarding mask image usage and found that the scenario that set the mask images of soybeans and stages before SfM and soybeans only after SfM yielded the highest-quality point-cloud data and was the second fastest in processing. Finally, we compared the Poisson reconstruction and B-spline surface fitting for leaf surface reconstruction from point clouds. B-spline fitting shows a greater correlation with destructive measurements. We proposed an optimal workflow for estimating leaf area based on these results. Additionally, to contribute to the development of plant phenotyping methods, we provided a web application for mask generation, a command-line tool for leaf surface reconstruction, and validation datasets.