A machine learning approach for cross-domain plant identification using herbarium specimens

Abstract

AbstractThe preservation of plant specimens in herbaria has been carried out for centuries in efforts to study and confirm plant taxa. With the increasing collection of herbaria made available digitally, it is practical to use herbarium specimens for the automation of plant identification. They are also substantially more accessible and less expensive to obtain compared to field images. In fact, in remote and inaccessible habitats, field images of rare plant species are still immensely lacking. As a result, rare plant species identification is challenging due to the deficiency of training data. To address this problem, we investigate a cross-domain adaptation approach that allows knowledge transfer from a model learned from herbarium specimens to field images. We propose a model called Herbarium–Field Triplet Loss Network (HFTL network) to learn the mapping between herbarium and field domains. Specifically, the model is trained to maximize the embedding distance of different plant species and minimize the embedding distance of the same plant species given herbarium–field pairs. This paper presents the implementation and performance of the HFTL network to assess the herbarium–field similarity of plants. It corresponds to the cross-domain plant identification challenge in PlantCLEF 2020 and PlantCLEF 2021. Despite the lack of field images, our results show that the network can generalize and identify rare species. Our proposed HFTL network achieved a mean reciprocal rank score of 0.108 and 0.158 on the test set related to the species with few training field photographs in PlantCLEF 2020 and PlantCLEF 2021, respectively.