Cycle-consistent Generative Adversarial Network for computational hematoxylin-and-eosin staining of fluorescence confocal microscopic images of basal cell carcinoma tissue

Abstract

Abstract Background Histopathology based on Hematoxylin-and-Eosin (H&E) staining is the gold standard for basal cell carcinoma (BCC) diagnosis but requires lengthy and laborious tissue preparation. Fluorescence confocal microscopy (FCM) enables fluorescence detection and high-resolution imaging in less time and with minimal tissue preparation. This work proposes a deep learning model for the computational staining of FCM images based on an unsupervised approach, which correlates color with H&E images. Methods In this study, we investigated the utility of FCM on BCC tissue sections stained with Acridine Orange (AO) and computationally colored to simulate H&E dyes. We adopted Cycle-consistent Generative Adversarial Network (CycleGAN), an unsupervised framework, for computational staining. The dataset consists of unpaired and unannotated thousands of FCM and H&E patches from whole slide images of BCC tissue sections. CycleGAN includes two forward and backward GANs that are coupled together and complete a cycle to ensure a reliable transformation between the two domains. In addition to the adversarial and cycle consistency constraints, a saliency constraint was employed to enhance the correct mapping direction between FCM and H&E images and avoid appearance distortions of the image content. Results The generated H&E-like images from FCM through this CycleGAN model were visually and quantitatively similar to real H&E images. Computationally stained images had skin tissue characteristics. The trained models showed a high performance in the structure preservation of the original image when comparing the source and reconstructed images. Quantitative metrics (by Fréchet Inception Distance) showed that CycleGAN could improve the similarity of training images by up to 63% after mapping images from a source domain to a target domain. Conclusion FCM could expand the application and usefulness of rapid tissue analysis. Thus, using the CycleGAN model for computational staining is beneficial for diagnostic applications while simplifying laboratory staining procedures. The proposed approach has significant potential in clinical computational staining and advanced computer-aided histology image analysis.