Segmentation of nucleus based on dynamic convolution and deep features of stain distribution

Abstract

AbstractAnalysis of the pathology image is important to diagnose cancer of lung, breast and stomach. Segmenting the nucleus is a key step for quantitative analysis, and has significance to the pathology researches and computer aided diagnosis systems. The inconsistency of colour, fuzzy boundary of nucleus and overlapping of cells are the universally acknowledged challenges. To solve these problems, the difference between the inside and outside of nucleus is enhanced by obtaining the distribution of the haematoxylin based on Lambert–Beer's law and the optical characteristics of stains. An inferior encoder, which is supervised by the inferior decoder, is proposed to extract the deep features of the distribution of stains. And these features are fed into the primary encoder to improve the accuracy of segmentation. To relieve the problem that some nuclei are segmented as background because the deep feature is inapparent, dynamic convolution is introduced into the encoders. The experiments show that the proposed model can segment the nucleus in the pathological images more precisely than the compared models. The Dice similarity coefficient (DSC) and panoptic quality (PQ) are 0.810 and 0.512, respectively.