Abstract
Fluorescence in situ hybridization (FISH) is a molecular cytogenetic technique. It provides a powerful tool for understanding genetic and genomic processes, diagnosing genetic disorders, and studying the structure and function of genes and chromosomes. This paper proposes a method for automatic object segmentation of preparations of blood cell nuclei and a method for detecting chromosomes with the aim of further studying them for chromosomal mosaicism. Based on the data provided by the laboratory of the Institute of Biology and Biomedicine of Lobachevsky University, the SOTA deep learning model YOLOv8-seg was trained. This was made possible by marking up a small portion of the 87 images. Experiment on model training for segmentation showed very good quality metrics: Precision = 0.940, Recall = 0.980, mAP[0.5] = 0.991 and mAP[0.5:0.95] = 0.764. After that, a method for detecting chromosomes was proposed, based on the classical principles of image processing and computer vision, due to the lack of the necessary labelled data.
Publisher
Keldysh Institute of Applied Mathematics