Correction of image distortion in large-field ssEM stitching by an unsupervised intermediate-space solving network

Abstract

Abstract Motivation Serial-section electron microscopy (ssEM) is a powerful technique for cellular visualization, especially for large-scale specimens. Limited by the field of view, a megapixel image of whole-specimen is regularly captured by stitching several overlapping images. However, suffering from distortion by manual operations, lens distortion or electron impact, simple rigid transformations are not adequate for perfect mosaic generation. Non-linear deformation usually causes ‘ghosting’ phenomenon, especially with high magnification. To date, existing microscope image processing tools provide mature rigid stitching methods but have no idea with local distortion correction. Results In this article, following the development of unsupervised deep learning, we present a multi-scale network to predict the dense deformation fields of image pairs in ssEM and blend these images into a clear and seamless montage. The model is composed of two pyramidal backbones, sharing parameters and interacting with a set of registration modules, in which the pyramidal architecture could effectively capture large deformation according to multi-scale decomposition. A novel ‘intermediate-space solving’ paradigm is adopted in our model to treat inputted images equally and ensure nearly perfect stitching of the overlapping regions. Combining with the existing rigid transformation method, our model further improves the accuracy of sequential image stitching. Extensive experimental results well demonstrate the superiority of our method over the other traditional methods. Availability and implementation The code is available at https://github.com/HeracleBT/ssEM_stitching. Supplementary information Supplementary data are available at Bioinformatics online.