IsoVEM: Isotropic Reconstruction for Volume Electron Microscopy Based on Transformer

Abstract

AbstractVolume electron microscopy (vEM) has become a rapidly developing technique for studying the 3D architecture of biological specimens, such as cells, tissues, and organs at nanometer resolution; this technique involves collecting a series of electron micrographs of axial sequential sections and reconstructing the 3D volume, providing useful information on the cellular ultrastructural spectrum. This technique currently suffers from anisotropic resolution between the lateral (x, y) and axial (z) directions and the loss/damage of sections. Here, we develop a new algorithm, IsoVEM, based on a video transformer model to boost the axial resolution and achieve isotropic reconstruction of vEM. By learning high-resolution axial structures and utilizing the 3D continuity of biological structures, IsoVEM can recover axial information and repair random lost/damaged sections based on a self-supervision strategy, achieving a higher resolution than existing methods, which has been validated for both simulated FIB-SEM datasets and experimental ssTEM datasets. In addition to visual validation, the segmentation efficiency and statistical precision of various ultrastructures, e.g., neurons, mitochondria, vesicles, and membrane bilayers, also prove the better performance of IsoVEM. Therefore, using IsoVEM, we achieve isotropic reconstruction via anisotropic axial sampling, which increases the vEM throughput for studying large-scale biological architectures.