Fiber Bundle Image Reconstruction Using Convolutional Neural Networks and Bundle Rotation in Endomicroscopy

Abstract

Fiber-bundle endomicroscopy has several recognized drawbacks, the most prominent being the honeycomb effect. We developed a multi-frame super-resolution algorithm exploiting bundle rotation to extract features and reconstruct underlying tissue. Simulated data was used with rotated fiber-bundle masks to create multi-frame stacks to train the model. Super-resolved images are numerically analyzed, which demonstrates that the algorithm can restore images with high quality. The mean structural similarity index measurement (SSIM) improved by a factor of 1.97 compared with linear interpolation. The model was trained using images taken from a single prostate slide, 1343 images were used for training, 336 for validation, and 420 for testing. The model had no prior information about the test images, adding to the robustness of the system. Image reconstruction was completed in 0.03 s for 256 × 256 images indicating future real-time performance is within reach. The combination of fiber bundle rotation and multi-frame image enhancement through machine learning has not been utilized before in an experimental setting but could provide a much-needed improvement to image resolution in practice.