Combining physics‐based models with deep learning image synthesis and uncertainty in intraoperative cone‐beam CT of the brain

Abstract

AbstractBackgroundImage‐guided neurosurgery requires high localization and registration accuracy to enable effective treatment and avoid complications. However, accurate neuronavigation based on preoperative magnetic resonance (MR) or computed tomography (CT) images is challenged by brain deformation occurring during the surgical intervention.PurposeTo facilitate intraoperative visualization of brain tissues and deformable registration with preoperative images, a 3D deep learning (DL) reconstruction framework (termed DL‐Recon) was proposed for improved intraoperative cone‐beam CT (CBCT) image quality.MethodsThe DL‐Recon framework combines physics‐based models with deep learning CT synthesis and leverages uncertainty information to promote robustness to unseen features. A 3D generative adversarial network (GAN) with a conditional loss function modulated by aleatoric uncertainty was developed for CBCT‐to‐CT synthesis. Epistemic uncertainty of the synthesis model was estimated via Monte Carlo (MC) dropout. Using spatially varying weights derived from epistemic uncertainty, the DL‐Recon image combines the synthetic CT with an artifact‐corrected filtered back‐projection (FBP) reconstruction. In regions of high epistemic uncertainty, DL‐Recon includes greater contribution from the FBP image. Twenty paired real CT and simulated CBCT images of the head were used for network training and validation, and experiments evaluated the performance of DL‐Recon on CBCT images containing simulated and real brain lesions not present in the training data. Performance among learning‐ and physics‐based methods was quantified in terms of structural similarity (SSIM) of the resulting image to diagnostic CT and Dice similarity metric (DSC) in lesion segmentation compared to ground truth. A pilot study was conducted involving seven subjects with CBCT images acquired during neurosurgery to assess the feasibility of DL‐Recon in clinical data.ResultsCBCT images reconstructed via FBP with physics‐based corrections exhibited the usual challenges to soft‐tissue contrast resolution due to image non‐uniformity, noise, and residual artifacts. GAN synthesis improved image uniformity and soft‐tissue visibility but was subject to error in the shape and contrast of simulated lesions that were unseen in training. Incorporation of aleatoric uncertainty in synthesis loss improved estimation of epistemic uncertainty, with variable brain structures and unseen lesions exhibiting higher epistemic uncertainty. The DL‐Recon approach mitigated synthesis errors while maintaining improvement in image quality, yielding 15%–22% increase in SSIM (image appearance compared to diagnostic CT) and up to 25% increase in DSC in lesion segmentation compared to FBP. Clear gains in visual image quality were also observed in real brain lesions and in clinical CBCT images.ConclusionsDL‐Recon leveraged uncertainty estimation to combine the strengths of DL and physics‐based reconstruction and demonstrated substantial improvements in the accuracy and quality of intraoperative CBCT. The improved soft‐tissue contrast resolution could facilitate visualization of brain structures and support deformable registration with preoperative images, further extending the utility of intraoperative CBCT in image‐guided neurosurgery.