Impact of non-Gaussian noise on GMI and LDPC performance in neural network equalized systems

Abstract

In this Letter, the impact of non-Gaussian noise caused by a nonlinear equalizer on low-density parity-check code (LDPC) performance is investigated in a 25-km 50-Gb/s pulse amplitude modulation4 (PAM4) direct detection system. The lookup table (LUT)-based log-likelihood ratio (LLR) calculation method is proposed to enhance the LDPC performance for the non-Gaussian noise case. Compared to the conventional LLR calculation method based on Gaussian distribution, the proposed method can improve 0.6-dB sensitivity in artificial neural network (ANN) equalizer systems. In addition, the conventional generalized mutual information (GMI) is proven to be an imperfect predictor of LDPC performance after nonlinear equalizers, such as decision feedback equalization (DFE) and ANN equalizer.