Convolutional Neural Network-Aided DP-64 QAM Coherent Optical Communication Systems

Abstract

Optical nonlinearity impairments have been a major obstacle for high-speed, long-haul and large-capacity optical transmission. In this paper, we propose a novel convolutional neural network (CNN)-based perturbative nonlinearity compensation approach in which we reconstruct a feature map with two channels that rely on first-order perturbation theory and build a classifier and a regressor as a nonlinear equalizer. We experimentally demonstrate the CNN equalizer in 375 km 120-Gbit/s dual-polarization 64-quadrature-amplitude modulation (64-QAM) coherent optical communication systems. We studied the influence of the dropout value and nonlinear activation function on the convergence of the CNN equalizer. We measured the bit-error-ratio (BER) performance with different launched optical powers. When the channel size is 11, the optimum BER for the CNN classifier is 0.0012 with 1 dBm, and for the CNN regressor, it is 0.0020 with 0 dBm; the BER can be lower than the 7 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> hard decision-forward threshold of 0.0038 from −3 dBm to 3 dBm. When the channel size is 15, the BERs at −4 dBm, 4 dBm and 5 dBm can be lower than 0.0020. The network complexity is also analyzed in this paper. Compared with perturbative nonlinearity compensation using a fully connected neural network (2392-64-64), we can verify that the time complexity is reduced by about 25 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> , while the space complexity is reduced by about 50 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> .