LightCodec: A High Fidelity Neural Audio Codec with Low Computation Complexity

Abstract

The audio codec is one of the core modules in audio communication for real-time transmission. With the development of neural networks, end-to-end audio codecs have emerged and demonstrated effects beyond conventional codecs. However, current neural network-based codecs have the weakness of high computational complexity, and the performance of these methods decreases rapidly after decreasing the complexity, which is not conducive to deployment under low computational resources. In this paper, a low-complexity audio codec is proposed. To realize the low complexity of the model with high quality, a structure based on frequency band division is designed, which is implemented using a within bandacross band interaction (WBABI) module to learn the features across and within the subband. Further, we propose a new quantization-compensation module, which reduces the quantization error by 90%. The experimental results show that for audio with a sample rate of 24kHz, the model shows excellent performance at 3~6kbps compared to other codecs, and the complexity is only 0.8 Giga Multiply-Add Operations per Second(GMACs).