On the statistics of fixed-point roundoff error

Abstract

Roundoff error after fixed-point multiplication is commonly modeled as uniformly distributed white noise that is uncorrelated with the signal. This paper presents a statistical analysis of fixed-point roundoff error that identifies the conditions under which this model is valid, and examines the statistical behavior of roundoff error when these conditions are not satisfied. The results show that if the multiplier coefficient is expressed as a = N/M, where M is a positive integral power of two and N is an odd integer, then the errors generated by roundoff after multiplication can generally be modeled as uniformly distributed, white, and uncorrelated with the signal, if the signal has sufficiently wide bandwidth and has a dynamic range that extends over approximately M quantum steps. For narrow-band low-level signals, the roundoff error statistics can differ significantly from the uniform, white, uncorrelated model. In addition, these results show that statistical behavior of roundoff error can differ significantly from that of the quantization error that is generated when a continuous random variable is quantized.