Characteristic Features of Rank-Probability Word Distribution in Scientific and Belletristic Literature

Abstract

Abstract In the present work, an optimal function for the approximation of word probability distribution in natural language is tentatively sought. An analysis of Zipf's rank-frequency distribution and its modifications is performed. The original Lavalette's law is modified to get the most accurate approximation of rank-frequency word distribution in natural language. The suggested modification of Lavalette's function has been tested on a broad variety of belletristic and scientific texts. Implication of text features in fitting parameters of the modified Lavalette's distribution function is suggested. The conclusions are made for the general tendencies and the reasons for possible deviations are discussed.