PROSE AND POETRY CLASSIFICATION AND BOUNDARY DETECTION USING WORD ADJACENCY NETWORK ANALYSIS

Abstract

Word adjacency networks constructed from written works reflect differences in the structure of prose and poetry. We present a method to disambiguate prose and poetry by analyzing network parameters of word adjacency networks, such as the clustering coefficient, average path length and average degree. We determine the relevant parameters for disambiguation using linear discriminant analysis (LDA) and the effect size criterion. The accuracy of the method is 74.9 ± 2.9% for the training set and 73.7 ± 6.4% for the test set which are greater than the acceptable classifier requirement of 67.3%. This approach is also useful in locating text boundaries within a single article which falls within a window size where the significant change in clustering coefficient is observed. Results indicate that an optimal window size of 75 words can detect the text boundaries.