Digital Scholarship in the Humanities · 2018 · 14 citations · 14 references
EducationComputer ScienceNatural Language ProcessingText AnalyticsVocabulary DevelopmentVocabulary AssessmentVocabulary SizeWord Frequency EffectsComparison Of Content WordsFrequency-based Vocabulary AnalysisMeasure-theoretic EntropyShannon EntropyVocabulary FrequencyVocabulary NormalizationVocabulary Size AssessmentVocabulary Size Estimation
A Shannon’s entropy-based measure for assessing a person’s vocabulary strength has been proposed, which has been termed vocabulary quotient. The measure proposed has been made independent of the text size by normalization and determines the degree of uniqueness in the document. It takes into consideration the number of different words used and the frequency of their usage. The vocabulary quotient has been calculated for two cases: one over the original and the other on the text after the removal of function words. The value obtained in the second case serves as a more reasonable quantification, as content words give a more realistic view of a person’s vocabulary. The measure, hence, computed can also be used to determine vocabulary growth of a person over time.
14
Prediction and Entropy of Printed English
Claude E. Shannon · Bell System Technical Journal · 1951 · 2.7K citations
Early vocabulary growth: Relation to language input and gender.
Janellen Huttenlocher, Wendy Haight, Anthony S. Bryk et al. · Developmental Psychology · 1991 · 1.8K citations