International Journal of Advanced Computer Science and Applications · 2020 · 56 citations · 22 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingText AnalyticsAutomated Essay ScoringLanguage ModelingLanguage ModelsText Feature EngineeringWord EmbeddingsWriting AssessmentAccuracyBenchmark Dataset EvaluationCohen's Kappa InterpretationEmbedding LearningEssay ClassificationKappa Statistic
Automated Essay Scoring (AES) is a difficult NLP task because essay length, spelling errors, and feature representation all affect scoring quality. The study compares AES models built from different combinations of feature sets. We evaluated AES models using 30 manually extracted features, 300‑word Word2Vec, and 768‑word BERT embeddings, treating scoring as rescaled regression and quantized classification, and compared them to existing ensembles via Kappa and accuracy. The best combination achieved 77.2 ± 1.7 Kappa on regression and 75.2 ± 1.0 accuracy on classification on a 12,000‑essay benchmark, showing that combining manual and deep‑encoded features improves AES reliability.
Automated Essay Scoring (AES) is one of the most challenging problems in Natural Language Processing (NLP). The significant challenges include the length of the essay, the presence of spelling mistakes affecting the quality of the essay and representing essay in terms of relevant features for the efficient scoring of essays. In this work, we present a comparative empirical analysis of Automatic Essay Scoring (AES) models based on combinations of various feature sets. We use 30-manually extracted features, 300-word2vec representation, and 768-word embedding features using BERT model and forms different combinations for evaluating the performance of AES models. We formulate an automated essay scoring problem as a rescaled regression problem and quantized classification problem. We analyzed the performance of AES models for different combinations. We compared them against the existing ensemble approaches in terms of Kappa Statistics and Accuracy for rescaled regression problem and quantized classification problem respectively. A combination of 30-manually extracted features, 300-word2vec representation, and 768-word embedding features using BERT model results up to 77.2 ± 1.7 of Kappa statistics for rescaled regression problem and 75.2 ± 1.0 of accuracy value for Quantized Classification problem using a benchmark dataset consisting of about 12,000 essays divided into eight groups. The reporting results provide directions to the researchers in the field to use manually extracted features along with deep encoded features for developing a more reliable AES model.
22
Distributed Representations of Words and Phrases and their Compositionality
Tomáš Mikolov, Ilya Sutskever, Kai Chen et al. · arXiv (Cornell University) · 2013 · 18.1K citations · Full text
Efficient Estimation of Word Representations in Vector Space
Tomáš Mikolov, Kai Chen, Greg S. Corrado et al. · arXiv (Cornell University) · 2013 · 11.7K citations · Full text