arXiv (Cornell University) · 2022 · 11 citations · 0 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingText AnalyticsLanguage ModelingLanguage ModelsNatural Language UnderstandingQuestion AnsweringLow-resource Language ProcessingResource Consumption Modeling
Recent NLP advances rely on scaling model size and data, but this increases resource consumption—data, time, storage, and energy—which are limited and unevenly distributed. The survey aims to guide resource‑constrained NLP research and highlight promising directions for developing more efficient methods. The authors review and relate existing efficient NLP techniques, categorizing them by resource focus and application.
Abstract Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources include data, time, storage, or energy, all of which are naturally limited and unevenly distributed. This motivates research into efficient methods that require fewer resources to achieve similar results. This survey synthesizes and relates current methods and findings in efficient NLP. We aim to provide both guidance for conducting NLP under limited resources, and point towards promising research directions for developing more efficient methods.