Lobachevskii Journal of Mathematics · 2024 · 11 citations · 29 references
ComputingLinguisticsComputational LinguisticsNatural Language ProcessingText AnalyticsLanguage ModelingLanguage ModelsNamed Entity RecognitionOpinion MiningRussian LanguageSentiment AnalysisDecoding StrategiesLarge Language ModelsTargeted Sentiment AnalysisTextual Sentiment AnalysisTransformer Language ModelsZero-shot Language ModelingZero-shot Named Entity Recognition
In this paper, we investigate the use of decoder-based generative transformers for extracting sentiment towards the named entities in Russian news articles. We study sentiment analysis capabilities of instruction-tuned large language models (LLMs). We consider the dataset of RuSentNE-2023 in our study. The first group of experiments was aimed at the evaluation of zero-shot capabilities of LLMs with closed and open transparencies. The second covers the fine-tuning of Flan-T5 using the ‘‘chain-of-thought’’ (CoT) three-hop reasoning framework (THoR). We found that the results of the zero-shot approaches are similar to the results achieved by baseline fine-tuned encoder-based transformers (BERT $${}_{\text{base}}$$ ). Reasoning capabilities of the fine-tuned Flan-T5 models with THoR achieve at least $$5\%$$ increment with the base-size model compared to the results of the zero-shot experiment. The best results of sentiment analysis on RuSentNE-2023 were achieved by fine-tuned Flan-T5 $${}_{\text{xl}}$$ , which surpassed the results of previous state-of-the-art transformer-based classifiers. Our CoT application framework is publicly available: https://github.com/nicolay-r/Reasoning-for-Sentiment-Analysis-Framework
29
Decoupled Weight Decay Regularization
Ilya Loshchilov, Frank Hutter · arXiv (Cornell University) · 2017 · 9.1K citations · Full text
Transformers: State-of-the-Art Natural Language Processing
Thomas Wolf, Lysandre Debut, Victor Sanh et al. · 2020 · 8K citations · Full text
LLaMA: Open and Efficient Foundation Language Models
Hugo Touvron, Thibaut Lavril, Gautier Izacard et al. · arXiv (Cornell University) · 2023 · 3.9K citations · Full text