2023 · 18 citations · 14 references
ComputingComputer ScienceComputational LinguisticsNatural Language ProcessingSocial Media AnalyticsHate SpeechLanguage ModelingLanguage ModelsSpeech RecognitionAutomated Hate Speech DetectionBenchmark Dataset EvaluationCross-dataset ValidationData DistributionLarge Language ModelsMetricsRecall MetricsZero-shot Classification
This study evaluates OpenAI’s ChatGPT, a large language model, for its efficacy in detecting hate speech in Portuguese tweets, comparing it with purpose-trained models. Despite incurring considerable computational costs, ChatGPT as a zero-shot classifier demonstrated commendable performance, even superior to or on par with state-of-the-art methods, with an F1-score of 73.0% on the ToLD-BR. In a cross-dataset evaluation on the HLPHSP dataset, it secured a superior F1-score of 73%. The choice of prompt significantly impacts the outcome, with a wider scope prompt balancing precision and recall metrics. ChatGPT, due to its interpretability and resilience against data distribution shifts, could be a preferred choice for tasks prioritizing these factors.
14
LLaMA: Open and Efficient Foundation Language Models
Hugo Touvron, Thibaut Lavril, Gautier Izacard et al. · arXiv (Cornell University) · 2023 · 3.9K citations · Full text