IEEE Communications Surveys & Tutorials · 2024 · 172 citations · 104 references
ComputingComputer ScienceInformation TechnologyDigital CommunicationNatural Language ProcessingProgramming LanguagesCommunication ModelsCommunication NetworksCommunication SystemsLanguage ModelingLanguage ModelsRegular LanguagesLanguage Model PretrainingLanguage Model RescoringLarge Language ModelsLarge-scale Natural Language ProcessingRecurrent Neural Language ModelsRecurrent Neural Network Language ModelsTransformer Language Models
Large language models (LLMs) have received considerable attention recently due to their outstanding comprehension and reasoning capabilities, leading to great progress in many fields. The advancement of LLM techniques also offers promising opportunities to automate many tasks in the telecommunication (telecom) field. After pre-training and fine-tuning, LLMs can perform diverse downstream tasks based on human instructions, paving the way to artificial general intelligence (AGI)-enabled 6G. Given the great potential of LLM technologies, this work aims to provide a comprehensive overview of LLM-enabled telecom networks. In particular, we first present LLM fundamentals, including model architecture, pre-training, fine-tuning, inference and utilization, model evaluation, and telecom deployment. Then, we introduce LLM-enabled key techniques and telecom applications in terms of generation, classification, optimization, and prediction problems. Specifically, the LLM-enabled generation applications include telecom domain knowledge, code, and network configuration generation. After that, the LLM-based classification applications involve network security, text, image, and traffic classification problems. Moreover, multiple LLM-enabled optimization techniques are introduced, such as automated reward function design for reinforcement learning and verbal reinforcement learning. Furthermore, for LLM-aided prediction problems, we discussed time-series prediction models and multi-modality prediction problems for telecom. Finally, we highlight the challenges and identify the future directions of LLM-enabled telecom networks.
104
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
SciBERT: A Pretrained Language Model for Scientific Text
Iz Beltagy, Kyle Lo, Arman Cohan · 2019 · 3K citations · Full text