2024 · 15 citations · 11 references
ComputingComputer ScienceComputational LinguisticsMachine LearningNatural Language ProcessingDeep LearningLanguage ModelingLanguage ModelsLarge-scale OptimizationOptimization For Machine LearningAccuracyAdaptive AlgorithmsAdaptive OptimizationComputational EfficiencyEfficiencyLarge Language ModelsLarge-scale Natural Language ProcessingOptimization Algorithms
With the rapid development of natural language processing technology, large-scale language models (LLM) have achieved remarkable results in a variety of tasks. However, how to effectively train these huge models and improve their performance and computational efficiency remains an important challenge. This paper proposes an improved method based on adaptive optimization algorithm, aiming to improve the training efficiency and final performance of LLM. Through comparative experiments on the SQuAD and GLUE data sets, the experimental results show that compared with traditional optimization algorithms (such as SGD, Momentum, AdaGrad, RMSProp and Adam), the adaptive optimization algorithm we proposed has better accuracy and F1 score. Both have achieved significant improvements, especially showed stronger training capabilities when processed large-scale texts and complex tasks. The research results verify the advantages of adaptive optimization algorithms in large-scale language model training and provide new ideas and directions for future optimization methods.
11
A Study of the Optimization Algorithms in Deep Learning
Raniah Zaheer, Humera Shaziya · 2019 · 158 citations
Yan Xu, Yaoting Jiang, Wenyi Liu et al. · 2024 · 57 citations
Computing, Data Science, Data Mining +14
Emotion-Aware Interaction Design in Intelligent User Interface Using Multi-Modal Deep Learning
Shiyu Duan, Ziyi Wang, Shixiao Wang et al. · 2024 · 33 citations
Computing, Computer Science, Design +12