2022 16th International Conference on Ubiquitous Information Management and Communication (IMCOM) · 2022 · 14 citations · 9 references
ComputingComputer ScienceComputational LinguisticsLegal InterpretationNatural Language ProcessingData SummarizationLanguage ModelingLanguage ModelsLegal LanguageText SummarizationAutoregressive ModelsAutoregressive ProcessesEncoder-decoder ArchitecturesLegal Document SummarizationNeural Language ModelsSummarizationSummarization DatasetsSummarization Of Legal Texts
AI technology in the legal domain has developed at a rapid pace around the world, but not much research is being conducted in the Korean legal field due to barriers of language and the high level of expertise required. We first attempt abstractive summarization of Korean legal decision text and publicly release our collected dataset. We utilize two pretrained language models, i.e., BERT2BERT and BART, for our task. They are based on the encoder-decoder approach under transformer architecture. While BERT2BERT is pre-trained with BERT on both the encoder and decoder, BART combines BERT and GPT as the encoder and the decoder. We then evaluate the baseline models and show that, despite the difference in language style, the high-quality summary was generated using applied models. We also show that pre-training using both autoencoder and autoregressive method makes better performance than using solely denoising autoencoder.
9
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
ROUGE: A Package for Automatic Evaluation of Summaries
Chin-Yew Lin · 2004 · 8.3K citations
Gaussian Error Linear Units (GELUs)
Dan Hendrycks, Kevin Gimpel · arXiv (Cornell University) · 2016 · 3.2K citations · Full text