Japanese abstractive text summarization using BERT

Abstract

In this study, we developed an automatic abstractive text summarization algorithm in Japanese using a neural network. We used a sequence-to-sequence encoder-decoder model for experimentation purposes. The encoder obtained a feature-based input vector of sentences using BERT. A transformer-based decoder returned the summary sentence from the output as generated by the encoder. This experiment was performed using the livedoor news corpus with the above model. However, issues arose as the same texts were repeated in the summary sentence.