Empirical Analysis of Training Strategies of Transformer-Based Japanese Chit-Chat Systems

Abstract

In recent years, several high-performance conversational systems have been proposed based on the Transformer encoder-decoder model. Although previous studies analyzed the effects of the model parameters and the decoding method on subjective dialogue evaluations with overall metrics, it is not analyzed enough how the differences of fine-tuning datasets affect the user's detailed impressions. In addition, the Transformer-based approach has mostly been verified for English, not for such languages as Japanese that have large inter-language distances. In this study, we developed large-scale Transformer-based Japanese dialogue models and Japanese chit-chat datasets and examined their effectiveness. We analyzed the relationships between users' multifaceted impressions and fine-tuning datasets.