Style Transfer in Text: Exploration and Evaluation

TLDR

Style transfer of text is a key AI benchmark, yet progress lags behind vision due to scarce parallel data and unreliable evaluation metrics. The study aims to learn style transfer from non‑parallel data and introduce two new metrics—transfer strength and content preservation—to evaluate it. Two adversarial‑network models that learn separate content and style representations were proposed and benchmarked on paper‑news title and positive‑negative review transfer tasks. The content‑preservation metric correlates strongly with human judgments, and the models achieve higher style‑transfer strength while maintaining comparable content preservation relative to an auto‑encoder baseline.

Abstract

The ability to transfer styles of texts or images, is an important measurement of the advancement of artificial intelligence (AI). However, the progress in language style transfer is lagged behind other domains, such as computer vision, mainly because of the lack of parallel data and reliable evaluation metrics. In response to the challenge of lacking parallel data, we explore learning style transfer from non-parallel data. We propose two models to achieve this goal. The key idea behind the proposed models is to learn separate content representations and style representations using adversarial networks. Considering the problem of lacking principle evaluation metrics, we propose two novel evaluation metrics that measure two aspects of style transfer: transfer strength and content preservation. We benchmark our models and the evaluation metrics on two style transfer tasks: paper-news title transfer, and positive-negative review transfer. Results show that the proposed content preservation metric is highly correlate to human judgments, and the proposed models are able to generate sentences with similar content preservation score but higher style transfer strength comparing to auto-encoder.