Readers’ perception of computer-generated news: Credibility, expertise, and readability

TLDR

The study examined how people perceive computer-generated news. The experiment used a 2×2×2 design with 986 participants rating pairs of articles on credibility, readability, and journalistic expertise, manipulating topic, actual source, and declared source. Results showed that declaring an article as human-written slightly increased favorability, while computer-written articles were judged more credible and expert but less readable, with no topic differences, suggesting that computer-generated news is already moderately favored and likely to become more so, underscoring the need for ethical guidelines.

Abstract

We conducted an online experiment to study people’s perception of automated computer-written news. Using a 2 × 2 × 2 design, we varied the article topic (sports, finance; within-subjects) and both the articles’ actual and declared source (human-written, computer-written; between-subjects). Nine hundred eighty-six subjects rated two articles on credibility, readability, and journalistic expertise. Varying the declared source had small but consistent effects: subjects rated articles declared as human written always more favorably, regardless of the actual source. Varying the actual source had larger effects: subjects rated computer-written articles as more credible and higher in journalistic expertise but less readable. Across topics, subjects’ perceptions did not differ. The results provide conservative estimates for the favorability of computer-written news, which will further increase over time and endorse prior calls for establishing ethics of computer-written news.