Evaluating the Validity of Simplified Chinese Version of LIWC in Detecting Psychological Expressions in Short Texts on Social Network Services

TLDR

The growing demand for automated analysis of short social‑network texts has increased the need for reliable computerized text‑analysis tools such as LIWC. This study aimed to assess the validity of a Simplified Chinese version of LIWC (SCLIWC) for detecting psychological expressions in Weibo statuses. Researchers coded 60 Weibo statuses and 11 single statuses using human raters and SCLIWC, comparing the two to evaluate SCLIWC’s performance. SCLIWC showed significant correlations with human ratings and high sensitivity for detecting psychological expressions, especially when using status‑count scoring, but it over‑identified target expressions and performed poorly at identifying the meaning of single statuses, providing preliminary evidence of its validity and guidance for efficient use on SNS short texts.

Abstract

The increasing need of automated analyzing web texts especially the short texts on Social Network Services (SNS) brings new demands of computerized text analysis instruments. The psychometric properties are the basis of the extensive use of these instruments such as the Linguistic Inquiry and Word Count (LIWC). For this study, Sina Weibo statuses were analyzed via rater coding and Simplified Chinese version of LIWC (SCLIWC), in order to evaluate the validity of SCLIWC in detecting psychological expressions in Weibo statuses (n = 60) and in identifying the psychological meaning of a single Weibo status (n = 11). Significant correlations between human ratings and SCLIWC scores and the high sensitivities of capturing single statuses with certain expressions identified by raters, proved the validity of SCLIWC in detecting psychological expressions. The results also suggested that, the efficiency of SCLIWC in detecting psychological expressions of SNS short texts could be higher if using status count scoring method, rather than the word count method as the common usage of LIWC. However, SCLIWC may not perform well in identifying the psychological meaning of a single piece of SNS short text because of its over-identification of target expressions. This study provided primary evidence of validity of SCLIWC, as well as the proper way of using it efficiently on SNS short texts.