Overestimating or underestimating communication findings? Comparing self-reported with log mobile data by data donation method

Abstract

Mobile phone usage is typically measured via self-reporting. However, scholars have questioned the validity of self-reported data, which may lead to Type I or Type II errors. Using an online survey ( n = 777), this study compared self-reported and log mobile phone usage data using a simplified version of the mobile data donation method. The results showed that people generally underreported their mobile phone usage in terms of time duration, the number of notifications, and apps used. Moreover, self-reported data may either have no additional effect on or overestimate the communication findings depending on the outcome variables. This challenges the Type II error explanation and suggests that the effect sizes of self-reported data might not be underestimated after all. Instead, past research examining mobile use and pertinent outcomes may have false-positive findings and Type I errors. Given the potential inaccuracies of self-reported data, future research on mobile media and communications should go beyond self-reported data to enhance the validity of findings.