[Paper] Measurement of advertisement effect based on multimodal emotional responses considering personality

Abstract

The market size of online video advertising is expanding rapidly along with the spread of smartphones and social media. In this study, we estimate advertising effectiveness in the natural environment using online data collection and the remote measurement of webcam facial expressions and physiological responses. We collected 4, 108 videos of the faces of 411 Japanese people who were watching the video advertisement in their natural environment via the Internet. Facial expression and physiological responses such as heart rate and gaze were remotely measured by analyzing facial videos. We found that the accuracies of ad liking and purchase intent prediction are better when various acquired features are combined and machine learning is used than when only single-mode features are used. In addition, we aim to improve prediction accuracy by clustering the personality of the subjects and designing an estimation model for each personality.