Mine Your Own Business: Market-Structure Surveillance Through Text Mining

TLDR

Web 2.0 platforms generate vast amounts of user‑generated content that could reveal consumer insights, yet extracting market‑structure information from these posts is challenging. The study proposes a method to transform user‑generated content into market‑structure and competitive‑landscape insights. Using text mining combined with semantic network analysis, the authors build perceptual maps and compare them with traditional sales‑based structures to validate the approach. The method produced perceptual maps for sedan cars and diabetes drugs, revealing meaningful insights and differences from conventional data without any consumer interviews.

Abstract

Web 2.0 provides gathering places for Internet users in blogs, forums, and chat rooms. These gathering places leave footprints in the form of colossal amounts of data regarding consumers' thoughts, beliefs, experiences, and even interactions. In this paper, we propose an approach for firms to explore online user-generated content and “listen” to what customers write about their and their competitors' products. Our objective is to convert the user-generated content to market structures and competitive landscape insights. The difficulty in obtaining such market-structure insights from online user-generated content is that consumers' postings are often not easy to syndicate. To address these issues, we employ a text-mining approach and combine it with semantic network analysis tools. We demonstrate this approach using two cases—sedan cars and diabetes drugs—generating market-structure perceptual maps and meaningful insights without interviewing a single consumer. We compare a market structure based on user-generated content data with a market structure derived from more traditional sales and survey-based data to establish validity and highlight meaningful differences.