Biased sampling of early users and the direction of startup innovation

Abstract

Using data from a prominent online platform for launching new digital products, we document that the composition of the platform's `beta testers' on the day a new product is launched has a systematic and persistent impact on success. Specifically, we use word embedding methods to classify products launched on this platform as more or less focused on the needs of female customers, and show that female-focused products launched on a typical day—when nine-in-ten users on the platform are men—experience 40% less growth and are 5 percentage points less likely to have an any users a year after launch. Using exogenous variation driven by the platform's daily newsletter, we find that that the product gender gap shrinks on days when women are more likely to engage with the platform. Conversely, entrepreneurs who happen to launch a female-focused product on an especially male-dominated day reduce their product development efforts by roughly 30% and are 4 percentage points less likely to raise venture funding. Overall, our findings suggest that sample bias can systematically corrupt signals of a startup's market potential, bias entrepreneurial strategy, and so lead to a dearth of innovations aimed at consumers who are underrepresented among early-users.