Real-Time Evaluation of E-mail Campaign Performance

TLDR

The authors develop a real‑time testing methodology using a split‑hazard model with virtual time to predict email campaign performance via open and click propensities. The methodology employs a split‑hazard model with virtual time, applies it to 25 campaigns, and integrates a decision‑theoretic framework to generate sequential real‑time evaluation procedures. The approach yields accurate, reliable estimates within 1 h 15 min—better than the 14‑hour doubling‑time method—provides time‑of‑day‑independent testing, confidence intervals, and enables live monitoring.

Abstract

We develop a testing methodology that can be used to predict the performance of e-mail marketing campaigns in real time. We propose a split-hazard model that makes use of a time transformation (a concept we call virtual time) to allow for the estimation of straightforward parametric hazard functions and generate early predictions of an individual campaign's performance (as measured by open and click propensities). We apply this pretesting methodology to 25 e-mail campaigns and find that the method is able to produce in an hour and fifteen minutes estimates that are more accurate and more reliable than those that the traditional method (doubling time) produces after 14 hours. Other benefits of our method are that we make testing independent of the time of day and we produce meaningful confidence intervals. Thus, our methodology can be used not only for testing purposes, but also for live monitoring. The testing procedure is coupled with a formal decision theoretic framework to generate a sequential testing procedure useful for the real time evaluation of campaigns.