A Sample Size Calculation for Training and Certifying Targeting Policies (Revision 1 )

The authors propose an approach for determining the sample size required when using an experiment to train and certify a targeting policy. Calculating the rate at which the performance of a targeting model improves with additional training data is a complex problem. They address this challenge by assuming that customers are grouped into segments that capture relevant information about their responsiveness to the firm’s marketing actions. The authors consider two problem formulations. The first formulation identifies the sample size required to train a targeting policy and certify that its expected performance exceeds a predefined threshold. The second formulation identifies the sample size required to train a targeting policy and certify that it outperforms a baseline in an out-of-sample statistical test. The authors establish theoretical properties of these problems, based on which they propose computationally efficient algorithms for optimal sample size calculations. The authors illustrate their algorithms and analysis using data from a luxury fashion retailer.