IEDA Webinar - Adaptive A/B Tests and Simultaneous Treatment Parameter Optimization

Constructing asymptotically valid confidence intervals through a valid central limit theorem (CLT) is crucial for controlled experiments or the so-called A/B tests. In particular, establishing a valid CLT can help statistically assert whether a treatment plan (B) is significantly better than a control plan (A) when the sample size is moderately large. That said, in some emerging applications from online platforms, the treatment plan is not a single plan, but instead encompasses an infinite continuum of plans indexed by a continuous treatment parameter. As such, the experimenter has two tasks: (1) provide valid statistical inference, and (2) simultaneously find the optimal choice of value for the treatment parameter to use for the treatment plan. In this presentation, we discuss some theoretical challenges that arise in jointly delivering these two tasks. We then discuss some partial solutions through a new algorithm and the associated analysis.

This work is joint with Yuhang Wu, Guangyu Zhang, Zuohua Zhang, and Chu Wang.