The Quantitative Research Methods Workshop presents Mohsen Bayati, Associate Professor of Operations, Information & Technology, Stanford Graduate School of Business: “Optimal Experimental Design for Staggered Rollouts”.
Abstract: Experimentation has become an increasingly prevalent practice for optimizing the development of data products or making policy decisions. A common hurdle in designing experiments is lack of statistical power, especially when the sample size is limited or the noise level is high. In this paper, we study the optimal multi-period experimental design, to maximize statistical power, under the constraint that the treatment cannot be easily removed once implemented; for example, a government might implement a public health intervention in different geographies at different times, where the treatment cannot be easily removed due to practical constraints. The treatment design problem is to select which geographies to treat at which time, intending to test hypotheses about the effect of the treatment. Even though solving the optimal design is, in general, an NP-hard problem, we provide a set of simple and interpretable sufficient conditions for the optimal solution. When the potential outcome has two-way fixed effects and possibly has (observed or latent) covariates, we can obtain an analytical solution that has a staggered treatment adoption pattern. Furthermore, we propose a local search algorithm that actively improves this analytical solution, using historical control data. Via synthetic experiments on real data sets, we show that our analytical solution can significantly and consistently outperform benchmark designs. LINK TO PAPER (with R. Xiong, S. Athey, and G. Imbens)
Mohsen Bayati received a B.Sc. degree in Mathematics from Sharif University of Technology in 2000, and an M.Sc. degree in Mathematics, and Ph.D. in Electrical Engineering from Stanford University in 2007. He was a Postdoctoral Researcher at Microsoft Research and at Stanford University during 2007-2011, and since 2011, he has been a faculty in the Operations, Information, and Technology group at Stanford University Graduate School of Business. His research interests include graphical models, high-dimensional statistics, personalized decision-making, and healthcare management. He was awarded the INFORMS Healthcare Applications Society best paper (Pierskalla) award in 2014 and in 2016, INFORMS Applied Probability Society best paper award in 2015, and National Science Foundation CAREER award in 2016.