Adaptive Experimental Design and Counterfactual Inference
Machine Learning
2022-10-27 v1 Methodology
Abstract
Adaptive experimental design methods are increasingly being used in industry as a tool to boost testing throughput or reduce experimentation cost relative to traditional A/B/N testing methods. This paper shares lessons learned regarding the challenges and pitfalls of naively using adaptive experimentation systems in industrial settings where non-stationarity is prevalent, while also providing perspectives on the proper objectives and system specifications in these settings. We developed an adaptive experimental design framework for counterfactual inference based on these experiences, and tested it in a commercial environment.
Cite
@article{arxiv.2210.14369,
title = {Adaptive Experimental Design and Counterfactual Inference},
author = {Tanner Fiez and Sergio Gamez and Arick Chen and Houssam Nassif and Lalit Jain},
journal= {arXiv preprint arXiv:2210.14369},
year = {2022}
}
Comments
In Workshops of the Conference on Recommender Systems (RecSys), 2022