按年被提到
20162016 · 1 次2026
社会、经济与科学研究›统计、金融与复杂系统·arXiv:1602.05549 · stat.AP

Continuous Monitoring of A/B Tests without Pain: Optional Stopping in Bayesian Testing

Alex Deng, Jiannan Lu, Shouyuan Chen · 2016
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2016-06-03 · 最近 2017-09-28
2不同的人
2活跃月份
2讨论串
67被引(OpenAlex)
arXiv 摘要(候选层不写导读)

A/B testing is one of the most successful applications of statistical theory in modern Internet age. One problem of Null Hypothesis Statistical Testing (NHST), the backbone of A/B testing methodology, is that experimenters are not allowed to continuously monitor the result and make decision in real time. Many people see this restriction as a setback against the trend in the technology toward real time data analytics. Recently, Bayesian Hypothesis Testing, which intuitively is more suitable for real time decision making, attracted growing interest as an alternative to NHST. While corrections of NHST for the continuous monitoring setting are well established in the existing literature and known in A/B testing community, the debate over the issue of whether continuous monitoring is a proper practice in Bayesian testing exists among both academic researchers and general practitioners. In this paper, we formally prove the validity of Bayesian testing with continuous monitoring when proper stopping rules are used, and illustrate the theoretical results with concrete simulation illustrations. We point out common bad practices where stopping rules are not proper and also compare our methodology to NHST corrections. General guidelines for researchers and practitioners are also provided.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)

讨论串里没有 ≥100 字的评论,只有提交记录。

被提交为 story · 2 次