相关论文: The power of A/B testing under interference

We present current methods for estimating treatment effects and spillover effects under "interference", a term which covers a broad class of situations in which a unit's outcome depends not only on treatments received by that unit, but also…

In the era of fast-paced precision medicine, observational studies play a major role in properly evaluating new treatments in clinical practice. Yet, unobserved confounding can significantly compromise causal conclusions drawn from…

Online evaluation of machine learning models is typically conducted through A/B experiments. Sequential statistical tests are valuable tools for analysing these experiments, as they enable researchers to stop data collection early without…

Betweenness centrality (BC) was proposed as an indicator of the extent of an individual's influence in a social network. It is measured by counting how many times a vertex (i.e., an individual) appears on all the shortest paths between…

A/B tests have been widely adopted across industries as the golden rule that guides decision making. However, the long-term true north metrics we ultimately want to drive through A/B test may take a long time to mature. In these situations,…

Experimentation is widely utilized for causal inference and data-driven decision-making across disciplines. In an A/B experiment, for example, an online business randomizes two different treatments (e.g., website designs) to their customers…

Randomized experiments, or A/B tests are used to estimate the causal impact of a feature on the behavior of users by creating two parallel universes in which members are simultaneously assigned to treatment and control. However, in social…

Causal mediation analysis seeks to determine whether an independent variable affects a response variable directly or whether it does so indirectly, by way of a mediator. The existing statistical tests to determine the existence of an…

This study presents a theoretical analysis on the efficiency of interleaving, an efficient online evaluation method for rankings. Although interleaving has already been applied to production systems, the source of its high efficiency has…

In recommender systems, online A/B testing is a crucial method for evaluating the performance of different models. However, conducting online A/B testing often presents significant challenges, including substantial economic costs, user…

Online A/B tests have become increasingly popular and important for social platforms. However, accurately estimating the global average treatment effect (GATE) has proven to be challenging due to network interference, which violates the…

Evaluating the causal effect of recommendations is an important objective because the causal effect on user interactions can directly leads to an increase in sales and user engagement. To select an optimal recommendation model, it is common…

This paper investigates the case of interference, when a unit's treatment also affects other units' outcome. When interference is at work, policy evaluation mostly relies on the use of randomized experiments under cluster interference and…

Increasingly, there is a marked interest in estimating causal effects under network interference due to the fact that interference manifests naturally in networked experiments. However, network information generally is available only up to…

Every design choice will have different effects on different units. However traditional A/B tests are often underpowered to identify these heterogeneous effects. This is especially true when the set of unit-level attributes is…

Experimental testing is vital in the optimization of web applications, and as such A/B testing has been widely adopted as a methodology for determining optimal content for many web applications. While some testing platforms provide…

The importance of Facebook advertising has risen dramatically in recent years, with the platform accounting for almost 20% of the global online ad spend in 2017. An important consideration in advertising is incrementality: how much of the…

We review and conceptualize recent advances in causal inference under network interference, drawing on a complex and diverse body of work that ranges from causal inference, statistical network analysis, economics, the health sciences, and…

Given a set of people and a set of events they attend, we address the problem of measuring connectedness or tie strength between each pair of persons given that attendance at mutual events gives an implicit social network between people. We…

In modern recommendation systems, the standard pipeline involves training machine learning models on historical data to predict user behaviors and improve recommendations continuously. However, these data training loops can introduce…