Related papers: Markovian Interference in Experiments
This paper develops new methods for causal inference in observational studies on a single large network of interconnected units, addressing two key challenges: long-range dependence among units and the presence of general interference. We…
This paper is concerned with constructing a confidence interval for a target policy's value offline based on a pre-collected observational data in infinite horizon settings. Most of the existing works assume no unmeasured variables exist…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…
We consider the problem of off-policy evaluation in Markov decision processes. Off-policy evaluation is the task of evaluating the expected return of one policy with data generated by a different, behavior policy. Importance sampling is a…
I study identification, estimation and inference for spillover effects in experiments where units' outcomes may depend on the treatment assignments of other units within a group. I show that the commonly-used reduced-form linear-in-means…
Randomized experiments are the preferred approach for evaluating the effects of interventions, but they are costly and often yield estimates with substantial uncertainty. On the other hand, in silico experiments leveraging foundation models…
Many decision-making problems feature multiple objectives. In such problems, it is not always possible to know the preferences of a decision-maker for different objectives. However, it is often possible to observe the behavior of…
We study the problem of off-policy evaluation in the multi-armed bandit model with bounded rewards, and develop minimax rate-optimal procedures under three settings. First, when the behavior policy is known, we show that the Switch…
This chapter covers different approaches to policy evaluation for assessing the causal effect of a treatment or intervention on an outcome of interest. As an introduction to causal inference, the discussion starts with the experimental…
Most reinforcement learning methods are based upon the key assumption that the transition dynamics and reward functions are fixed, that is, the underlying Markov decision process is stationary. However, in many real-world applications, this…
In longitudinal studies where units are embedded in space or a social network, interference may arise, meaning that a unit's outcome can depend on treatment histories of others. The presence of interference poses significant challenges for…
We study the variance of the REINFORCE policy gradient estimator in environments with continuous state and action spaces, linear dynamics, quadratic cost, and Gaussian noise. These simple environments allow us to derive bounds on the…
We characterize a class of Markovian dynamics using the concept of divisible dynamical map. Moreover we provide a family of criteria which can distinguish Markovian and non-Markovian dynamics. These Markovianity criteria are based on a…
The quantum fluctuation-dissipation theorem is a central ingredient in the construction of quantum dynamics of Brownian motion which necessarily is non-Markovian. Yet, often Markovian approximations to quantum dynamics are studied in the…
We study the problem of off-policy evaluation (OPE) in Reinforcement Learning (RL), where the aim is to estimate the performance of a new policy given historical data that may have been generated by a different policy, or policies. In…
This paper investigates the case of interference, when a unit's treatment also affects other units' outcome. When interference is at work, policy evaluation mostly relies on the use of randomized experiments under cluster interference and…
We study estimation of a multivariate function $f:\mathbf{R}^d\to\mathbf{R}$ when the observations are available from the function $Af$, where $A$ is a known linear operator. Both the Gaussian white noise model and density estimation are…
Reinforcement Learning (RL) techniques have been increasingly applied in optimizing control systems. However, their application in quantum systems is hampered by the challenge of performing closed-loop control due to the difficulty in…
When a quantum system interacts with multiple reservoirs, the environmental effects are usually treated in an additive manner. We show that this assumption breaks down for non-Markovian environments that have finite memory times.…
While contemporary reinforcement learning research and applications have embraced policy gradient methods as the panacea of solving learning problems, value-based methods can still be useful in many domains as long as we can wrangle with…