Related papers: Comparing experiments in discounted problems
Many organizations utilize large-scale online controlled experiments (OCEs) to accelerate innovation. Having high statistical power to detect small differences between control and treatment accurately is critical, as even small changes in…
Reinforcement learning algorithms are typically designed for generic Markov Decision Processes (MDPs), where any state-action pair can lead to an arbitrary transition distribution. In many practical systems, however, only a subset of the…
The idea is advanced that self-organization in complex systems can be treated as decision making (as it is performed by humans) and, vice versa, decision making is nothing but a kind of self-organization in the decision maker nervous…
Preferences often change -- even in short time intervals -- due to either the mere passage of time (present-biased preferences) or changes in environmental conditions (state-dependent preferences). On the basis of the empirical findings in…
We address the problem of approximate model minimization for MDPs in which the state is partitioned into endogenous and (much larger) exogenous components. An exogenous state variable is one whose dynamics are independent of the agent's…
We present analytical expressions for the time-dependent and stationary probability distributions corresponding to a stochastically perturbed one-dimensional flow with critical points, in two physically relevant situations: delayed…
We model the joint distribution of choice probabilities and decision times in binary choice tasks as the solution to a problem of optimal sequential sampling, where the agent is uncertain of the utility of each action and pays a constant…
The problem of scheduling with testing in the framework of explorable uncertainty models environments where some preliminary action can influence the duration of a task. In the model, each job has an unknown processing time that can be…
Event-based methods carefully select when to transmit information to enable high-performance control and estimation over resource-constrained communication networks. However, they come at a cost. For instance, event-based communication…
This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be…
Recently, it has been recognized that phase transitions play an important role in the probabilistic analysis of combinatorial optimization problems. However, there are in fact many other relations that lead to close ties between computer…
In the changepoint problem, we determine when the distribution observed has changed to another one. We expand this problem to the quantum case where copies of an unknown pure state are being distributed. We study the fundamental case, which…
The problem of detecting a single anomalous process among multiple independent processes is considered. Under a constraint on the number of processes that can be probed simultaneously, the decision maker should decide which processes to…
Suppose that at any stage of a statistical experiment a control variable $X$ that affects the distribution of the observed data $Y$ can be used. The distribution of $Y$ depends on some unknown parameter $\theta$, and we consider the…
We study off-policy evaluation and learning from sequential data in a structured class of Markov decision processes that arise from repeated interactions with an exogenous sequence of arrivals with contexts, which generate unknown…
We study an individual-based model in which two spatially-distributed species, characterized by different diffusivities, compete for resources. We consider three different ecological settings. In the first, diffusing faster has a cost in…
The effects of a stochastic reset, to its initial configuration, is studied in the exactly solvable one-dimensional coagulation-diffusion process. A finite resetting rate leads to a modified non-equilibrium stationary state. If in addition…
In this note, we focus on a selection model problem: a mono-exponential model versus a bi-exponential one. This is done in the biological context of living cells, where small data are available. Classical statistics are revisited to improve…
An important question in economics is how people choose between different payments in the future. The classical normative model predicts that a decision maker discounts a later payment relative to an earlier one by an exponential function…
Event occurrence is not only subject to the environmental changes, but is also facilitated by the events that have occurred in a system. Here, we develop a method for estimating such extrinsic and intrinsic factors from a single series of…