Related papers: Stopping times in the game Rock-Paper-Scissors

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

In the compulsive gambler process there is a finite set of agents who meet pairwise at random times ($i$ and $j$ meet at times of a rate-$\nu_{ij}$ Poisson process) and, upon meeting, play an instantaneous fair game in which one wins the…

We develop a martingale approach for studying continuous-time stochastic differential games of control and stopping, in a non-Markovian framework and with the control affecting only the drift term of the state-process. Under appropriate…

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…

The paper solves constrained Dynkin games with risk-sensitive criteria, where two players are allowed to stop at two independent Poisson random intervention times, via the theory of backward stochastic differential equations. This…

In this paper we study the N-player nonzero-sum Dynkin game ($N\geq 3$) in continuous time, which is a non-cooperative game where the strategies are stopping times. We show that the game has a Nash equilibrium point for general payoff…

Without mutation and migration, evolutionary dynamics ultimately leads to the extinction of all but one species. Such fixation processes are well understood and can be characterized analytically with methods from statistical physics.…

Reinforcement learning has been successful both empirically and theoretically in single-agent settings, but extending these results to multi-agent reinforcement learning in general-sum Markov games remains challenging. This paper studies…

In this paper we model basketball plays as episodes from team-specific non-stationary Markov decision processes (MDPs) with shot clock dependent transition probabilities. Bayesian hierarchical models are employed in the modeling and…

Let 0<\alpha<1/2. We show that the mixing time of a continuous-time reversible Markov chain on a finite state space is about as large as the largest expected hitting time of a subset of stationary measure at least \alpha of the state space.…

The goal of the paper is to introduce a set of problems which we call mean field games of timing. We motivate the formulation by a dynamic model of bank run in a continuous-time setting. We briefly review the economic and game theoretic…

We consider random-turn positional games, introduced by Peres, Schramm, Sheffield and Wilson in 2007. A $p$-random-turn positional game is a two-player game, played the same as an ordinary positional game, except that instead of alternating…

The spatio-temporal arrangement of interacting populations often influences the maintenance of species diversity and is a subject of intense research. Here, we study the spatio-temporal patterns arising from the cyclic competition between…

In this paper, we consider the optimal stopping problem on semi-Markov processes (SMPs) with finite horizon, and aim to establish the existence and computation of optimal stopping times. To achieve the goal, we first develop the main…

Studies of fixation dynamics in Markov processes predominantly focus on the mean time to absorption. This may be inadequate if the distribution is broad and skewed. We compute the distribution of fixation times in one-step birth-death…

The long-run average payoff per transition (mean payoff) is the main tool for specifying the performance and dependability properties of discrete systems. The problem of constructing a controller (strategy) simultaneously optimizing several…

The number of ``carries'' when $n$ random integers are added forms a Markov chain [23]. We show that this Markov chain has the same transition matrix as the descent process when a deck of $n$ cards is repeatedly riffle shuffled. This gives…

Random walks (or Markov chains) are models extensively used in theoretical computer science. Several tools, including analysis of quantities such as hitting and mixing times, are helpful for devising randomized algorithms. A notable example…

A co-evolving and adaptive Rock (R)-Paper (P)-Scissors (S) game (ARPS) in which an agent uses one of three cyclically dominating strategies is proposed and studied numerically and analytically. An agent takes adaptive actions to achieve a…

Timeout is a short interruption during games used to communicate a change in strategy, to give the players a rest or to stop a negative flow in the game. Whatever the reason, coaches expect an improvement in their team's performance after a…