Related papers: MEXIT: Maximal un-coupling times for stochastic pr…
We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the sizes of the one-step jumps…
Let X_t, 0<=t<=T be a one-dimensional stochastic process with independent and stationary increments. This paper considers the problem of stopping the process X_t "as close as possible" to its eventual supremum M_T:=sup{X_t: 0<=t<=T}, when…
In classical probability theory, the term "cutoff" describes the property of some Markov chains to jump from (close to) their initial configuration to (close to) completely mixed in a very narrow window of time. We investigate how coherent…
We study the transition to synchronization in large, dense networks of chaotic circle maps, where an exact solution of the mean-field dynamics in the infinite network and all-to-all coupling limit is known. In dense networks of finite size…
We show how coupling techniques can be used in some metastable systems to prove that mean metastable exit times are almost constant as functions of the starting microscopic configuration within a "meta-stable set." In the example of the…
A generic non-integrable (unitary) out-of-equilibrium quantum process, when interrogated across many times, is shown to yield the same statistics as an (non-unitary) equilibrated process. In particular, using the tools of quantum stochastic…
This paper extends to Continuous-Time Jump Markov Decision Processes (CTJMDP) the classic result for Markov Decision Processes stating that, for a given initial state distribution, for every policy there is a (randomized) Markov policy,…
We investigate the problem of minimizing the entropy production for a physical process that can be described in terms of a Markov jump dynamics. We show that, without any further constraints, a given time-evolution may be realized at…
A general setting for nested subdivisions of a bounded real set into intervals defining the digits $X_1,X_2,...$ of a random variable $X$ with a probability density function $f$ is considered. Under the weak condition that $f$ is almost…
Extreme multistability (EM) is characterized by the emergence of infinitely many coexisting attractors or continuous families of stable states in dynamical systems. EM implies complex and hardly predictable asymptotic dynamical behavior. We…
A Markov chain $X^i$ on a finite state space $S$ has transition matrix $P$ and initial state $i$. We may run the chains $(X^i: i\in S)$ in parallel, while insisting that any two such chains coalesce whenever they are simultaneously at the…
We analyze a controllable generation of maximally entangled mixed states of a circuit containing two-coupled superconducting charge qubits. Each qubit is based on a Cooper pair box connected to a reservoir electrode through a Josephson…
It is often possible to speed up the mixing of a Markov chain $\{ X_{t} \}_{t \in \mathbb{N}}$ on a state space $\Omega$ by \textit{lifting}, that is, running a more efficient Markov chain $\{ \hat{X}_{t} \}_{t \in \mathbb{N}}$ on a larger…
A continuous time mixed state branching process is constructed as the scaling limits of two-type Galton-Watson processes. The process can also be obtained by the pathwise unique solution to a stochastic equation system. From the stochastic…
We propose a new method for solving optimal stopping problems (such as American option pricing in finance) under minimal assumptions on the underlying stochastic process $X$. We consider classic and randomized stopping times represented by…
We study the random acceleration model, which is perhaps one of the simplest, yet nontrivial, non-Markov stochastic processes, and is key to many applications. For this non-Markov process, we present exact analytical results for the…
Coupling is a widely used technique in the theoretical study of interacting stochastic processes. In this paper I present an example demonstrating its usefulness also in the efficient computer simulation of such processes. I first describe…
This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is…
Many distributional quantities in reinforcement learning are intrinsically joint across actions, including distributions of gaps and probabilities of superiority. However, the classical Markov decision process (MDP) formalism specifies only…
We derive sufficient and necessary optimality conditions in terms of a stochastic maximum principle (SMP) for controls associated with cost functionals of mean-field type, under dynamics driven by a class of Markov chains of mean-field type…