Related papers: Stationary, Markov, stochastic processes with poly…
In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the…
We present a class of Gauss-Markov processes which can be represented as space-time scaled stationary Ornstein-Uhlenbeck processes defined on the real line. We give several explicit examples of the representation for certain Gauss bridge…
This paper presents sufficient conditions for the existence of stationary optimal policies for average-cost Markov Decision Processes with Borel state and action sets and with weakly continuous transition probabilities. The one-step cost…
New algorithms for computing of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes are presented. The algorithms are based on special techniques of sequential phase space reduction, which can…
We present a class of stochastic processes in which the large deviation functions of time-integrated observables exhibit singularities that relate to dynamical phase transitions of trajectories. These illustrative examples include Brownian…
In this paper, we study quasi-stationary distributions of nonlinearly perturbed semi-Markov processes in discrete time. This type of distributions is of interest for the analysis of stochastic systems which have finite lifetimes, but are…
We study the quenched invariance principle for random conductance models with long range jumps on $\Z^d$, where the transition probability from $x$ to $y$ is, on average, comparable to $|x-y|^{-(d+\alpha)}$ with $\alpha\in (0,2)$ but is…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…
We consider the problem of `discrete-time persistence', which deals with the zero-crossings of a continuous stochastic process, X(T), measured at discrete times, T = n(\Delta T). For a Gaussian Stationary Process the persistence (no…
A general class of stochastic gene expression models with self regulation is considered. One or more genes randomly switch between regulatory states, each having a different mRNA transcription rate. The gene or genes are self regulating…
We examine reaction networks (CRNs) through their associated continuous-time Markov processes. Studying the dynamics of such networks is in general hard, both analytically and by simulation. In particular, stationary distributions of…
We study some regularity properties in locally stationary Markov models which are fundamental for controlling the bias of nonparametric kernel estimators. In particular, we provide an alternative to the standard notion of derivative process…
A Markov process is registered. At random moment $\theta$ the distribution of observed sequence changes. Using probability maximizing approach the optimal stopping rule for detecting the change is identified. Some explicit solution is…
We propose a method to sample stationary properties of solutions of stochastic differential equations, which is accurate and efficient if there are rarely visited regions or rare transitions between distinct regions of the state space. The…
A variety of physical phenomena involve the nonlinear transfer of energy from weakly damped modes subjected to external forcing to other modes which are more heavily damped. In this work we explore this in (finite-dimensional) stochastic…
The linear fractional stable motion generalizes two prominent classes of stochastic processes, namely stable L\'evy processes, and fractional Brownian motion. For this reason it may be regarded as a basic building block for continuous time…
We study a stochastic multiplicative process with reset events. It is shown that the model develops a stationary power-law probability distribution for the relevant variable, whose exponent depends on the model parameters. Two qualitatively…
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or enhanced sampling methods may improve the…
We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…
We study stochastic Mean Field Games on networks with sticky transition conditions. In this setting, the diffusion process governing the agent's dynamics can spend finite time both in the interior of the edges and at the vertices. The…