Related papers: Approximating Performance Measures for Slowly Chan…
We introduce $(\varepsilon, \delta)$-bisimulation, a novel type of approximate probabilistic bisimulation for continuous-time Markov chains. In contrast to related notions, $(\varepsilon, \delta)$-bisimulation allows the use of different…
We study the accurate and efficient computation of the expected number of times each state is visited in discrete- and continuous-time Markov chains. To obtain sound accuracy guarantees efficiently, we lift interval iteration and…
We consider a finite number of $N$ statistically equal agents, each moving on a finite set of states according to a continuous-time Markov Decision Process (MDP). Transition intensities of the agents and generated rewards depend not only on…
The intermittent on-off switching of feedback control is considered as a major mechanism of postural stabilization during human quiet standing, which can be modeled by switched-type hybrid stochastic delay differential equations with…
We provide results of a deterministic approximation for non-Markovian stochastic processes modeling finite populations of individuals who recurrently play symmetric finite games and imitate each other according to payoffs. We show that a…
We study the performance of a stochastic algorithm based on the power method that adaptively learns the large deviation functions characterizing the fluctuations of additive functionals of Markov processes, used in physics to model…
Focusing on hybrid diffusion dynamics involving continuous dynamics as well as discrete events, this article investigates the explicit approximations for nonlinear switching diffusion systems modulated by a Markov chain. Different kinds of…
Stochastically monotone Markov chains arise in many applied domains, especially in the setting of queues and storage systems. Poisson's equation is a key tool for analyzing additive functionals of such models, such as cumulative sums of…
Advances in experimental techniques allow the collection of high-resolution spatio-temporal data that track individual motile entities over time. These tracking data motivate the use of mathematical models to characterise the motion…
We develop a novel class of MCMC algorithms based on a stochastized Nesterov scheme. With an appropriate addition of noise, the result is a time-inhomogeneous underdamped Langevin equation, which we prove emits a specified target…
Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be…
We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…
We propose a technique for the design and analysis of adaptation algorithms in dynamical systems. The technique applies both to systems with conventional Lyapunov-stable target dynamics and to ones of which the desired dynamics around the…
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a…
Many systems across the sciences evolve through a combination of multiplicative growth and diffusive transport. In the presence of disorder, these systems tend to form localized structures which alternate between long periods of relative…
By modeling the interaction of a system with an environment through a renewal approach, we demonstrate that completely positive non-Markovian dynamics may develop some unexplored non-standard statistical properties. The renewal approach is…
In a wide range of applications, the stochastic properties of the observed time series change over time. The changes often occur gradually rather than abruptly: the properties are (approximately) constant for some time and then slowly start…
The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the…