Related papers: Bounds for the expected value of one-step processe…
Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…
We consider models of the population or opinion dynamics which result in the non-linear stochastic differential equations (SDEs) exhibiting the spurious long-range memory. In this context, the correspondence between the description of the…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
We consider the piecewise-deterministic Markov process obtained by randomly switching between the flows generated by a finite set of smooth vector fields on a compact set. We obtain H\"ormander-type conditions on the vector fields…
Stochastic infection processes are continuous-time Markov chains on graphs that assign each vertex one of multiple states, such as susceptible, infected, or recovered. Depending on the model, vertices change their state based on random…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…
We develop an exhaustive study of Markov decision process (MDP) under mean field interaction both on states and actions in the presence of common noise, and when optimization is performed over open-loop controls on infinite horizon. Such…
For general, almost surely absorbed Markov processes, we obtain necessary and sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the…
We construct and study branching Markov processes on the space of finite configurations of the state space of a given standard process, controlled by a branching kernel and a killing one. In particular, we may start with a superprocess,…
We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such…
We consider the problem of conditioning a Markov process on a rare event and of representing this conditioned process by a conditioning-free process, called the effective or driven process. The basic assumption is that the rare event used…
We study Markov processes on weighted directed hypergraphs where the state of at most one vertex can change at a time. Our setting is general enough to include simplicial epidemic processes, processes on multilayered networks or even the…
In many biological systems, chemical reactions or changes in a physical state are assumed to occur instantaneously. For describing the dynamics of those systems, Markov models that require exponentially distributed inter-event times have…
We prove a large deviation principle on path space for a class of discrete time Markov processes whose state space is the intersection of a regular domain $\L\subset \R^d$ with some lattice of spacing $\e$. Transitions from $x$ to $y$ are…
Markov models are widely used to describe processes of stochastic dynamics. Here, we show that Markov models are a natural consequence of the dynamical principle of Maximum Caliber. First, we show that when there are different possible…
Parameter estimation in diffusion processes from discrete observations up to a first-hitting time is clearly of practical relevance, but does not seem to have been studied so far. In neuroscience, many models for the membrane potential…
This work contains two single-letter upper bounds on the entropy rate of a discrete-valued stationary stochastic process, which only depend on second-order statistics, and are primarily suitable for models which consist of relatively large…
We study a density-dependent Markov jump process describing a population where each individual is characterized by a type, and reproduces at rates depending both on its type and on the population type distribution. We are interested in the…
We develop a semi-parametric state-space model for time-series data with latent regime transitions. Classical Markov-switching models use fixed parametric transition functions, such as logistic or probit links, which restrict flexibility…