Related papers: A derivation of the master equation from path entr…
This review provides a pedagogic and self-contained introduction to master equations and to their representation by path integrals. We discuss analytical and numerical methods for the solution of master equations, keeping our focus on…
Inference for partially observed Markov process models has been a longstanding methodological challenge with many scientific and engineering applications. Iterated filtering algorithms maximize the likelihood function for partially observed…
Markov Population Models are a widespread formalism used to model the dynamics of complex systems, with applications in Systems Biology and many other fields. The associated Markov stochastic process in continuous time is often analyzed by…
An extension of non-deterministic processes driven by the random telegraph signal is introduced in the framework of "piecewise deterministic Markov processes" [Davis], including a broader category of random systems. The corresponding…
Stochastic processes are proposed whose master equations coincide with classical wave, telegraph, and Klein-Gordon equations. Similar to predecessors based on the Goldstein-Kac telegraph process, the model describes the motion of particles…
Given a semi-Markov law, using an additional parameter, we consider a family of stochastic flows corresponding to that law. Then we suitably select a particular flow, for which we obtain expressions of the meeting and merging probabilities…
Existing studies on the degree correlation of evolving networks typically rely on differential equations and statistical analysis, resulting in only approximate solutions due to inherent randomness. To address this limitation, we propose an…
Sample-based trajectory optimisers are a promising tool for the control of robotics with non-differentiable dynamics and cost functions. Contemporary approaches derive from a restricted subclass of stochastic optimal control where the…
Consider a uniformly sampled random $d$-regular graph on $n$ vertices. If $d$ is fixed and $n$ goes to $\infty$ then we can relate typical (large probability) properties of such random graph to a family of invariant random processes (called…
We introduce a general framework for Markov decision problems under model uncertainty in a discrete-time infinite horizon setting. By providing a dynamic programming principle we obtain a local-to-global paradigm, namely solving a local,…
We study three classes of continuous time Markov processes (inclusion process, exclusion process, independent walkers) and a family of interacting diffusions (Brownian energy process). For each model we define a boundary driven process…
This paper analyzes stochastic networks consisting of finite capacity nodes with different classes of requests which move according to some routing policy. The Markov processes describing these networks do not, in general, have…
Motivated by entropic optimal transport, time reversal of Markov jump processes in $\mathbb{R}^n$ is investigated. Relying on an abstract integration by parts formula for the carr\'e du champ of a Markov process recently obtained by…
The investigation of optimal processes has a long history in the field of thermodynamics. It is well known that finite-time processes that minimize dissipation often exhibit discontinuities. We use a combination of numerical and analytical…
Markov matrices have an important role in the filed of stochastic processes. In this paper, we will show and prove a series of conclusions on Markov matrices and transformations rather than pay attention to stochastic processes although…
We analyze the Gaussian approximation as a method to obtain the first and second moments of a stochastic process described by a master equation. We justify the use of this approximation with ideas coming from van Kampen's expansion approach…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
Normalizing flows, diffusion normalizing flows and variational autoencoders are powerful generative models. This chapter provides a unified framework to handle these approaches via Markov chains. We consider stochastic normalizing flows as…
We develop a quadratic regularization approach for the solution of high-dimensional multistage stochastic optimization problems characterized by a potentially large number of time periods/stages (e.g. hundreds), a high-dimensional resource…
We study various classes of random processes defined on the regular tree $T_d$ that are invariant under the automorphism group of $T_d$. Most important ones are factor of i.i.d. processes (randomized local algorithms), branching Markov…