Related papers: RAP-modulated Fluid Processes: First Passages and …
A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…
We propose a unifying theoretical framework for the analysis of first-passage time distributions in two important classes of stochastic processes in which the diffusivity of a particle evolves randomly in time. In the first class of…
Random processes with stationary increments and intrinsic random processes are two concepts commonly used to deal with non-stationary random processes. They are broader classes than stationary random processes and conceptually closely…
The methodology based on the random walk processes is adapted and applied to a comprehensive analysis of the statistical properties of the probability fluxes. To this aim we define a simple model of the Markovian stochastic dynamics on a…
The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of…
In this paper, we seek to understand the behavior of dynamical systems that are perturbed by a parameter that changes discretely in time. If we impose certain conditions, we can study certain embedded systems within a hybrid system as…
We present a numerical method to compute the survival function and the moments of the exit time for a piecewise-deterministic Markov process (PDMP). Our approach is based on the quantization of an underlying discrete-time Markov chain…
Probabilistic generative models based on measure transport, such as diffusion and flow-based models, are often formulated in the language of Markovian stochastic dynamics, where the choice of the underlying process impacts both algorithmic…
A stochastic model for a super-position of uncorrelated pulses with a random distribution of and correlations between amplitudes and velocities is analyzed. The pulses are assumed to move radially with fixed shape and amplitudes decreasing…
Robust estimates for the performance of complicated queueing networks can be obtained by showing that the number of jobs in the network is stochastically comparable to a simpler, analytically tractable reference network. Classical coupling…
A method is developed to estimate the properties of a global hydrodynamic instability in turbulent flows from measurement data of the limit-cycle oscillations. For this purpose, the flow dynamics are separated in deterministic contributions…
Piecewise-deterministic Markov processes combine continuous in time dynamics with jump events, the rates of which generally depend on the continuous variables and thus are not constants. This leads to a problem in a Monte-Carlo simulation…
We consider a linearized dynamical system modelling the flow rate of water along the rivers and hillslopes of an arbitrary watershed. The system is perturbed by a random rainfall in the form of a compound Poisson process. The model…
Piecewise-deterministic Markov processes (PDMPs) are often used to model abrupt changes in the global environment or capabilities of a controlled system. This is typically done by considering a set of "operating modes" (each with its own…
We consider a stochastic, dynamic job scheduling problem, formulated as a queueing control problem, in which a single server processes jobs of different types that arrive according to independent Poisson processes. The problem is defined on…
We propose a piecewise deterministic Markovian jump process in Hilbert space such that the covariance matrix of this stochastic process solves the thermodynamic quantum master equation. The proposed stochastic process is particularly simple…
A stochastic pump is a Markov model of a mesoscopic system evolving under the control of externally varied parameters. In the model, the system makes random transitions among a network of states. For such models, a "no-pumping theorem" has…
Markov Decision Processes (MDPs) are stochastic optimization problems that model situations where a decision maker controls a system based on its state. Partially observed Markov decision processes (POMDPs) are generalizations of MDPs where…
We present a general method for constructing integrable stochastic processes, with two-step discrete time Floquet dynamics, from the transfer matrix formalism. The models can be interpreted as a discrete time parallel update. The method can…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…