Related papers: Explicit computations for some Markov modulated co…
We derive the exact solution of a one-dimensional Markov functional model with log-normally distributed interest rates in discrete time. The model is shown to have two distinct limiting states, corresponding to small and asymptotically…
Markov chains are simple yet powerful mathematical structures to model temporally dependent processes. They generally assume stationary data, i.e., fixed transition probabilities between observations/states. However, live, real-world…
Through the analysis of a dataset of ultra high frequency order book updates, we introduce a model which accommodates the empirical properties of the full order book together with the stylized facts of lower frequency financial data. To do…
By monitoring the sampling of states with different magnetizations in transition matrix procedures a family of accurate and easily implemented techniques are constructed that automatically control the variation of the temperature or energy…
Branching processes are a class of continuous-time Markov chains (CTMCs) with ubiquitous applications. A general difficulty in statistical inference under partially observed CTMC models arises in computing transition probabilities when the…
Refinement calculus is a powerful and expressive tool for reasoning about sequential programs in a compositional manner. In this paper we present an extension of refinement calculus for reactive systems. Refinement calculus is based on…
This paper presents algorithms for identifying and reducing a dedicated set of controllable transition rates of a state-labelled continuous-time Markov chain model. The purpose of the reduction is to make states to satisfy a given…
For non-equilibrium systems described by finite Markov processes, we consider the number of times that a system traverses a cyclic sequence of states (a cycle). The joint distribution of the number of forward and backward instances of any…
Mean-field models are often used to approximate Markov processes with large state-spaces. One-step processes, also known as birth-death processes, are an important class of such processes and are processes with state space…
We prove an invariance principle for non-stationary random processes and establish a rate of convergence under a new type of mixing condition. The dependence is exponentially decaying in the gap between the past and the future and is…
This paper deals with optimal prediction in a regime-switching model driven by a continuous-time Markov chain. We extend existing results for geometric Brownian motion by deriving optimal stopping strategies that depend on the current…
Markovian-regime-switching (MRS) models are commonly used for modelling economic time series, including electricity prices where independent regime models are used, since they can more accurately and succinctly capture electricity price…
This chapter presents an introduction to Markovian modeling for the analysis of sequence data. Contrary to the deterministic approach seen in the previous sequence analysis chapters, Markovian models are probabilistic models, focusing on…
In piecewise-deterministic Markov processes (PDMPs) the state of a finite-dimensional system evolves continuously, but the evolutive equation may change randomly as a result of discrete switches. A running cost is integrated along the…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…
The reduction of a continuous Markov process with multiple metastable states to a discrete rate process is investigated in the presence of slow time dependent parameters such as periodic external forces or slowly fluctuating barrier…
A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…
This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (a.k.a. variable length…