Related papers: Continuous-Time Higher Order Markov Chains: Formul…
We present an optimization process to estimate parameters in systems of ordinary differential equations from chaotic time series. The optimization technique is based on a variational approach, and numerical studies on noisy time series…
Complex systems are characterized by a huge number of degrees of freedom often interacting in a non-linear manner. In many cases macroscopic states, however, can be characterized by a small number of order parameters that obey stochastic…
Complex systems may often be characterized by their hierarchical dynamics. In this paper do we present a method and an operational algorithm that automatically infer this property in a broad range of systems; discrete stochastic processes.…
We propose new continuous-time formulations for first-order stochastic optimization algorithms such as mini-batch gradient descent and variance-reduced methods. We exploit these continuous-time models, together with simple Lyapunov analysis…
A new branch based on Markov processes is developing in the recent literature of financial time series modeling. In this paper, an Indexed Markov Chain has been used to model high frequency price returns of quoted firms. The peculiarity of…
In the paper we consider a stochastic model which called Markov Q-processes that forms a continuous-time Markov population system. Markov Q-processes are defined as stochastic Markov branching processes with trajectories continuing in the…
Stochastic process discovery is concerned with deriving a model capable of reproducing the stochastic character of observed executions of a given process, stored in a log. This leads to an optimisation problem in which the model's parameter…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…
A continuous-time Markov chain rate change formula for simulation, model selection, filtering and theory is proven. It is used to develop Markov chain importance sampling, rejection sampling, branching particle filtering algorithms and…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
We present an efficient finite difference method for the computation of parameter sensitivities that is applicable to a wide class of continuous time Markov chain models. The estimator for the method is constructed by coupling the perturbed…
Stochastic dynamical systems are fundamental in state estimation, system identification and control. System models are often provided in continuous time, while a major part of the applied theory is developed for discrete-time systems.…
Various and ubiquitous information systems are being used in monitoring, exchanging, and collecting information. These systems are generating massive amount of event sequence logs that may help us understand underlying phenomenon. By…
Preferences play a key role in determining what goals/constraints to satisfy when not all constraints can be satisfied simultaneously. In this work, we study preference-based planning in a stochastic system modeled as a Markov decision…
We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…
Stochastic mathematical models are essential tools for understanding and predicting complex phenomena. The purpose of this work is to study the exit times of a stochastic dynamical system-specifically, the mean exit time and the…
We introduce bounds on the finite-time performance of Markov chain Monte Carlo algorithms in approaching the global solution of stochastic optimization problems over continuous domains. A comparison with other state-of-the-art methods…
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…
This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we…
The most common approaches for solving multistage stochastic programming problems in the research literature have been to either use value functions ("dynamic programming") or scenario trees ("stochastic programming") to approximate the…