Related papers: Bounding the coarse graining error in hidden Marko…
We consider an energy storage problem involving a wind farm with a forecasted power output, a stochastic load, an energy storage device, and a connection to the larger power grid with stochastic prices. Electricity prices and wind power…
We define a Markov process in a forward population model with backward genealogy given by the $\Lambda$-coalescent. This Markov process, called the fixation line, is related to the block counting process through its hitting times. Two…
Piecewise deterministic Markov processes (PDMPs) are a class of stochastic processes with applications in several fields of applied mathematics spanning from mathematical modeling of physical phenomena to computational methods. A PDMP is…
In this paper, we study a notion of local stationarity for discrete time Markov chains which is useful for applications in statistics. In the spirit of some locally stationary processes introduced in the literature, we consider triangular…
The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…
Euclidean Markov decision processes are a powerful tool for modeling control problems under uncertainty over continuous domains. Finite state imprecise, Markov decision processes can be used to approximate the behavior of these infinite…
A Markov approximation in open quantum dynamics can give unphysical results when a map acts on a state that is not in its domain. This is examined here in a simple example, an open quantum dynamics for one qubit in a system of two…
We study a class of Markov processes that combine local dynamics, arising from a fixed Markov process, with regenerations arising at a state-dependent rate. We give conditions under which such processes possess a given target distribution…
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results…
Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a…
We consider the application of fluctuation relations to the dynamics of coarse-grained systems, as might arise in a hypothetical experiment in which a system is monitored with a low-resolution measuring apparatus. We analyze a stochastic,…
Motivated by a risk process with positive and negative premium rates, we consider a real-valued Markov additive process with finitely many background states. This additive process linearly increases or decreases while the background state…
We show in detail how to determine the time-reversed representation of a stationary hidden stochastic process from linear combinations of its forward-time $\epsilon$-machine causal states. This also gives a check for the $k$-cryptic…
The new class of Markov processes is proposed to realize the flexible shrinkage effects for the dynamic models. The transition density of the new process consists of two penalty functions, similarly to Bayesian fused LASSO in its functional…
We analyse and interpret the effects of breaking detailed balance on the convergence to equilibrium of conservative interacting particle systems and their hydrodynamic scaling limits. For finite systems of interacting particles, we review…
In the paper, we study a new rate of convergence estimate for homogeneous discrete-time nonlinear Markov chains based on the Markov-Dobrushin condition. This result generalizes the convergence estimates for any positive number of transition…
We demonstrate how the dynamical coarse-graining approach can be systematically extended to higher orders in the coupling between system and reservoir. Up to second order in the coupling constant we explicitly show that dynamical…
We introduce a new method to accurately and efficiently estimate the effective dynamics of collective variables in molecular simulations. Such reduced dynamics play an essential role in the study of a broad class of processes, ranging from…
The efficient calculation of rare-event kinetics in complex dynamical systems, such as the rate and pathways of ligand dissociation from a protein, is a generally unsolved problem. Markov state models can systematically integrate ensembles…
Time bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving…