相关论文: Optimization of Markov process violates detailed b…

Improved rates of convergence for ergodic homogeneous Markov chains are studied. In comparison to the earlier papers the setting is also generalised to the case without a unique dominated measure. Examples are provided where the new bound…

The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…

Previous work has separately addressed different forms of action, state and action-state entropy regularization, pure exploration and space occupation. These problems have become extremely relevant for regularization, generalization,…

This paper is concerned with the maximum principle of stochastic optimal control problems, where the coefficients of the state equation and the cost functional are uncertain, and the system is generally under Markovian regime switching.…

Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…

Given two probability measures on sequential data, we investigate the transport problem with time-inconsistent preferences in a discrete-time setting. Motivating examples are nonlinear objectives, state-dependent costs, and regularized…

The present paper considers the constrained optimal control problem with total undiscounted criteria for a continuous-time Markov decision process (CTMDP) in Borel state and action spaces. Under the standard compactness and continuity…

We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model…

We consider an additive functional driven by a time-inhomogeneous Markov chain with a finite state space. Our study focuses on the joint distribution of the two-sided exit time and the state of the driving Markov chain at the time of exit,…

The Markov Chain Monte Carlo method is the dominant paradigm for posterior computation in Bayesian analysis. It is common to control computation time by making approximations to the Markov transition kernel. Comparatively little attention…

In this paper, a general stochastic model with controls applied at the moments when the random process hits the boundary of a given subset of the state set is proposed and studied. The general concept of the model is formulated and its…

We consider the time-optimal problem for a classical system of "double integrator" under the presence of a linear state constraint. By using the Maximum Principle of Dubovitskii and Milyutin, we determine a complete synthesis of optimal…

We discuss applications of a recently developed method for model reduction based on linear response theory of weakly coupled dynamical systems. We apply the weak coupling method to simple stochastic differential equations with slow and fast…

Rate processes are simple and analytically tractable models for many dynamical systems which switch stochastically between a discrete set of quasi stationary states but they may also approximate continuous processes by coarse grained,…

A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

In this paper we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time…

The variational principle for conformational dynamics has enabled the systematic construction of Markov state models through the optimization of hyperparameters by approximating the transfer operator. In this note we discuss why lag time of…

Markovian master equations (formally known as quantum dynamical semigroups) can be used to describe the evolution of a quantum state $\rho$ when in contact with a memoryless thermal bath. This approach has had much success in describing the…

The work treats systems combining slow and fast motions depending on each other where fast motions are perturbations of families of either dynamical systems or Markov processes with freezed slow variable. In the first case we consider…