Related papers: Optimization of Markov process violates detailed b…
We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the…
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…
In recent work, Baez, Fong and the author introduced a framework for describing Markov processes equipped with a detailed balanced equilibrium as open systems of a certain type. These `open Markov processes' serve as the building blocks for…
Bayesian optimization is a methodology to optimize black-box functions. Traditionally, it focuses on the setting where you can arbitrarily query the search space. However, many real-life problems do not offer this flexibility; in…
Recent studies have experienced the acceleration of convergence in Markov chain Monte Carlo methods implemented by the systems without detailed balance condition (DBC). However, such advantage of the violation of DBC has not been confirmed…
We study an irreversible Markov chain Monte Carlo method based on a skew detailed balance condition for an one-dimensional Ising model. Dynamical behavior of the magnetization density is analyzed in order to understand the properties of…
We introduce an alternative way to derive the generalized form of the master equation recently presented by J. P. Pekola et al. [Phys. Rev. Lett. 105, 030401 (2010)] for an adiabatically steered two-level quantum system interacting with a…
This paper studies a dynamic real-time optimization in the context of model-based time-optimal operation of batch processes under parametric model mismatch. In order to tackle the model-mismatch issue, a receding-horizon policy is usually…
When clustering molecular dynamics (MD) trajectories into a few metastable conformational states, the Markov state models (MSMs) assumption of timescale separation between fast intrastate fluctuations and rarely occurring interstate…
In this paper, we propose a novel technique to implement stochastic gradient methods, which are beneficial for learning from large datasets, through accelerated stochastic dynamics. A stochastic gradient method is based on mini-batch…
We study merchant energy production modeled as a compound switching and timing option. The resulting Markov decision process is intractable. State-of-the-art approximate dynamic programming methods applied to realistic instances of this…
An irreversible Markov-chain Monte Carlo (MCMC) algorithm with skew detailed balance conditions originally proposed by Turitsyn et al. is extended to general discrete systems on the basis of the Metropolis-Hastings scheme. To evaluate the…
This paper is devoted to studying the weak convergence for a slow-fast system with jumps modulated by Markovian switching regimes with the martingale method. However, due to the coexistence of fast component and Markovian switching regimes,…
Molecular dynamics simulations allow to study the structure and dynamics of single biomolecules in microscopic detail. However, many processes occur on time scales beyond the reach of fully atomistic simulations and require coarse-grained…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
We derive a Markovian master equation that models the evolution of systems subject to driving and control fields. Our approach combines time rescaling and weak-coupling limits for the system-environment interaction with a secular…
We present a number of new physical systems that may be addressed using methods of time dependent transformation. A recap of results available for two-state systems is given, with particular emphasis on the AC stark effect. We give some…
Transitions in the qualitative behavior of chemical reaction dynamics with a decrease in molecule number have attracted much attention. Here, a method based on a Markov process with a tridiagonal transition matrix is applied to the analysis…
We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…
The violation of detailed balance poses a serious problem for the majority of current quasiclassical methods for simulating nonadiabatic dynamics. In order to analyze the severity of the problem, we predict the long-time limits of the…