Related papers: Optimization of Markov process violates detailed b…
We define a wide class of Markovian load balancing networks of identical single-server infinite-buffer queues. These networks may implement classic parallel server or work stealing load balancing policies, and may be asymmetric, for…
We show that the convergence of finite state space Markov chains to stationarity can often be considerably speeded up by alternating every step of the chain with a deterministic move. Under fairly general conditions, we show that not only…
Consider the problem of a government that wants to reduce the debt-to-GDP (gross domestic product) ratio of a country. The government aims at choosing a debt reduction policy which minimises the total expected cost of having debt, plus the…
Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations.…
This paper deals with a new Tikhonov regularized primal-dual dynamical system with variable mass and Hessian-driven damping for solving a convex optimization problem with linear equality constraints. The system features several…
The principle of detailed balance (DB) states that every kinetic transition in a system with many micro-states, $\mu$, is balanced, on average, with the opposite transition, $\mu_i\leftrightharpoons\mu_j$. Since its introduction by…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
This paper develops a low-nonnegative-rank approximation method to identify the state aggregation structure of a finite-state Markov chain under an assumption that the state space can be mapped into a handful of meta-states. The number of…
Adiabatic protocols are often employed in state preparation schemes but require the system to be driven by a slowly varying Hamiltonian so that transitions between instantaneous eigenstates are exponentially suppressed. Counterdiabatic…
We study Markov decision problems where the agent does not know the transition probability function mapping current states and actions to future states. The agent has a prior belief over a set of possible transition functions and updates…
We consider optimal stopping problems for a Brownian motion and a geometric Brownian motion with a "disorder", assuming that the moment of a disorder is uniformly distributed on a finite interval. Optimal stopping rules are found as the…
Minimum-time quantum control protocols can be obtained from the quantum brachistochrone formalism [Carlini, Hosoya, Koike, and Okudaira, Phys. Rev. Lett. 96, 06053, (2006)]. We point out that the original treatment implicitly applied the…
In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…
We propose a method to sample stationary properties of solutions of stochastic differential equations, which is accurate and efficient if there are rarely visited regions or rare transitions between distinct regions of the state space. The…
We establish existence, uniqueness and regularity of solution results for a class of backward stochastic partial differential equations with singular terminal condition. The equation describes the value function of non-Markovian stochastic…
In this paper we study the stochastic control problem of partially observed (multi-dimensional) stochastic system driven by both Brownian motions and fractional Brownian motions. In the absence of the powerful tool of Girsanov…
We present the conditions under which the time-optimal control problem for a nonlinear non-autonomous linearizable system can be solved by the method of successive approximations, at each step of which a power Markov moment min-problem is…
We present a detailed microscopic derivation for a non-Markovian master equation for a driven two-state system interacting with a general structured reservoir. The master equation is derived using the time-convolutionless projection…
We address the problem of identifying the dynamical law governing the evolution of a population of indistinguishable particles, when only aggregate distributions at successive times are observed. Assuming a Markovian evolution on a discrete…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly…