Related papers: Optimization of Markov process violates detailed b…
Within the framework of probably approximately correct Markov decision processes (PAC-MDP), much theoretical work has focused on methods to attain near optimality after a relatively long period of learning and exploration. However,…
Our purpose is to study a particular class of optimal stopping problems for Markov processes. We justify the value function convexity and we deduce that there exists a boundary function such that the smallest optimal stopping time is the…
We consider a risk-sensitive optimization of consumption-utility on infinite time horizon where the one-period investment gain depends on an underlying economic state whose evolution over time is assumed to be described by a discrete-time,…
Optimal control problems involving hybrid binary-continuous control costs are challenging due to their lack of convexity and weak lower semicontinuity. Replacing such costs with their convex relaxation leads to a primal-dual optimality…
A discrete-time method for solving problems in optimal quantum control is presented. Controlling the time discretized markovian dynamics of a quantum system can be reduced to a Markov-decision process. We demonstrate this method in this…
We connect self-interacting processes, that is, stochastic processes where transitions depend on the time spent by a trajectory in each configuration, to Doob conditioning. In this way we demonstrate that Markov processes with constrained…
The paper addresses two variants of the stochastic shortest path problem ('optimize the accumulated weight until reaching a goal state') in Markov decision processes (MDPs) with integer weights. The first variant optimizes partial expected…
In this paper, we derive first-order Pontryagin optimality conditions for risk-averse stochastic optimal control problems subject to final time inequality constraints, and whose costs are general, possibly non-smooth finite coherent risk…
We consider damped stochastic systems in a controlled (time-varying) quadratic potential and study their transition between specified Gibbs-equilibria states in finite time. By the second law of thermodynamics, the minimum amount of work…
Applying the mathematical circulation theory of Markov chains, we investigate the synchronized stochastic dynamics of a discrete network model of yeast cell-cycle regulation where stochasticity has been kept rather than being averaged out.…
In this paper we prove a weak necessary and sufficient maximum principle for Markovian regime switching stochastic optimal control problems. Instead of insisting on the maximum condition of the Hamiltonian, we show that 0 belongs to the sum…
An analysis of the motion of a relativistic electron under a linear constraint in four dimensions is presented. Interesting results are given that show that the state of the electron is well defined under the formalism of time optimal…
In this work, we introduce an information-theoretic approach for considering changes in dynamics of finitely dimensional open quantum systems governed by master equations. This experimentally motivated approach arises from considering how…
The minimum entropy production principle provides an approximative variational characterization of close-to-equilibrium stationary states, both for macroscopic systems and for stochastic models. Analyzing the fluctuations of the empirical…
In this work we provide explicit conditions on the existence of optimal feedback controls for stochastic processes with regime-switching. We use the compactification method which needs less regularity conditions on the coefficients of the…
This article is concerned with the second order necessary conditions for the stochastic optimal control problem of stochastic evolution equation with model uncertainty when the traditional Pontryagin-type maximum principle holds trivially…
The minimum work principle states that work done on a thermally isolated equilibrium system is minimal for the adiabatically slow (reversible) realization of a given process. This principle, one of the formulations of the second law, is…
Quantum trajectory techniques have been used in the theory of open systems as a starting point for numerical computations and to describe the monitoring of a quantum system in continuous time. Here we extend this technique and use it to…
A Markov decision problem is called reversible if the stationary controlled Markov chain is reversible under every stationary Markovian strategy. A natural application in which such problems arise is in the control of Metropolis-Hastings…
We consider sequences-indexed by time (discrete stages)-of families of multistage stochastic optimization problems. At each time, the optimization problems in a family are parameterized by some quantities (initial states, constraint…