Related papers: Blackwell optimality and policy stability for long…
Markov Decision Processes (MDPs) are a formal framework for modeling and solving sequential decision-making problems. In finite-time horizons such problems are relevant for instance for optimal stopping or specific supply chain problems,…
Stochastic models such as Continuous-Time Markov Chains (CTMC) and Stochastic Hybrid Automata (SHA) are powerful formalisms to model and to reason about the dynamics of biological systems, due to their ability to capture the stochasticity…
The performance of decision policies and prediction models often deteriorates when applied to environments different from the ones seen during training. To ensure reliable operation, we analyze the stability of a system under distribution…
One often encounters the curse of dimensionality in the application of dynamic programming to determine optimal policies for controlled Markov chains. In this paper, we provide a method to construct sub-optimal policies along with a bound…
We study the evaluation of a policy under best- and worst-case perturbations to a Markov decision process (MDP), using transition observations from the original MDP, whether they are generated under the same or a different policy. This is…
Adaptive optimal control using value iteration initiated from a stabilizing control policy is theoretically analyzed in terms of stability of the system during the learning stage without ignoring the effects of approximation errors. This…
This paper conducts sensitivity analysis of random constraint and variational systems related to stochastic optimization and variational inequalities. We establish efficient conditions for well-posedness, in the sense of robust Lipschitzian…
In this paper, we consider a stochastic recursive optimal control problem under model uncertainty. In this framework, the cost function is described by solutions of a family of backward stochastic differential equations. With the help of…
Stochastic optimal control of dynamical systems is a crucial challenge in sequential decision-making. Recently, control-as-inference approaches have had considerable success, providing a viable risk-sensitive framework to address the…
Risk-sensitive planning aims to identify policies maximizing some tail-focused metrics in Markov Decision Processes (MDPs). Such an optimization task can be very costly for the most widely used and interpretable metrics such as threshold…
In this paper, we consider an integrated MSP-MDP framework which captures features of Markov decision process (MDP) and multistage stochastic programming (MSP). The integrated framework allows one to study a dynamic decision-making process…
Control of continuous time dynamics with multiplicative noise is a classic topic in stochastic optimal control. This work addresses the problem of designing infinite horizon optimal controls with stability guarantees for \textit{a single…
Following the recent resurgence in establishing linear control theoretic benchmarks for reinforcement leaning (RL)-based policy optimization (PO) for complex dynamical systems with continuous state and action spaces, an optimal control…
This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is…
We consider a piecewise deterministic Markov decision process, where the expected exponential utility of total (nonnegative) cost is to be minimized. The cost rate, transition rate and post-jump distributions are under control. The state…
The literature on continuous-time stochastic optimal control seldom deals with the case of discrete state spaces. In this paper, we provide a general framework for the optimal control of continuous-time Markov chains on finite graphs. In…
A stochastic model predictive control (SMPC) approach is presented for discrete-time linear systems with arbitrary time-invariant probabilistic uncertainties and additive Gaussian process noise. Closed-loop stability of the SMPC approach is…
We study discrete-time discounted constrained Markov decision processes (CMDPs) on Borel spaces with unbounded reward functions. In our approach the transition probability functions are weakly or set-wise continuous. The reward functions…
In this paper, we study a discrete-time stochastic optimal control problem under distribution uncertainty with convex control domain. By weak convergence method and Sion's minimax theorem, we obtain the variational inequality for cost…
We consider the economic problem of optimal consumption and investment with power utility. We study the optimal strategy as the relative risk aversion tends to infinity or to one. The convergence of the optimal consumption is obtained for…