Related papers: Policy iteration for the deterministic control pro…
We study the policy iteration algorithm (PIA) for entropy-regularized stochastic control problems on an infinite time horizon with a large discount rate, focusing on two main scenarios. First, we analyze PIA with bounded coefficients where…
The paper aims at the development of an apparatus for analysis and construction of near optimal solutions of singularly perturbed (SP) optimal controls problems (that is, problems of optimal control of SP systems) considered on the infinite…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
This paper studies an optimal control problem governed by a semilinear elliptic equation, in which the control acts in a multiplicative or bilinear way as the reaction coefficient of the equation. We focus on the numerical discretization of…
We consider optimal control problems for diffusion processes, where the objective functional is defined by a time-consistent dynamic risk measure. We focus on coherent risk measures defined by $g$-evaluations. For such problems, we…
Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes proximal gradient…
We consider the infinite-horizon discounted optimal control problem formalized by Markov Decision Processes. We focus on Policy Search algorithms, that compute an approximately optimal policy by following the standard Policy Iteration (PI)…
In this paper, we investigate and analyze numerical solutions for the Volterra integrodifferential equations with tempered multi-term kernels. Firstly we derive some regularity estimates of the exact solution. Then a temporal-discrete…
In this paper, a semi-discrete spatial finite volume (FV) method is proposed and analyzed for approximating solutions of anomalous subdiffusion equations involving a temporal fractional derivative of order $\alpha \in (0,1)$ in a…
For optimal stopping problems with time-inconsistent preference, we measure the inherent level of time-inconsistency by taking the time needed to turn the naive strategies into the sophisticated ones. In particular, when in a repeated…
Convergence of the policy iteration method for discrete and continuous optimal control problems holds under general assumptions. Moreover, in some circumstances, it is also possible to show a quadratic rate of convergence for the algorithm.…
Equipping approximate dynamic programming (ADP) with inputconstraints has a tremendous significance. This enables ADP to be applied tothe systems with actuator limitations, which is quite common for dynamicalsystems. In a conventional…
Probabilistic control design is founded on the principle that a rational agent attempts to match modelled with an arbitrary desired closed-loop system trajectory density. The framework was originally proposed as a tractable alternative to…
We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations. There has been…
We analyse a version of the policy iteration algorithm for the discounted infinite-horizon problem for controlled multidimensional diffusion processes, where both the drift and the diffusion coefficient can be controlled. We prove that,…
The autonomous systems need to decide how to react to the changes at runtime efficiently. The ability to rigorously analyze the environment and the system together is theoretically possible by the model-driven approaches; however, the model…
This paper concerns discrete-time infinite-horizon stochastic control systems with Borel state and action spaces and universally measurable policies. We study optimization problems on strategic measures induced by the policies in these…
This paper studies a {\it reversible} investment problem where a social planner aims to control its capacity production in order to fit optimally the random demand of a good. Our model allows for general diffusion dynamics on the demand as…
We analyze a semi-explicit time discretization scheme of first order for poro\-elasticity with nonlinear permeability provided that the elasticity model and the flow equation are only weakly coupled. The approach leads to a decoupling of…
This paper extends the optimal covariance steering problem for linear stochastic systems subject to chance constraints to account for optimal risk allocation. Previous works have assumed a uniform risk allocation to cast the optimal control…