相关论文: Long run control of nonhomogeneous Markov processe…

In this paper we consider an infinite time horizon risk-sensitive optimal stopping problem for a Feller--Markov process with an unbounded terminal cost function. We show that in the unbounded case an associated Bellman equation may have…

A class of stochastic optimal control problems involving optimal stopping is considered. Methods of Krylov are adapted to investigate the numerical solutions of the corresponding normalized Bellman equations and to estimate the rate of…

We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a…

In this paper, we consider risk-sensitive discounted control problem for continuous-time jump Markov processes taking values in general state space. The transition rates of underlying continuous-time jump Markov processes and the cost rates…

This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is…

We conduct an investigation of the differentiability and continuity of reward functionals associated to Markovian randomized stopping times. Our focus is mostly on the differentiability, which is a crucial ingredient for a common approach…

This paper studies three ways to construct a nonhomogeneous jump Markov process: (i) via a compensator of the random measure of a multivariate point process, (ii) as a minimal solution of the backward Kolmogorov equation, and (iii) as a…

We provide general conditions ensuring that the value functions of some nonlinear stopping problems with finite horizon converge to the value functions of the corresponding problems with infinite horizon. Our result can be formulated as…

We study time-consistency questions for processes of monetary risk measures that depend on bounded discrete-time processes describing the evolution of financial values. The time horizon can be finite or infinite. We call a process of…

This paper studies function approximation for finite horizon discrete time Markov decision processes under certain convexity assumptions. Uniform convergence of these approximations on compact sets is proved under several sampling schemes…

Recent work has led to the development of an elegant theory of Linearly Solvable Markov Decision Processes (LMDPs) and related Path-Integral Control Problems. Traditionally, MDPs have been formulated using stochastic policies and a control…

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation…

In this paper we study a family of nonlinear (conditional) expectations that can be understood as a semimartingale with uncertain local characteristics. Here, the differential characteristics are prescribed by a time and path-dependent…

This paper describes the structure of solutions to Kolmogorov's equations for nonhomogeneous jump Markov processes and applications of these results to control of jump stochastic systems. These equations were studied by Feller (1940), who…

A general result on the method of randomized stopping is proved. It is applied to optimal stopping of controlled diffusion processes with unbounded coefficients to reduce it to an optimal control problem without stopping. This is motivated…

In this paper, we study the controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters.…

A continuous-time random walk in the quarter plane with homogeneous transition rates is considered. Given a non-negative reward function on the state space, we are interested in the expected stationary performance. Since a direct derivation…

Given a unichain Markov reward process (MRP), we provide an explicit expression for the bias values in terms of mean first passage times. This result implies a generalization of known Markov chain perturbation bounds for the stationary…

We study Markov decision processes with Polish state and action spaces. The action space is state dependent and is not necessarily compact. We first establish the existence of an optimal ergodic occupation measure using only a near-monotone…

Bellman equations of ergodic type related to risk-sensitive control are considered. We treat the case that the nonlinear term is positive quadratic form on first-order partial derivatives of solution, which includes linear exponential…