Related papers: Markov control of continuous time Markov processes…

We consider mean-field control problems in discrete time with discounted reward, infinite time horizon and compact state and action space. The existence of optimal policies is shown and the limiting mean-field problem is derived when the…

We consider a stochastic, dynamic job scheduling problem, formulated as a queueing control problem, in which a single server processes jobs of different types that arrive according to independent Poisson processes. The problem is defined on…

In this paper, a class of piecewise deterministic Markov processes with underlying fast dynamic is studied. Using a "penalty method" , an averaging result is obtained when the underlying dynamic is infinitely accelerated. The features of…

We consider the stochastic optimal control problem of nonlinear mean-field systems in discrete time. We reformulate the problem into a deterministic control problem with marginal distribution as controlled state variable, and prove that…

Conditions are established under which the optimal control of processes having both absolutely continuous and singular (with respect to time) controls are equivalent to linear programs over a space of measures on the state and control…

First-order methods are often analyzed via their continuous-time models, where their worst-case convergence properties are usually approached via Lyapunov functions. In this work, we provide a systematic and principled approach to find and…

We consider a control problem for a finite-state Markov system whose performance is evaluated by a coherent Markov risk measure. For each policy, the risk of a state is approximated by a function of its features, thus leading to a…

A Markov process fluctuating away from its typical behavior can be represented in the long-time limit by another Markov process, called the effective or driven process, having the same stationary states as the original process conditioned…

In this paper we present the concept of description of random processes in complex systems with the discrete time. It involves the description of kinetics of discrete processes by means of the chain of finite-difference non-Markov equations…

Continuous time financial market models are often motivated as scaling limits of discrete time models. The objective of this paper is to establish such a connection for a robust framework. More specifically, we consider discrete time models…

We present an elementary state augmentation method for a class of static risk measure applied to the total cost for both Markov decision processes and stochastic optimal control, such that dynamic programming equations can be derived on the…

It is well-known that compositions of Markov processes with inverse subordinators are governed by integro-differential equations of generalized fractional type. This kind of processes are of wide interest in statistical physics as they are…

For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…

We study discrete-time discounted constrained Markov decision processes (CMDPs) on Borel spaces with unbounded reward functions. In our approach the transition probability functions are weakly or set-wise continuous. The reward functions…

In this paper, we study a class of finite-time control problems for discrete-time positive linear systems with time-varying state parameters. Although several interesting control problems appearing in population biology, economics, and…

This paper is concerned with the solution of the optimal stopping problem associated to the valuation of Perpetual American options driven by continuous time Markov chains. We introduce a new dynamic approach for the numerical pricing of…

Our purpose is to study a particular class of optimal stopping problems for Markov processes. We justify the value function convexity and we deduce that there exists a boundary function such that the smallest optimal stopping time is the…

We introduce a Markov-functional approach to construct local volatility models that are calibrated to a discrete set of marginal distributions. The method is inspired by and extends the volatility interpolation of Bass (1983) and Conze and…

Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially…

This work investigates the almost sure stabilization of a class of regime-switching systems based on discrete-time observations of both continuous and discrete components. It develops Shao's work [SIAM J. Control Optim., 55(2017), pp.…