Related papers: Continuous-Time Markov Decision Processes with Con…

This paper concerns discrete-time infinite-horizon stochastic control systems with Borel state and action spaces and universally measurable policies. We study optimization problems on strategic measures induced by the policies in these…

Switching dynamical systems provide a powerful, interpretable modeling framework for inference in time-series data in, e.g., the natural sciences or engineering applications. Since many areas, such as biology or discrete-event systems, are…

We study reinforcement learning with delayed state observation, where the agent observes the current state after some random number of time steps. We propose an algorithm that combines the augmentation method and the upper confidence bound…

Many decision problems in economics, information technology, and industry can be transformed to an optimal stopping of adapted random vectors with some utility function over the set of Markov times with respect to filtration build by the…

We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov…

In this paper we consider discrete and continuous time risk sensitive optimal stopping problem. Using suitable properties of the underlying Feller-Markov process we prove continuity of the optimal stopping value function and provide formula…

We consider a finite number of $N$ statistically equal agents, each moving on a finite set of states according to a continuous-time Markov Decision Process (MDP). Transition intensities of the agents and generated rewards depend not only on…

We study selective monitors for labelled Markov chains. Monitors observe the outputs that are generated by a Markov chain during its run, with the goal of identifying runs as correct or faulty. A monitor is selective if it skips…

This paper addresses the design of a state observer for networked systems with random delays and dropouts. The model of plant and network covers the cases of multiple sensors, out-of-sequence and buffered measurements. The measurement…

This article considers the average optimality for a continuous-time Markov decision process with Borel state and action spaces and an arbitrarily unbounded nonnegative cost rate. The existence of a deterministic stationary optimal policy is…

Markov jump processes are continuous-time stochastic processes with a wide range of applications in both natural and social sciences. Despite their widespread use, inference in these models is highly non-trivial and typically proceeds via…

We consider mean-field control problems in discrete time with discounted reward, infinite time horizon and compact state and action space. The existence of optimal policies is shown and the limiting mean-field problem is derived when the…

In this paper, we consider stochastic optimal control of Markov Jump Linear Systems with state feedback but without observation of the jumping parameter. The proposed control law is assumed to be linear with constant gains that can be…

Trajectory optimization is a fundamental stochastic optimal control problem. This paper deals with a trajectory optimization approach for dynamical systems subject to measurement noise that can be fitted into linear time-varying stochastic…

In this paper we consider the maximum principle of optimal control for a stochastic control problem. This problem is governed by a system of fully coupled multi-dimensional forward-backward doubly stochastic differential equation with…

For a Markov decision process with countably infinite states, the optimal value may not be achievable in the set of stationary policies. In this paper, we study the existence conditions of an optimal stationary policy in a countable-state…

Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters…

Among various rare events, the effective computation of transition paths connecting metastable states in a stochastic model is an important problem. This paper proposes a stochastic optimal control formulation for transition path problems…

The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…

This paper is concerned with the solution of the optimal stopping problem associated to the valuation of Perpetual American options driven by continuous time Markov chains. We introduce a new dynamic approach for the numerical pricing of…