Related papers: Markov control of continuous time Markov processes…
We describe a simple method that can be used to sample the rare fluctuations of discrete-time Markov chains. We focus on the case of Markov chains with well-defined steady-state measures, and derive expressions for the large-deviation rate…
We provide a framework for empirical process theory of locally stationary processes using the functional dependence measure. Our results extend known results for stationary Markov chains and mixing sequences by another common possibility to…
We prove that the class of discrete time stationary max-stable process satisfying the Markov property is equal, up to time reversal, to the class of stationary max-autoregressive processes of order $1$. A similar statement is also proved…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be…
We establish the existence of optimal scheduling strategies for time-bounded reachability in continuous-time Markov decision processes, and of co-optimal strategies for continuous-time Markov games. Furthermore, we show that optimal control…
The recent study by B. De Bruyne, S. N. Majumdar, H. Orland and G. Schehr [arXiv:2110.07573], concerning the conditioning of the Brownian motion and of random walks on global dynamical constraints over a finite time-window $T$, is…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…
Markov automata (MAs) extend labelled transition systems with random delays and probabilistic branching. Action-labelled transitions are instantaneous and yield a distribution over states, whereas timed transitions impose a random delay…
We develop a method for computing policies in Markov decision processes with risk-sensitive measures subject to temporal logic constraints. Specifically, we use a particular risk-sensitive measure from cumulative prospect theory, which has…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
The paper proposes an algorithm for a discretization (sampled-time implementation) of a homogeneous control preserving the finite-time and nearly fixed-time stability property of the original (sampling-free) system. The sampling period is…
We study a class of Markov processes with finite state space and continuous time that have product form stationary distributions. We obtain a number of examples that can generate conjectures for diffusions with inert drift.
In this paper we study controlled continuous time random walks (CTRWs) and heuristically derive pay-off function dynamic programming (DP) equations which turn in the limit of standard scaling to fractional Hamilton Jacobi Bellman type…
We give a characterization of the controllability for discrete-time linear systems with convex output constraints. It extends all previously known characterizations in the literature, as well as our previous results on controllability of…
Continuous Time Markov Chains, Hawkes processes and many other interesting processes can be described as solution of stochastic differential equations driven by Poisson measures. Previous works, using the Stein's method, give the…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
Modern control systems must operate in increasingly complex environments subject to safety constraints and input limits, and are often implemented in a hierarchical fashion with different controllers running at multiple time scales. Yet…
We aim at characterizing the asymptotic behavior of value functions in the control of piece-wise deterministic Markov processes (PDMP) of switch type under nonexpansive assumptions. For a particular class of processes inspired by temperate…
The ever increasing complexity of real-time control systems results in significant deviations in the timing of sensing and actuation, which may lead to degraded performance or even instability. In this paper we present a method to analyze…