Related papers: Markov control of continuous time Markov processes…
We consider almost upper semi-continuous processes defined on a finite Markov chain. The distributions of the functionals associated with the exit from a finite interval are studied. We also consider some modification of these processes.
A discrete-time method for solving problems in optimal quantum control is presented. Controlling the time discretized markovian dynamics of a quantum system can be reduced to a Markov-decision process. We demonstrate this method in this…
In this research paper, the relationship between finite / countable state space stochastic processes and point processes is explored. Utilizing the known relationship between Poisson processes and continuous time Markov chains, finite /…
In this paper, we study the regularity of the value function associated with a stochastic control problem where two controls act simultaneously on a modulated multidimensional diffusion process. The first is a switching control modelling a…
Markov processes with stochastic resetting towards the origin generically converge towards non-equilibrium steady-states. Long dynamical trajectories can be thus analyzed via the large deviations at Level 2.5 for the joint probability of…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
In this paper, we consider a type of time-changed Markov process, where the time-change is an inverse killed subordinator. This can be seen as an extension of Chen (Chen, Z., Time fractional equations and probabilistic representation, Chaos…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
The Semi-Markov property of Continuous Time Random Walks (CTRWs) and their limit processes is utilized, and the probability distributions of the bivariate Markov process $(X(t),V(t))$ are calculated: $X(t)$ is a CTRW limit and $V(t)$ a…
We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such…
For a class of piecewise deterministic Markov processes, the supports of the invariant measures are characterized. This is based on the analysis of controllability properties of an associated deterministic control system. Its invariant…
We address stability of a class of Markovian discrete-time stochastic hybrid systems. This class of systems is characterized by the state-space of the system being partitioned into a safe or target set and its exterior, and the dynamics of…
This paper proposes an event-triggered parameterized control method using a control Lyapunov function approach for discrete time linear systems with external disturbances. In this control method, each control input to the plant is a linear…
In this paper, we study the controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters.…
A Markov decision problem is called reversible if the stationary controlled Markov chain is reversible under every stationary Markovian strategy. A natural application in which such problems arise is in the control of Metropolis-Hastings…
In this paper, we consider the problem of computing robust controlled invariants for discrete-time monotone dynamical systems. We consider different classes of monotone systems depending on whether the sets of states, control inputs and…
We investigate the problem of optimal control synthesis for Markov Decision Processes (MDPs), addressing both qualitative and quantitative objectives. Specifically, we require the system to satisfy a qualitative task specified by a Linear…
We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their…
In this work, we establish different control design approaches for discrete-time systems, which build upon the notion of finite-step control Lyapunov functions (fs-CLFs). The design approaches are formulated as optimization problems and…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…